How I added CircleCI Runner support for Terraform provider
I have released support for the CircleCI Runner resource-class and token in my unofficial Terraform provider for CircleCI, as per v0.10.3.
Developers can now manage the provisioning (and teardown) of CircleCI self-hosted Runners within Terraform. You can explore an example here.
This was a fun challenge, and I wanted to document my journey on this work.
Investigation
Unlike other resources, self-hosted runners are not manageable under the official V2 API. Developers had to use the CircleCI CLI to manage resource-classes and tokens instead.
To port this into my Terraform provider, I was hoping there was a HTTP API available. This way, I can continue using my approach of abstracting the HTTP API away to a Go SDK.
I assumed (wrongly) the CLI was using GraphQL under the hood for Runner operations, as with many others (e.g., for Orbs).
Digging into the source-code, I then realized Runner resource-classes and tokens can be managed via a HTTP API; It was simply not publicly documented, yet.
Glue
After the legwork mentioned above, I tested the HTTP APIs with an OpenAPI (Swagger 2.0) document. This enabled me to generate a Go SDK for Runner APIs.
Why are the Go SDKs separated? Wouldn't it be easier to keep it all in one?
That was something I mulled over for some time indeed. I have documented my reasons for keeping them separate.
Assembly
With the Go SDK published, I “simply” have to then expose the Runner resource-classes and tokens in the Terraform provider codebase.
The main work was done within a pull request here. This also included acceptance tests, and examples.
Ultimately, I noted self-hosted (machine) runners are also available for CircleCI's Server customers (i.e., self-hosted CircleCI). We would want to extend and ensure this addition can be used by platform teams using CircleCI Server.
It turned out that the Runner API:
- is via https://runner.circleci.com for CircleCI Cloud
- is via https://YOUR-SELF-HOSTED-DOMAIN for CircleCI Server
Thankfully, this was a quick patch. I was also able to verify this fix against my own CircleCI Server instance.
Things I learnt
This feature was satisfying for me to build, and I had many learning points along the journey.
Read the code: This feature would not have been completed if I did not dig deeper into the publicly-available source code 📖
Keep trying: I am still learning (and failing) at Go. However, I think it is important to keep trying and learning. Keeping this source code open-sourced forces me to keep myself honest too about my lack of knowledge. For fellow engineers out there, let's keep at it! 🤓