I have released support for the CircleCI Runner resource-class and token in my unofficial Terraform provider for CircleCI, as per v0.10.3.

Developers can now manage the provisioning (and teardown) of CircleCI self-hosted Runners within Terraform. You can explore an example here.

This was a fun challenge, and I wanted to document my journey on this work.

Investigation

Unlike other resources, self-hosted runners are not manageable under the official V2 API. Developers had to use the CircleCI CLI to manage resource-classes and tokens instead.

To port this into my Terraform provider, I was hoping there was a HTTP API available. This way, I can continue using my approach of abstracting the HTTP API away to a Go SDK.

I assumed (wrongly) the CLI was using GraphQL under the hood for Runner operations, as with many others (e.g., for Orbs).

Digging into the source-code, I then realized Runner resource-classes and tokens can be managed via a HTTP API; It was simply not publicly documented, yet.

Glue

After the legwork mentioned above, I tested the HTTP APIs with an OpenAPI (Swagger 2.0) document. This enabled me to generate a Go SDK for Runner APIs.

Why are the Go SDKs separated? Wouldn't it be easier to keep it all in one?

That was something I mulled over for some time indeed. I have documented my reasons for keeping them separate.

Assembly

With the Go SDK published, I “simply” have to then expose the Runner resource-classes and tokens in the Terraform provider codebase.

The main work was done within a pull request here. This also included acceptance tests, and examples.

Ultimately, I noted self-hosted (machine) runners are also available for CircleCI's Server customers (i.e., self-hosted CircleCI). We would want to extend and ensure this addition can be used by platform teams using CircleCI Server.

It turned out that the Runner API:

• is via https://runner.circleci.com for CircleCI Cloud
• is via https://YOUR-SELF-HOSTED-DOMAIN for CircleCI Server

Thankfully, this was a quick patch. I was also able to verify this fix against my own CircleCI Server instance.

Things I learnt

This feature was satisfying for me to build, and I had many learning points along the journey.

1. Read the code: This feature would not have been completed if I did not dig deeper into the publicly-available source code 📖

2. Keep trying: I am still learning (and failing) at Go. However, I think it is important to keep trying and learning. Keeping this source code open-sourced forces me to keep myself honest too about my lack of knowledge. For fellow engineers out there, let's keep at it! 🤓

#terraform #circleci #runner #go

buy Kelvin a cup of coffee ☕

Preface

In a CircleCI pipeline, workflows run independent of one another. As such, there is no built-in feature to ensure workflow B runs after workflow A.

However, you can still achieve ordering, through some trickery.

💡 You can control the sequence of jobs within a workflow. I recommend you consider if you can combine/merge workflow B into workflow A itself, first. This article is for when you require separate workflows somehow.

How

To achieve ordering, we simply set an approval job as the first job for workflow B.

# contrived snippet of a .circleci/config.yaml

workflows:
  aaa:
    jobs:
      - one
      - two
  bbb:
    jobs:
      - start:
          type: approval
      - next:
          requires:
            - start

Subsequent jobs in workflow B will only run when the approval job is approved. As such, you can “force” a wait, and only approve this job when workflow A is completed.

Note that this requires manual intervention, of course.

However, a benefit in this approach is that your team can take the time to confirm the outcomes of workflow A. For example, workflow A has deployed some infrastructure changes (e.g., terraform apply), and you prefer inspecting these changes before running workflow B.

One Step Further

You can automate this approval, at the end of workflow A, via the Approve a job API.

Specifically, you would need to create a job that does the following:

1. Find workflow B's ID from the current pipeline.
2. Find the approval job's ID from the invoked workflow B.
3. Approve the job.

jobs:
  ...
  approve-workflow:
    parameters:
      workflow-name:
        type: string
        description: workflow name
      job-name:
        type: string
        description: name of approval job in workflow
    docker:
      - image: cimg/base:current
    steps:
      - run:
          name: Find Workflow ID for << parameters.workflow-name >>
          command: |
            curl -H "Circle-Token: $CIRCLE_TOKEN" https://circleci.com/api/v2/pipeline/<< pipeline.id >>/workflow > workflows.json
            WORKFLOW_ID=$(jq -r '.items | map(select(.name == "<< parameters.workflow-name >>")) | .[0].id' workflows.json)
            echo "export WORKFLOW_ID='${WORKFLOW_ID}'" >> $BASH_ENV
      - run:
          name: Find Job ID for << parameters.job-name >>
          command: |
            curl -H "Circle-Token: $CIRCLE_TOKEN" "https://circleci.com/api/v2/workflow/${WORKFLOW_ID}/job" > jobs.json
            APPROVAL_JOB_ID=$(jq -r '.items | map(select(.name == "<< parameters.job-name >>" and .type == "approval")) | .[0].id' jobs.json)
            echo "export APPROVAL_JOB_ID='${APPROVAL_JOB_ID}'" >> $BASH_ENV
      - run:
          name: Approve job
          command: |
            curl -X POST -H "Circle-Token: $CIRCLE_TOKEN" "https://circleci.com/api/v2/workflow/${WORKFLOW_ID}/approve/${APPROVAL_JOB_ID}" | jq .

In the spirit of sharing, I have created a CircleCI Orb that codifies the above job for your convenience.

https://circleci.com/developer/orbs/orb/kelvintaywl/control-flow

I hope this article and the Orb will be useful. Keep on building, folks!

#circleci #cicd #workflow

buy Kelvin a cup of coffee ☕

This is an attempt to explain CircleCI's Docker Layer Cache (DLC) feature in simple terms.

What is Docker Layer Cache

Docker creates an image via layers. Each layer is created via Dockerfile commands like RUN, COPY and ADD, and they represent a unit of filesystem change.

When Docker builds an image, the built layers are stored. Rebuilding the same image becomes faster because Docker can now retrieve the stored layers as cache. This is known as layer caching on Docker, or Docker Layer Cache (DLC).

DLC is useful when we are rebuilding the same layers. It works well when we build images on the same machine; The stored layers are still there when we build a second time.

However, this becomes tricky on CI/CD systems, since most CI/CD pipelines run on ephemeral environments. The execution environment is destroyed once your pipeline completes.

How does CircleCI implement DLC

For DLC to work on CI/CD systems, we need to ensure stored/cached layers can be shared between CI/CD pipelines.

To achieve this, CircleCI assigns an external volume (store) to a job when we enable DLC. Think of it like passing around an external hard drive amongst friends.

This external volume will store the built Docker images' layers in a job. When CircleCI assigns same external volume to the next job, we should see faster image builds since the cached layers can be found.

An external volume can be assigned only to one job at a time. Hence, a job may see cache misses when building images due to a different volume being assigned.

A cache miss will mean a longer time taken to build the Docker image, which translates to a longer job duration. However, it does not and should not cause a job on CircleCI to fail.

#docker #dockerlayercache #circleci #explanation

buy Kelvin a cup of coffee ☕