Kelvin Tay

enjoys technical writing, and a cheeky drink 🥃


I passed the HashiCorp Certified: Terraform Associate 002 exam on Jan 14th 2023. This was my first try, fortunately.

In terms of my Terraform experience prior to the exams, I have yet to build any Terraform project at a production level. My recent exposure was mainly around trying to play around with CircleCI’s Nomad client Terraform setup.


In my past career, I have used loads of AWS CloudFormation. Yes, I have had some battle scars re: IaC. I always wanted to learn more about Terraform, given that it supports more than 1 cloud platform.

In particular, I am interested in contributing a Terraform provider for CircleCI. This means creating a provider/plugin for creating CircleCI webhooks / scheduled pipelines via Terraform for example.

This above may likely be a lofty ambition, but I thought I’d hedge this by also trying for the certification. Kill two birds in one quarter, as they say.

Breakdown of Score

Content Area Score (%)
Understand infrastructure as code (IaC) concepts 100
Understand Terraform’s purpose (vs other IaC) 100
Understand Terraform basics 85
Use the Terraform CLI (outside of core workflow) 100
Interact with Terraform modules 50
Navigate Terraform Workflow 100
Implement and maintain state 75
Read, generate and modify configuration 72
Understand Terraform Cloud and Enterprise capabilities 100

Overall, I scored 84% (70% required).

Word of Advice

If you are not using Terraform on a daily basis, practice Terraform CLI commands definitely!

The exam has a few questions where they may test you to type the code / command required. See sample questions here.

I did have a “doh!” moment, since it wasn’t clear how “complete“ the answer should be.

In particular, one question I got was: How would you reference the vpc_id from this module in the code snippet?

To reference the module output, it would be module.aws_vpc.vpc_id for example. I think the full answer should be: vpc_id = module.aws_vpc.vpc_id.

I had only typed module.aws_vpc.vpc_id due to lack of clarity in the question. I think I likely lost points there!

In terms of studying for it, I highly recommend the following Udemy course + practice exam!

Lastly, there will be a newer exam version (003). I understand this update would remove older questions referring to already-deprecated commands (e.g., taint). So if you can wait, do go for the 003 version instead!

Final Thoughts

It was a fun exam to take, for sure!

Given that it is really about a tool and its ecosystem, I’d highly encourage folks to try out Terraform while practicing for the exam.

Your mileage on the Terraform tool will go a long way if you practice. This way, the exam questions will make way more sense too, as you practice.

buy Kelvin a cup of coffee

I am here in Taipei for 2 weeks

I survived a 17% reduction (layoff) at CircleCI last week. As soon as the CEO letter was published, many recruiters reached out.

I can understand that. However, one email irked me bad, that I had to respond in curt.

Without naming the company and the actual person, here is the snippet in raw form:

We learned about the CircleCI lay-offs on Linkedin via your post and [redacted]'s post. It looks like you are not affected, which is great news, but with such incident striking instability for a previously believed to be COVID-proof and recession-proof business, would it have spiked some of your interests to hear out available jobs in the market?

So feel free to refer us [redacted], any other colleague you may know in CircleCI affected by the lay-offs, or anyone in Japan open for a job change in Software Engineering.

In these times, we see many tech companies reducing their workforce. These are sudden events for people affected; No one wants to be in such positions. As a recruiter, please please please give your words some more thought before pressing “send”.

If your team is looking for amazing engineers and people in Japan and beyond, please check out this link of our CircleCI alumni:

buy Kelvin a cup of coffee

Infrastructure as Code (IaC) is not a new concept for many. Prior to joining CircleCI, I have been wrangling with AWS CloudFormation long enough (2 years) to bear some of its pain-points.

Since joining CircleCI, I have been exposed to Terraform through internal and external projects. So far, I really like that Terraform is not tied to specific cloud providers.

I also like that any service can contribute a Terraform provider, thereby allowing users to define your service's resources as code. (Any cloud resource that exposes an CRUD / RESTful API is a good candidates for Terraform.)

Recently, I have been wishing many things to be “Terraform-able”.

As a support engineer, I use Zendesk daily. We create Zendesk macros for canned responses in particular. Overtime, these macros grow, and it may be hard to manage or validate changes to our macros. I do wish Zendesk has a Terraform provider, so we can institute our macros in code.

Recently, I also wished that I could Terraform my resume. Then, I snapped out of it.

When you have a powerful hammer like Terraform, everything looks like nails.

#terraform #infrastructureascode #declarative

buy Kelvin a cup of coffee

man on bicycle


This post reflects my opinions on how I decide to continue staying at the current job, or consider switching.

As such, this is purely from my perspective. You would likely have your own principles and outlook on life and career; If anything, I hope this post motivates you to reflect, rethink, and refresh.

For context, I work a full-time job with no additional side gigs or secondary jobs.

My Guiding Principles

I use the following 3 factors to decide if I should continue with the current job:

  1. Compensation
  2. Job Fulfillment
  3. Work-Life Balance

You likely have your own principles, so you can replace this list with your own!

I find that realistically, most jobs should hit 2 out of the 3 factors at least.

When I find myself seriously contemplating a career change, I like to check this list. If I evaluate that the current opportunity hits only 1 factor (i.e., score of 1/3), it is a good indication that I can do better.

1. Compensation

Without mincing my words, this is just about the main salary. I am not swayed by or overly concerned with stock options, or welfare benefits.

If you are working in Japan, I do encourage you to check out This can help you get some idea about what others with similar tech backgrounds are compensated.

2. Job Fulfillment

This can a tough one to judge or measure.

You can go all logical about it (i.e., making a list of the positives and negatives).

Or, you can also simply base it on gut feeling (e.g., say 8 out of 10).

For me, I tend to find my answer by asking myself:

“If someone asks me what I do for a living, how enthusiastic would I be about introducing my current job?”

In terms of motivation, I like to measure how I feel when I get up on a Monday morning.

I like to point out that a good team can also offer so much support at work. Your motivation and fulfillment can be influenced greatly by your colleagues.

3. Work-Life Balance

I like to think that we spend half of our waking hours working. That is, 8 out of 16 hours trading our skills for compensation.

You may work longer than 8 hours on some days due to circumstances. However, if overwork becomes consistent, something is not healthy with your current environment.

Realistically, not everyone is built to grind out 10+ hours everyday. This is not sustainable for yourself, and your team.

You also likely have aspirations outside of work, hobbies you want to pursue, and family members you want to spend time with.

Ensure that you have time for them.

Final Thoughts

This is just a general guide, based on my perspectives.

That said, I understand there can be inertia to “go”; It feels easier to stay, naturally. I have left out any discussions around evaluating the other opportunities available. This is purely focused on evaluating your current job.

My (check)list describes my definition of what work means to me. I would encourage you to think what work means for you and your family.

Happy to connect, and talk about this over coffee if you are in Tokyo!

#career #motivation #principles

buy Kelvin a cup of coffee

Today's communication tools allows us to include links in our writings easily in many ways.

This sentence includes a link within itself, but does not stop you from reading the entire message.

I find such links useful when the linked article is supplementary to your message.

In situations where I want to direct the reader to click on the link explicitly, I find it best to simply show them raw below:

Personally, I find that showing the link raw (as-is) at the end helps:

  1. draw attention, especially if the main point is about the linked article
  2. to allow readers to inspect the link URL, which can help in credibility too

buy Kelvin a cup of coffee


In a CircleCI pipeline, workflows run independent of one another. As such, there is no built-in feature to ensure workflow B runs after workflow A.

However, you can still achieve ordering, through some trickery.

💡 You can control the sequence of jobs within a workflow. I recommend you consider if you can combine/merge workflow B into workflow A itself, first. This article is for when you require separate workflows somehow.


To achieve ordering, we simply set an approval job as the first job for workflow B.

# contrived snippet of a .circleci/config.yaml

      - one
      - two
      - start:
          type: approval
      - next:
            - start

Subsequent jobs in workflow B will only run when the approval job is approved. As such, you can “force” a wait, and only approve this job when workflow A is completed.

Note that this requires manual intervention, of course.

However, a benefit in this approach is that your team can take the time to confirm the outcomes of workflow A. For example, workflow A has deployed some infrastructure changes (e.g., terraform apply), and you prefer inspecting these changes before running workflow B.

One Step Further

You can automate this approval, at the end of workflow A, via the Approve a job API.

Specifically, you would need to create a job that does the following:

  1. Find workflow B's ID from the current pipeline.
  2. Find the approval job's ID from the invoked workflow B.
  3. Approve the job.
        type: string
        description: workflow name
        type: string
        description: name of approval job in workflow
      - image: cimg/base:current
      - run:
          name: Find Workflow ID for << parameters.workflow-name >>
          command: |
            curl -H "Circle-Token: $CIRCLE_TOKEN"<< >>/workflow > workflows.json
            WORKFLOW_ID=$(jq -r '.items | map(select(.name == "<< parameters.workflow-name >>")) | .[0].id' workflows.json)
            echo "export WORKFLOW_ID='${WORKFLOW_ID}'" >> $BASH_ENV
      - run:
          name: Find Job ID for << parameters.job-name >>
          command: |
            curl -H "Circle-Token: $CIRCLE_TOKEN" "${WORKFLOW_ID}/job" > jobs.json
            APPROVAL_JOB_ID=$(jq -r '.items | map(select(.name == "<< parameters.job-name >>" and .type == "approval")) | .[0].id' jobs.json)
            echo "export APPROVAL_JOB_ID='${APPROVAL_JOB_ID}'" >> $BASH_ENV
      - run:
          name: Approve job
          command: |
            curl -X POST -H "Circle-Token: $CIRCLE_TOKEN" "${WORKFLOW_ID}/approve/${APPROVAL_JOB_ID}" | jq .

In the spirit of sharing, I have created a CircleCI Orb that codifies the above job for your convenience.

I hope this article and the Orb will be useful. Keep on building, folks!

#circleci #cicd #workflow

buy Kelvin a cup of coffee

I have built a quick tool (in Python) that calculates the time taken for agents to follow up on a customer ticket on Zendesk.

If this is also useful for your support team, feel free to copy from the source code here.


As a senior support engineer, I follow up with customers who left a negative review on their support experience.

We noted that some dissatisfied customers felt that they waited too long for a response. However, this is usually due to genuine misunderstandings and expectations around the response time SLAs for their support plan.


To make it easy for customers and ourselves to confirm our response times within a ticket, I built this tool that:

  • calculates the time taken for the first agent response to each comment from the customer
  • provides the arithmetic for proof
  • includes information of when weekends were involved

In this tool, I elected to keep the calculation of response times agnostic to the specific plan of the customer; It merely calculates the time taken between a customer comment and an agent's response.

I find this useful, since Support plans (and their promised SLAs) may change overtime. Keeping the tool really simple allows anyone to build on top of the output (CSV file), for flexibility.

Please feel free to copy as needed. 🍺

buy Kelvin a cup of coffee


Before Diving in

This is an attempt to explain and explore how teams can use Docker Buildx for delivering Docker images.

Since we will not be covering all features around Docker Buildx, this is a wide snorkel rather than a deep dive.

This is a quick article for developers who have yet to use Docker Buildx but are curious on its use-cases.

What is Docker Buildx?

Let's take a few steps back before plunging in.

We use Docker Build to build Docker images from Dockerfiles.

Since 18.09, BuildKit was introduced as an improved version of the previous builder. As an example, we can mount secrets when building our images with BuildKit. BuildKit will also ensure that these secrets are not exposed within the built image's layers.

Buildx builds (no pun intended) on top of BuildKit. It comes with more operations besides image-building, as you can see from its available commands. Importantly, Buildx provides features for caching and cross-platform image builds.

Why should we use Docker Buildx?

For software teams shipping Docker images often, Docker Buildx can be an important tool in the box.

Caching image layers ensure a next rebuild of the image will be faster.

Before, teams would need various machines on different platforms to build images for each platform. For example, we would need a ARM64 machine to build a Docker image for ARM64 architectures.

With Docker Buildx's cross-platform feature, we can now use the same AMD64 machine to build both AMD64 and ARM64 Docker images.

Why is it relevant in CI/CD?

Many teams are building Docker images as part of their CI/CD pipelines. Hence, they can lean on the build cache and cross-platform capabilities of Docker Buildx to build various images faster and cheaper.

Let's discuss the two mentioned features a little deeper.


This pertains to the cache-from and cache-to options with the docker buildx build command.

Docker Buildx allows you to choose your caching strategy (e.g., inline, local, registry and etc), and each comes with its pros and cons.

Your choice will depend largely on your team's philosophy and the CI/CD provider.

For example, you can leverage GitHub's Cache service when running Docker Buildx on GitHub Actions.

For CircleCI users, you may find my exploratory project here useful.


When building an ARM64 Docker image on a CI/CD pipeline, you would need to do so on an ARM64-based machine runner then (if not using Buildx).

Depending on your CI/CD provider, there may not be ARM64 support.

This can be worked around, if your CI/CD provider allows you to “bring you own runners” (also known as self-hosted runners). GitHub Actions and CircleCI support self-hosted runners. However, it does mean someone in your team now has to manage these runners on your infrastructure.

With Docker Buildx, we can now build cross-platform images within any arbitrary machine runner.

This can be a big win for team that prefers not owning additional infrastructures.

Resurfacing to Shore

We have explored the appeal of Docker Buildx, particularly in a CI/CD context here. As mentioned, it is ultimately a tool. For teams building Docker images in their CI/CD pipelines, I do encourage you to look into Docker Buildx if you have not!

#docker #buildx #cicd #performance

buy Kelvin a cup of coffee

This is an attempt to explain CircleCI's Docker Layer Cache (DLC) feature in simple terms.

What is Docker Layer Cache

Docker creates an image via layers. Each layer is created via Dockerfile commands like RUN, COPY and ADD, and they represent a unit of filesystem change.

When Docker builds an image, the built layers are stored. Rebuilding the same image becomes faster because Docker can now retrieve the stored layers as cache. This is known as layer caching on Docker, or Docker Layer Cache (DLC).

DLC is useful when we are rebuilding the same layers. It works well when we build images on the same machine; The stored layers are still there when we build a second time.

However, this becomes tricky on CI/CD systems, since most CI/CD pipelines run on ephemeral environments. The execution environment is destroyed once your pipeline completes.

How does CircleCI implement DLC

For DLC to work on CI/CD systems, we need to ensure stored/cached layers can be shared between CI/CD pipelines.

To achieve this, CircleCI assigns an external volume (store) to a job when we enable DLC. Think of it like passing around an external hard drive amongst friends.

This external volume will store the built Docker images' layers in a job. When CircleCI assigns same external volume to the next job, we should see faster image builds since the cached layers can be found.

An external volume can be assigned only to one job at a time. Hence, a job may see cache misses when building images due to a different volume being assigned.

A cache miss will mean a longer time taken to build the Docker image, which translates to a longer job duration. However, it does not and should not cause a job on CircleCI to fail.

#docker #dockerlayercache #circleci #explanation

buy Kelvin a cup of coffee

Enter your email to subscribe to updates.