How to: Create Sequential Workflows in CircleCI pipeline
Preface
In a CircleCI pipeline, workflows run independent of one another. As such, there is no built-in feature to ensure workflow B runs after workflow A.
However, you can still achieve ordering, through some trickery.
💡 You can control the sequence of jobs within a workflow. I recommend you consider if you can combine/merge workflow B into workflow A itself, first. This article is for when you require separate workflows somehow.
How
To achieve ordering, we simply set an approval job as the first job for workflow B.
# contrived snippet of a .circleci/config.yaml
workflows:
aaa:
jobs:
- one
- two
bbb:
jobs:
- start:
type: approval
- next:
requires:
- start
Subsequent jobs in workflow B will only run when the approval job is approved. As such, you can “force” a wait, and only approve this job when workflow A is completed.
Note that this requires manual intervention, of course.
However, a benefit in this approach is that your team can take the time to confirm the outcomes of workflow A.
For example, workflow A has deployed some infrastructure changes (e.g., terraform apply
), and you prefer inspecting these changes before running workflow B.
One Step Further
You can automate this approval, at the end of workflow A, via the Approve a job API.
Specifically, you would need to create a job that does the following:
- Find workflow B's ID from the current pipeline.
- Find the approval job's ID from the invoked workflow B.
- Approve the job.
jobs:
...
approve-workflow:
parameters:
workflow-name:
type: string
description: workflow name
job-name:
type: string
description: name of approval job in workflow
docker:
- image: cimg/base:current
steps:
- run:
name: Find Workflow ID for << parameters.workflow-name >>
command: |
curl -H "Circle-Token: $CIRCLE_TOKEN" https://circleci.com/api/v2/pipeline/<< pipeline.id >>/workflow > workflows.json
WORKFLOW_ID=$(jq -r '.items | map(select(.name == "<< parameters.workflow-name >>")) | .[0].id' workflows.json)
echo "export WORKFLOW_ID='${WORKFLOW_ID}'" >> $BASH_ENV
- run:
name: Find Job ID for << parameters.job-name >>
command: |
curl -H "Circle-Token: $CIRCLE_TOKEN" "https://circleci.com/api/v2/workflow/${WORKFLOW_ID}/job" > jobs.json
APPROVAL_JOB_ID=$(jq -r '.items | map(select(.name == "<< parameters.job-name >>" and .type == "approval")) | .[0].id' jobs.json)
echo "export APPROVAL_JOB_ID='${APPROVAL_JOB_ID}'" >> $BASH_ENV
- run:
name: Approve job
command: |
curl -X POST -H "Circle-Token: $CIRCLE_TOKEN" "https://circleci.com/api/v2/workflow/${WORKFLOW_ID}/approve/${APPROVAL_JOB_ID}" | jq .
In the spirit of sharing, I have created a CircleCI Orb that codifies the above job for your convenience.
https://circleci.com/developer/orbs/orb/kelvintaywl/control-flow
I hope this article and the Orb will be useful. Keep on building, folks!