GitOps step-by-step guide: deploy a CI/CD using CircleCI, ArgoCD and Kubernetes

GitOps step-by-step guide: deploy a CI/CD using CircleCI, ArgoCD and Kubernetes

Nov 18, 2021
GitOps is the framework that automates your infrastructure deployment and maintenance using version control (Git). Along with CI/CD, you can allow developers to focus on the code, and have it tested, deployed automatically on a Kubernetes cluster.
This tutorial will show you how to set up this framework, using CircleCI to manage the Continuous Integration, and ArgoCD for the Continuous Deployment.
For sake of simplicity, we will use a frontend and backend services. I have released sample project that does the basic:
  • gitops-sample: Example that contains all the charts for this project and will be used as the mais GitOps repository for this tutorial
Do not hesitate to fork/clone these projects, or use your own setup.

Set up CircleCI

First, sign up to CircleCI and link your Github account.
Go to the Projects tabs, and set up the two projects that we just created:
notion image
Since we already have on the repo a CircleCI configuration file .circleci/config.yaml, CircleCI can start the build right way. If you do not have it you can check its code here.
notion image
Now that both projects are linked to CircleCI, we need to add the environment variables:
  • DOCKERHUB_USERNAME: Since we are pushing the docker image to a docker Hub (private repo), we need to set the credentials.
  • DOCKERHUB_PASSWORD: Password of your Docker Hub account (we will show you later how to get that token).
  • GITOPS_REPO: (example gitops-sample)
  • GITOPS_REPO_OWNER: (example melalj)
and a SSH key so that we can push changes to the GitOps repo.
You can either do it manually, or programmatically using a CircleCI API.
First, let create a working directory that will be used across this tutorial:
mkdir ./init cd init
Let's now create an SSH key (no passphrase) that will be used to deploy changes to the GitOps repo:
ssh-keygen -t ed25519 -f ./id_rsa_circleci -C "circleci"
Now, we need to set the SSH public key as deploy key for the GitOps repository (with Write access):
notion image
We will now get the personal API Token for CircleCI on your user settings, that will be used to programmatically set the SHH key and environment variables to our projects:
notion image
Keep the value handy as we will use it on the next steps.
All services will be dockerized and pushed into Docker Hub. Let's get a token that will be used to login to your docker account:
notion image
Keep that token saved somewhere (password manager) as we will be using it as well for ArgoCD later on.
Now that we have:
  • SSH Key id_rsa_circleci
  • CircleCi Personal API Token
  • Docker Hub credentials
Let's run this script env.zsh to set the environment variables and SSH key to your repositories:
#!/usr/bin/zsh CIRCLE_TOKEN="<YOUR-CIRCLECI-PERSONAL-API-TOKEN>" PRIVATE_KEY=`cat id_rsa_circleci` GIT_ACCOUNT="<YOUR-GITHUB-USERNAME>" # All repositories that are linked to Circlci GIT_REPOS=( "myproject-api" "myproject-frontend" ) # Since DOCKERHUB_USERNAME="<YOUR-DOCHERHUB-USERNAME>" DOCKERHUB_PASSWORD="<YOUR-DOCHERHUB-PASSWORD>" GITOPS_REPO_OWNER="<YOUR-GITHUB-USERNAME>" # Edit this based on your setup GITOPS_REPO="gitops-sample" # Edit this based on your setup ENV_VARS=( "DOCKERHUB_USERNAME" "DOCKERHUB_PASSWORD" "GITOPS_REPO_OWNER" "GITOPS_REPO" ) for GIT_REPO in "${GIT_REPOS[@]}" do echo $GIT_REPO for ENV_VAR in "${ENV_VARS[@]}" do # Add env var ENV_VAR_VALUE=${(P)${:-${ENV_VAR}}} curl -X POST \ --header "Content-Type: application/json" \ -d "{\"name\":\"${ENV_VAR}\", \"value\":\"${ENV_VAR_VALUE}\"}" \ "${GIT_ACCOUNT}/${GIT_REPO}/envvar?circle-token=${CIRCLE_TOKEN}" done echo "" echo "Added SSH Key" # Add ssh key curl -X POST \ --header "Content-Type: application/json" \ -d "{\"hostname\":\"\", \"private_key\":\"${PRIVATE_KEY}\"}" \ -d PRIVATE_KEY \ "${GIT_ACCOUNT}/${GIT_REPO}/ssh-key?circle-token=${CIRCLE_TOKEN}" sleep 0.3 echo "-" done
One last step is to set in the CircleCI config file on every project which private key should we use to deploy the changes to the GitOps repo:
ssh-keygen -E md5 -lf
notion image
We will use this value in the circleci/config.yaml on every project repository:
version: 2 workflows: version: 2 ci: jobs: - build_test - deploy: requires: - build_test filters: branches: only: - master - staging jobs: build_test: docker: - image: circleci/node:16.13 steps: - checkout - restore_cache: name: Restore Yarn Package Cache keys: - yarn-packages-{{ checksum "yarn.lock" }} - run: name: Install Dependencies command: yarn install - save_cache: name: Save Yarn Package Cache key: yarn-packages-{{ checksum "yarn.lock" }} paths: - ~/.cache/yarn - run: name: Test command: yarn test:all - run: name: Check linting command: yarn run lint - persist_to_workspace: root: . paths: - . deploy: docker: - image: google/cloud-sdk:360.0.0-slim # TODO: Find a better image steps: - add_ssh_keys: fingerprints: - "eb:27:e9:00:a6:87:73:58:ff:af:e0:39:ea:e2:29:04" # EDIT this with the SSH key fingerprint from circleci seettings - attach_workspace: at: . - run: name: Update PATH and Define Environment Variable at Runtime command: | echo 'export REPO_NAME=${CIRCLE_PROJECT_REPONAME,,}' >> $BASH_ENV if [ "$CIRCLE_BRANCH" = 'staging' ]; then echo "export ENV=staging" >> $BASH_ENV echo "export IMAGE_TAG=staging-${CIRCLE_SHA1:0:7}" >> $BASH_ENV else echo 'export ENV=production' >> $BASH_ENV echo "export IMAGE_TAG=production-${CIRCLE_SHA1:0:7}" >> $BASH_ENV fi source $BASH_ENV - run: name: Check required environment variables command: | required_env=( "DOCKERHUB_USERNAME" "DOCKERHUB_PASSWORD" "GITOPS_REPO" "GITOPS_REPO_OWNER" "ENV" "IMAGE_TAG" ) for var in "${required_env[@]}" do if [ -v $var ]; then echo "Found ${var}" else echo "${var} is not set as environement variable in your CI project" exit 1 fi done - setup_remote_docker: version: 19.03.13 - run: name: Login to docker hub command: echo $DOCKERHUB_PASSWORD | docker login -u $DOCKERHUB_USERNAME --password-stdin - run: name: Build Docker image command: docker build -t $DOCKERHUB_USERNAME/${REPO_PREFIX}$REPO_NAME:$IMAGE_TAG . - run: name: Tag Docker image command: docker tag $DOCKERHUB_USERNAME/${REPO_PREFIX}$REPO_NAME:$IMAGE_TAG $DOCKERHUB_USERNAME/${REPO_PREFIX}$REPO_NAME:$IMAGE_TAG - run: name: Push image $IMAGE_TAG to docker registry command: docker push $DOCKERHUB_USERNAME/${REPO_PREFIX}$REPO_NAME:$IMAGE_TAG - run: name: Update infra repo command: | mkdir -p ~/.ssh ssh-keyscan >> ~/.ssh/known_hosts ssh-add -D ssh-add ~/.ssh/id_rsa_eb27e900a6877358ffafe039eae22904 # EDIT this with the SSH key fingerprint from circleci settings (without columns) git clone [email protected]:$GITOPS_REPO_OWNER/$GITOPS_REPO.git cd $GITOPS_REPO git config "[email protected]" git config "CircleCI" sed -i -e "s|tag:.*|tag: \"$IMAGE_TAG\" # $(date -u +'%Y-%m-%dT%H:%M:%SZ') $CIRCLE_BUILD_URL|" ./charts/app/${REPO_NAME}/values-$ENV.yaml sed -i -e "s|$REPO_NAME:$ENV-.*|$REPO_NAME:$IMAGE_TAG # $(date -u +'%Y-%m-%dT%H:%M:%SZ') $CIRCLE_BUILD_URL|" ./charts/tasks/values-$ENV.yaml cat ./charts/app/${REPO_NAME}/values-$ENV.yaml git add -f ./charts/app/${REPO_NAME}/values-$ENV.yaml git add -f ./charts/tasks/values-$ENV.yaml git commit -m "CircleCI update $IMAGE_TAG" git push -u origin master
Make sure, you edit the ssh key fingerprints on both occurrences (add_ssh_keys and on the ssh-add) in your config.yaml file
Now as soon as you push into the master or staging branches, a new build will start, it will push a new docker image to your docker hub, and it will commit the new image tag to your GitOps repository.
You might need to re-trigger the build on CircleCI to be able to push the image to DockerHub and get the proper image tags set on your repo.

Setup ArgoCD

Now that you have your CI set up with CircleCI, we will to set up now ArgoCD.
ArgoCD is an open-source software that deploys a Kubernetes cluster definitions based on a git repository. This helps to keep track on the history changes that have been made to an infrastructure. This also ensure a quick portability between different cloud providers.
The only prerequisite is to have a Kubernetes ready. I made a full article on how to Create a Kubernetes cluster on Hetzner.
First, we need to create a new SSH key so that ArgoCD can read the content of your GitOps repository.
ssh-keygen -t ed25519 -f ./id_rsa_argocd -C "argocd"
Set the SSH public key as deploy key for the Gitops repository (Read only access):
notion image
Let's install argo on your cluster:
kubectl create namespace argocd kubectl apply -n argocd -f
I recommend you download the argo install yaml file and keep it on your GitOps repo (to have fixed version if you need to redeploy your stack
Let's create a secret that contains our ArgoCD private key:
kubectl create secret -n argocd generic argo-secret --from-file=ssh-privatekey=./id_rsa_argocd
Create a yaml file argocd-ssh.yaml to define the SSH key details
apiVersion: v1 kind: ConfigMap metadata: name: argocd-cm namespace: argocd labels: argocd-cm argocd data: repositories: | - name: gitops-sample url: [email protected]:melalj/gitops-sample.git # change this to your gitops repo type: git insecure: true insecureIgnoreHostKey: true sshPrivateKeySecret: name: argo-secret key: ssh-privatekey
Let's apply it to our cluster:
kubectl apply -f ./argocd-ssh.yaml -n argocd
Now let's build our infrastructure.
You can go ahead and clone the repo and get started
git clone [email protected]:melalj/gitops-sample.git cd gitops-sample
The GitOps sample project have the following structure:
  • /charts: We define the infrastructure using Helm charts
    • /app: Contains all your apps Helm charts (myproject-api, myproject-frontend, gsheet-api)
    • /dep: Contains project dependencies (redis). You can clone any chart from the bitnami charts and keep it on your GitOps repo.
    • /projects: Defines your ArgoCD projects (production, staging)
    • /secrets: Contains your app secrets, check the secret.sample.yaml for a sample.
    • /tasks: Contains cronJobs and Jobs that will be run within your cluster.
Edit the charts based on your needs, as the explanation of every chart file is out of the scope of this article. You can learn more about Helm charts here.
First make sure your add your Docker hub credentials into the ignored files:
echo $DOCKERHUB_EMAIL > ./init/id_dockerhub_email echo $DOCKERHUB_USERNAME > ./init/id_dockerhub_username echo $DOCKERHUB_PASSWORD > ./init/id_dockerhub_password
Now you can run the installation of your charts:
Now that your projects are setup on ArgoCD, we can access it on the dashboard:
kubectl port-forward svc/argocd-server -n argocd 8000:80
The default username is admin and you can get the password using the following command:
(kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d) && echo ""
When you log in you can see that all services are up and running!
notion image
To access to the external IP for your frontend you can run
kubectl get svc -n production # or staging to get the staging environment
notion image
When you access that IP, you will reach the frontend site deployed:
notion image
The sample project is a frontend that proxy the backend on /api – for every visit we display the last date when the endpoint was hit, and save the current date in redis.
You can link that IP to a domain name with Cloudflare for example.


Now, we can say that you have a Continuous Integration and Deployment on your stack.
As soon as some code is pushed on the master or staging CircleCI will run the tests, build the docker image, push it to the docker registry, and update the image tag to the GitOps repository.
While ArgoCD will listen to changes on your GitOps repository and apply the changes to your Kubernetes infrastructure. All done on the background automatically, so that you can focus on your code. This of course can be overkill for simple apps, but can be extended much easily by having such structure if your app scales.

This article is brought you by tonoïd – we are a micro-startup studio building small businesses that are profitable and solve a specific problem without any external funding nor billion-dollar market-size. Most notably, RefurbMe, a comparison site for refurbished products – and Notion Automations.
If you have any feedback, do not hesitate to reach out at