GitOps step-by-step guide: deploy a CI/CD using CircleCI, ArgoCD and Kubernetes

GitOps step-by-step guide: deploy a CI/CD using CircleCI, ArgoCD and Kubernetes

Created
Nov 18, 2021
Tags
Tech
GitOps is the framework that automates your infrastructure deployment and maintenance using version control (Git). Along with CI/CD, you can allow developers to focus on the code, and have it tested, deployed automatically on a Kubernetes cluster.
This tutorial will show you how to set up this framework, using CircleCI to manage the Continuous Integration, and ArgoCD for the Continuous Deployment.
For sake of simplicity, we will use a frontend and backend services. I have released sample project that does the basic:
  • gitops-sample: Example that contains all the charts for this project and will be used as the mais GitOps repository for this tutorial
Do not hesitate to fork/clone these projects, or use your own setup.

Set up CircleCI

First, sign up to CircleCI and link your Github account.
Go to the Projects tabs, and set up the two projects that we just created:
notion image
Since we already have on the repo a CircleCI configuration file .circleci/config.yaml, CircleCI can start the build right way. If you do not have it you can check its code here.
notion image
Now that both projects are linked to CircleCI, we need to add the environment variables:
  • DOCKERHUB_USERNAME: Since we are pushing the docker image to a docker Hub (private repo), we need to set the credentials.
  • DOCKERHUB_PASSWORD: Password of your Docker Hub account (we will show you later how to get that token).
  • GITOPS_REPO: (example gitops-sample)
  • GITOPS_REPO_OWNER: (example melalj)
and a SSH key so that we can push changes to the GitOps repo.
You can either do it manually, or programmatically using a CircleCI API.
First, let create a working directory that will be used across this tutorial:
Let's now create an SSH key (no passphrase) that will be used to deploy changes to the GitOps repo:
Now, we need to set the SSH public key as deploy key for the GitOps repository (with Write access):
notion image
 
We will now get the personal API Token for CircleCI on your user settings, that will be used to programmatically set the SHH key and environment variables to our projects:
notion image
Keep the value handy as we will use it on the next steps.
All services will be dockerized and pushed into Docker Hub. Let's get a token that will be used to login to your docker account:
notion image
Keep that token saved somewhere (password manager) as we will be using it as well for ArgoCD later on.
Now that we have:
  • SSH Key id_rsa_circleci
  • CircleCi Personal API Token
  • Docker Hub credentials
Let's run this script env.zsh to set the environment variables and SSH key to your repositories:
One last step is to set in the CircleCI config file on every project which private key should we use to deploy the changes to the GitOps repo:
notion image
We will use this value in the circleci/config.yaml on every project repository:
⚠️
Make sure, you edit the ssh key fingerprints on both occurrences (add_ssh_keys and on the ssh-add) in your config.yaml file
Now as soon as you push into the master or staging branches, a new build will start, it will push a new docker image to your docker hub, and it will commit the new image tag to your GitOps repository.
You might need to re-trigger the build on CircleCI to be able to push the image to DockerHub and get the proper image tags set on your repo.

Setup ArgoCD

Now that you have your CI set up with CircleCI, we will to set up now ArgoCD.
ArgoCD is an open-source software that deploys a Kubernetes cluster definitions based on a git repository. This helps to keep track on the history changes that have been made to an infrastructure. This also ensure a quick portability between different cloud providers.
The only prerequisite is to have a Kubernetes ready. I made a full article on how to Create a Kubernetes cluster on Hetzner.
First, we need to create a new SSH key so that ArgoCD can read the content of your GitOps repository.
Set the SSH public key as deploy key for the Gitops repository (Read only access):
notion image
 
Let's install argo on your cluster:
💡
I recommend you download the argo install yaml file and keep it on your GitOps repo (to have fixed version if you need to redeploy your stack
Let's create a secret that contains our ArgoCD private key:
Create a yaml file argocd-ssh.yaml to define the SSH key details
Let's apply it to our cluster:
Now let's build our infrastructure.
You can go ahead and clone the repo and get started
The GitOps sample project have the following structure:
  • /charts: We define the infrastructure using Helm charts
    • /app: Contains all your apps Helm charts (myproject-api, myproject-frontend, gsheet-api)
    • /dep: Contains project dependencies (redis). You can clone any chart from the bitnami charts and keep it on your GitOps repo.
    • /projects: Defines your ArgoCD projects (production, staging)
    • /secrets: Contains your app secrets, check the secret.sample.yaml for a sample.
    • /tasks: Contains cronJobs and Jobs that will be run within your cluster.
Edit the charts based on your needs, as the explanation of every chart file is out of the scope of this article. You can learn more about Helm charts here.
First make sure your add your Docker hub credentials into the ignored files:
Now you can run the installation of your charts:
Now that your projects are setup on ArgoCD, we can access it on the dashboard:
The default username is admin and you can get the password using the following command:
When you log in you can see that all services are up and running!
notion image
To access to the external IP for your frontend you can run
notion image
When you access that IP, you will reach the frontend site deployed:
notion image
The sample project is a frontend that proxy the backend on /api – for every visit we display the last date when the endpoint was hit, and save the current date in redis.
You can link that IP to a domain name with Cloudflare for example.

Conclusion

Now, we can say that you have a Continuous Integration and Deployment on your stack.
As soon as some code is pushed on the master or staging CircleCI will run the tests, build the docker image, push it to the docker registry, and update the image tag to the GitOps repository.
While ArgoCD will listen to changes on your GitOps repository and apply the changes to your Kubernetes infrastructure. All done on the background automatically, so that you can focus on your code. This of course can be overkill for simple apps, but can be extended much easily by having such structure if your app scales.

This article is brought you by tonoïd – we are a micro-startup studio building small businesses that are profitable and solve a specific problem without any external funding nor billion-dollar market-size. Most notably, RefurbMe, a comparison site for refurbished products – and Notion Automations.
If you have any feedback, do not hesitate to reach out at
✉️
Contact