🔄Step-by-Step Guide to Building a GitOps CI/CD Pipeline with GitHub Actions🤖
Introduction
In the world of software development, the need for efficient, reliable, and scalable deployment strategies has never been more pressing. Enter GitOps, a paradigm-shifting approach that integrates the familiar Git version control system with operational workflows, bringing unparalleled transparency, consistency, and productivity to the CI/CD (Continuous Integration/Continuous Deployment) pipeline. This article dives into the practicalities of implementing a GitOps-centric CI/CD pipeline using GitHub Actions, a powerful automation tool that transforms the way developers build, test, and deploy software.
Our journey will explore the intricacies of setting up a pipeline that not only manages the building and pushing of Docker images but also intricately weaves in the updating of Helm charts, the robustness of Python-based smoke tests, and the crucial capability of performing rollbacks when these tests fail. This holistic approach ensures that every aspect of the deployment process is covered, from the initial code push to the final deployment in a production environment.
Whether you’re a seasoned DevOps professional or new to the concept of GitOps, this guide aims to equip you with the knowledge and skills to construct a CI/CD pipeline that epitomizes efficiency, reliability, and scalability. So, let’s embark on this journey to transform your deployment strategies and embrace the full potential of GitOps with GitHub Actions.
Prerequisites
Before diving into the details of implementing a GitOps CI/CD pipeline with GitHub Actions, there are certain prerequisites that you should have in place to fully benefit from this guide. These prerequisites ensure that you have the necessary foundation and tools to effectively follow along and implement the strategies discussed in the article.
1. Kubernetes Cluster Setup: You should have a working Kubernetes cluster. This cluster is where your applications will be deployed and managed. If you’re new to Kubernetes, it’s advisable to familiarize yourself with its basic concepts and functionalities.
2. Continuous Deployment (CD) Tool: A CD tool like ArgoCD should be installed in your Kubernetes cluster. ArgoCD is pivotal in this context as it facilitates the automatic watching of the Helm chart repository, enabling seamless deployment updates based on changes in your Git repository.
3. Familiarity with Python: A basic understanding of Python programming is required, as the article will cover writing Python scripts for smoke tests. These tests are an essential part of the CI/CD pipeline, ensuring that the deployed applications run as expected.
4. Experience with GitHub Actions: You should have a basic understanding of GitHub Actions, GitHub’s automation tool. This will be used for setting up the CI/CD pipeline, including tasks like automated testing, Docker image building, and pushing updates.
5. Basic Git Knowledge: Since GitOps revolves around the use of Git, familiarity with basic Git operations such as commit, push, pull, branch, and merge is necessary.
Now that we have covered all the prerequisites, lets get right in to the course
Dockerfile Setup
To build the docker image for your application, you need to create a Dockerfile in the Application code directory. A Dockerfile is a text file that contains instructions for how to assemble the image. Here is a sample Dockerfile for Python that you can customize for your application:
# Sample Dockerfile
FROM python:3.8-slim
WORKDIR /app
COPY . /app
RUN pip install -r requirements.txt
ENTRYPOINT ["python", "./my_script.py"]Github Actions Workflow
Our focus will be on dissecting each section of the pipeline, explaining its purpose and functionality in depth. This pipeline encompasses building and pushing Docker images, updating Helm charts, conducting smoke tests, and implementing rollbacks.
Section 1: Triggering the Workflow
name: GitOps Pipeline
on:
push:
branches:
- branchThis section defines the name of our workflow (GitOps Pipeline) and specifies the trigger for the workflow. Here, the workflow is triggered by a push event to a branch named branch. This means that any commit pushed to this specific branch will initiate the execution of the defined jobs in this workflow.
Section 2: Building and Pushing the Docker Image
Job: build_and_push_image
jobs:
build_and_push_image:
runs-on: ubuntu-latest
steps:
— name: Checkout Code from repository
uses: actions/checkout@v2
— name: Login to Docker Hub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
— name: Build and push Docker image
uses: docker/build-push-action@v2
with:
context: .
push: true
tags: image/path:dev-${{ github.run_number }}In this job, we’re setting up the actions to build and push a Docker image:
1. Checkout Code: The first step checks out the code from the Git repository, making it available for the subsequent steps.
2. Login to Docker Hub: This step logs into Docker Hub using credentials stored in GitHub secrets. This is crucial for pushing the built image to the Docker registry.
3. Build and Push Docker Image: Here, we use the docker/build-push-action@v2 action to build the Docker image based on the Dockerfile in the current context (.) and push it to the Docker registry. The image is tagged uniquely using the GitHub run number.
Section 3: Modifying the Image Tag in Helm Chart
Job: modify_image_tag
modify_image_tag:
needs: build_and_push_image
runs-on: ubuntu-latest
steps:
— uses: actions/checkout@v3
name: Changing the deployment of git repo
with:
repository: repo-link
token: ${{ secrets.GIT_TOKEN }}
ref: branch-to-pull
— name: modify the image
run: |
git config user.email ${{ secrets.email }}
git config user.name ${{ secrets.name }}
echo “Working Directory: $(pwd)”
# Navigate to the specific Helm chart directory
cd helm-chart-dir
# Print values.yaml for debugging before changes
echo “Before modification:”
cat values.yaml
sed -i ‘/^image:/,/^ tag:/{s/^ tag: .*/ tag: “'$ENVIRONMENT'-’$RUN_NUMBER’”/}’ values.yaml
# Print values.yaml for debugging after changes
echo “After modification:”
cat values.yaml
git add values.yaml
git commit -m “Update image tag by Github Actions Job change manifest: ${{ github.run_number }}”
git push origin dev
env:
GIT_USERNAME: ${{ secrets.GIT_USERNAME }}
GIT_PASSWORD: ${{ secrets.GIT_TOKEN }}
RUN_NUMBER: ${{ github.run_number }}
ENVIRONMENT: ${{ vars.ENVIRONMENT }}
The modify_image_tag job is responsible for updating the image tag in the Helm chart:
1. Checkout Helm Chart Repository: This step checks out the Git repository (or a specific branch) where the Helm chart is stored.
2. Modify the Image Tag: The script updates the values.yaml file within the Helm chart directory, replacing the image tag with the new one ($ENVIRONMENT-${{ github.run_number }}). This ensures that the Helm chart will deploy the latest Docker image built in the previous job.
3. Commit and Push Changes: After modifying the values.yaml file, the script commits and pushes these changes back to the Git repository. This update will trigger any automated deployment processes that are watching this repository, such as ArgoCD.
Section 4: Testing the Application
Job: test_application
test_application:
runs-on: ubuntu-latest
needs: modify_image_tag
steps:
— uses: actions/setup-python@v4
— run: |
pip install requests
python ./smoke_test.py $APP_URL - expected_content "Your Expected Response"
#Save current RUN_NUMBER
gh variable list
gh variable set PREVIOUS_RUN_NUMBER --body ${{ github.run_number }}
gh variable list
env:
APP_URL: ${{ var.APP_URL }}
The test_application job performs smoke tests on the deployed application:
1. Setup Python Environment: This step sets up a Python environment using actions/setup-python@v4.
2. Run Smoke Tests: It then installs the necessary Python packages (requests) and runs a Python script (smoke_test.py). This script performs basic checks against the deployed application to ensure its functionality. Below is the python code:
import requests
import sys
import argparse
def smoke_test(url, expected_status, expected_content=None):
try:
response = requests.get(url)
if response.status_code != expected_status:
print(f"Smoke test failed: {url} returned status code {response.status_code}")
sys.exit(1)
if expected_content and expected_content not in response.text:
print(f"Smoke test failed: '{expected_content}' not found in response from {url}")
sys.exit(1)
print(f"Smoke test passed: {url} returned status code {expected_status} and expected content was found.")
except requests.RequestException as e:
print(f"Smoke test failed: Could not connect to {url}")
print(f"Error: {e}")
sys.exit(1)
def parse_arguments():
parser = argparse.ArgumentParser(description='Smoke test a given URL.')
parser.add_argument('url', type=str, help='URL to test')
parser.add_argument('--expected_content', type=str, help='Expected content in response', required=False)
return parser.parse_args()
if __name__ == "__main__":
args = parse_arguments()
URL = args.url
EXPECTED_STATUS = 200
EXPECTED_CONTENT = args.expected_content
smoke_test(URL, EXPECTED_STATUS, EXPECTED_CONTENT)3. Handling GitHub Run Number:
gh variable list is used to list the current workflow variables.gh variable set PREVIOUS_RUN_NUMBER is setting the variable PREVIOUS_RUN_NUMBER with the value of the current GitHub run number (${{ github.run_number }}).
This will be needed for our rollback job.
Section 5: Performing Rollback in Case of Failure
Job: perform_rollback
perform_rollback:
runs-on: ubuntu-latest
needs: test_application
if: failure()
steps:
— uses: actions/checkout@v3
name: Changing the deployment of git repo
with:
repository: repo-link
token: ${{ secrets.GIT_TOKEN }}
ref: branch-to-pull
— run: |
echo “Perform Rollback Cos Smoke Tests Failed”
git config user.email ${{ secrets.email }}
git config user.name ${{ secrets.name }}
echo “Working Directory: $(pwd)”
# Navigate to the specific Helm chart directory
cd helm-chart-dir
# Print values.yaml for debugging before changes
echo “Before modification:”
cat values.yaml
sed -i ‘/^image:/,/^ tag:/{s/^ tag: .*/ tag: “‘$ENVIRONMENT’-’$PREVIOUS_RUN_NUMBER’”/}’ values.yaml
# Print values.yaml for debugging after changes
echo “After modification:”
cat values.yaml
git add values.yaml
git commit -m “Rolled back image tag by Github Actions Job change manifest: $PREVIOUS_RUN_NUMBER”
git push origin ‘$ENVIRONMENT’
env:
PREVIOUS_RUN_NUMBER: ${{ vars.PREVIOUS_RUN_NUMBER }}
GIT_USERNAME: ${{ secrets.GIT_USERNAME }}
GIT_PASSWORD: ${{ secrets.GIT_TOKEN }}
RUN_NUMBER: ${{ github.run_number }}
ENVIRONMENT: ${{ vars.ENVIRONMENT }}The perform_rollback job is a crucial part of the pipeline that ensures stability:
1. Conditional Execution: This job is executed only if the test_application job fails (if: failure()).
2. Checkout and Rollback: Similar to the modify_image_tag job, it checks out the repository containing the Helm chart, but instead of updating to a new image tag, it rolls back to a previous stable version using the PREVIOUS_RUN_NUMBER.
3. Commit and Push Rollback Changes: After modifying the values.yaml file for rollback, the changes are committed and pushed to the Git repository, triggering a rollback in the Kubernetes deployment.
Conclusion
In this article, we have explored how to create a GitOps CI/CD pipeline with GitHub Actions. We have seen how automation and careful configuration can improve the reliability and efficiency of software deployment processes. We have learned how to build and push Docker images, update Helm charts, conduct smoke tests, and implement rollback strategies. These steps demonstrate the flexibility and robustness of GitHub Actions in a GitOps framework.
For those eager to explore more, I regularly update a dedicated GitHub repository with various CI/CD pipeline configurations and examples. This repository serves as a living resource for anyone looking to expand their knowledge and skills in modern DevOps practices. You can access this wealth of information at 👨🏾💻geek0ps .
Moreover, I welcome any questions, discussions, or feedback. Whether you’re facing specific challenges in your CI/CD journey or simply wish to share insights and experiences, feel free to connect with me on LinkedIn. My LinkedIn profile can be found at 👨🏾💻Julius Omoleye. Additionally, you can explore my portfolio to see a broader spectrum of my work and contributions to the field of software development and DevOps at 👇geekops.opspoint.tech.
The world of CI/CD and GitOps is constantly changing, and staying abreast of these changes is key to maintaining a competitive edge in the industry. I look forward to engaging with fellow enthusiasts and professionals in this dynamic and exciting field.
Remember, the journey to mastering CI/CD is ongoing, and every challenge is an opportunity for growth and innovation👨🏾💻.
