Deployment of Django Notes App with Jenkins CI/CD

Project Overview

Here, In this project we will explore a DevOps project that involves deploying a Django notes app on an EC2 instance using Jenkins declarative CI/CD pipeline.

This project is a Django-notes-app web application. To automate the deployment process of this application, we have set up a CI/CD pipeline using Jenkins. The pipeline consists of several stages, including code checkout, building Docker images, deploying services to servers using Docker Compose, and updating the Docker registry, on the server.

By setting up this pipeline, we aim to streamline the deployment process of our Django web application and ensure that each new change to the codebase is automatically deployed. This approach will save us time and effort while ensuring the reliability and scalability of our application.

Architecture of Project

Pre-requisites

First of all, go to the AWS portal, and create a new instance. As,

· Name: Jenkins-Master

· AMI: ubuntu.

· Instance type: t2.micro (free tier).

· Key pair login: Create > jenkins-master.pem.

· Allow HTTP (Anywhere).

· Allow HTTPS (Anywhere).

Click on Launch Instance, then connect to the EC2 instance and install following packages:

# Mandatory step **
# Update all packages
$ sudo apt update

Install Docker on Jenkins-Master

Docker Installation

Install Jenkins on Jenkins-Master instance

Install Java(Pre-requisite for Jenkins):

$ sudo apt install fontconfig openjdk-17-jre

Now, Install Jenkins

Create script called jenkins.sh and add below commands.

$ sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
  https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key
$ echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc]" \
  https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
  /etc/apt/sources.list.d/jenkins.list > /dev/null
$ sudo apt-get update
$ sudo apt-get install jenkins

Then give executable permission to script jenkins.sh,

$ chmod 700 jenkins.sh

Execute script,

$ ./jenkins.sh

Access Jenkins using UI, Before that allow port 8080 in security group of instance.

Access of Jenkins server should be accessible from OWN IP only initially.

After above step you will get like,

Go to the location which is mentioned in Unlock Jenkins page at system level and copy the Admin password then paste in box.

$ sudo cat /var/lib/jenkins/secrets/initialAdminPassword
e8e6f212c37742b491de027910dc****

Click on Installed suggested plugins,

Now, create Admin user

Default Jenkins server URL,

Now, Jenkins is ready.

Install Docker compose on Jenkins-Master

$ sudo apt-get install docker-compose-v2

Additionally, we need to add users to the Docker group to grant them appropriate permissions. Run the following commands to add users to the Docker group:

# On Jenkins-Master
$ sudo usermod -aG docker $USER # Currently logged-in User
$ sudo init 6 # Reboot Jenkins server

Setup sample CI/CD pipeline on Jenkins

Below is an Welcome page of Jenkins after fresh installation,

From the Jenkins Dashboard, Click on “New Item” to create Job.
Item Name : django-notes-app
Select : Pipeline

Click 'OK'.
Put GitHub repository URL, django-notes-app

Sample Pipeline,

pipeline{

    agent any;

    stages{
        stage("Code"){
            steps{
                echo "Code Clone done"
            }
        }
        stage("Build"){
            steps{
                echo "Build done"
            }
        }
        stage("Test"){
            steps{
                echo "Testing done"
            }
        }   
        stage("Deploy"){
            steps{
                echo "Deployment done"
            }
        }
    }
}

Clink on "Save" and then run the Job.

Setup of two worker Slaves

Understanding Jenkins Master-Slave architecture,

Jenkins uses a Master-Slave architecture to manage distributed builds. In this architecture, Master and Slave communicate through TCP/IP protocol.

Jenkins Master

Your main Jenkins server is the Master. The Master’s job is to handle:

Scheduling build jobs.
Dispatching builds to the slaves for the actual execution.
Monitor the slaves (possibly taking them online and offline as required).
Recording and presenting the build results.
A Master instance of Jenkins can also execute build jobs directly.

Jenkins Slave

A Slave is a Java executable that runs on a remote machine. Following are the characteristics of Jenkins Slaves:

It hears requests from the Jenkins Master instance.
Slaves can run on a variety of operating systems.
The job of a Slave is to do as they are told to, which involves executing build jobs dispatched by the Master.
You can configure a project to always run on a particular Slave machine or a particular type of Slave machine, or simply let Jenkins pick the next available Slave.

-> Pre-requisites

Create two separate EC2 instance for two slaves nodes.
Install JDK 17 on both slaves.
Install Docker on both slaves and Add user to docker group.
Install docker-compose-v2 on both slaves
Caution : Do not install Jenkins on slaves ever then they will behave as master.

Connect both slaves using SSH Key.

Update all packages on both slaves

$ sudo apt update

Final setup of master and slaves

Now try to access slaves 1 & 2 from master

Hence, we need to generate key pairs on Jenkins master and then copy master's public key and paste it in the Slaves 1 & 2 authorized keys.

Let's generate SSH key pairs on Master

$ ssh-keygen -t rsa
# -t : type of algorithm
# rsa : Encryption algorithm

Copy Public key of master and paste it in the authorized key file of both slaves.

-> Slave 1

-> Slave 2

Now, try to perform SSH from master to slaves.

Install JDK 17 on both slaves

Slave-1

Slave-2

Above SSH connection done only for server to server not for Jenkins to Server.

Setup of nodes in Jenkins UI

Go to Dashboard -> Manage Jenkins -> Set up agent

For Slave-1

For remote root directory, Go to Slave-1 and create one directory called 'apps' and copy complete path and paste it in remote root directory tab.

Give label as 'dev'.

Select Launch method 'via SSH'
In Host, Put Public IP of Slave-1 and Create credentials.

Go to Credentials -> Add -> Jenkins

Select Kind : SSH Username and Private key

In Private key -> Select Enter directly and put private key of master.

Select verification strategy as 'non'.

Click on 'save'.
Above steps are for Slave-1 and follow same steps for Slave-2.

After setup of Slave-2, Refresh the page and both Slaves should in Sync with master.

Running Job on Slave-1(dev),

First Go to pipeline and Add agent label as 'dev'.

pipeline{

    agent{
        node{
            label "dev"
        }
    }

Click on 'save' and Build the Job. Job should be running on Slave-1.

Setup of remaining Pipeline

pipeline{

    agent{
        node{
            label "dev"
        }
    }

    stages{
        stage("Clone Code"){
            steps{
                git url: "https://github.com/ojasjawale/django-notes-app.git", branch: "main"
                echo "Code Clone done"
            }
        }
        stage("Build & Test"){
            steps{
                sh "whoami"
                sh "docker build -t notes-app:latest ."
                echo "Build done"
            }
        }
        stage("Deploy"){
            steps{
                sh "docker stop notes-app && docker rm notes-app"
                sh "docker run -d -p 8000:8000 --name notes-app notes-app:latest"
                echo "Deployment done"
            }
        }
    }
}

Allow Port 8000 in security group of Slave-1.

Now copy IP of Slave-1 and Paste it on browser with Port 8000.

Run pipeline using Docker compose

stage("Deploy"){
            steps{
                sh "docker compose up -d"
                echo "Deployment done"
            }
        }

Save and Build Job.

Push image to DockerHub

-> Pre-requisites

Define login credentials of Docker using Global variables in Jenkins.
Create variables, Go to manager Jenkins -> Security -> Credentials ->

Under Stores scoped to Jenkins -> Click on Global

Add credentials

Select Kind : username and password
Username : Put your DockerHub username(mine is ojasjawale)
Password : Create Personal Access Token on DockerHub

Creating Personal Access Token on DockerHub

Go to DockerHub and login.
Go to Icon -> Account Setting

Click on Personal Access Tokens tab,

Click in Generate New Token

Give name and Select permission level

Copy Access Token and Paste it in the password tab.

Define credentials variables in pipeline,

stage("Push to DockerHub"){
            steps{
                echo "Pushing to DockerHub..."
                withCredentials(
                    [usernamePassword(
                        credentialsId: "docker-hub-cred", 
                        passwordVariable: "dockerhubPassword", 
                        usernameVariable: "dockerhubUsername"
                        )
                    ]
                ){
                sh "docker image tag notes-app:latest ${env.dockerhubUsername}/notes-app:latest"
                sh "docker login -u ${env.dockerhubUsername} -p ${env.dockerhubPassword}"
                sh "docker push ${env.dockerhubUsername}/notes-app:latest"
                }
            }
        }