Jenkins master-slave architecture
Jenkins uses a Master-Slave architecture to manage distributed builds. In this architecture, Master and Slave communicate through TCP/IP protocol.
Jenkins Master
Your main Jenkins server is the Master. The Master’s job is to handle:
Scheduling build jobs.
Dispatching builds to the slaves for the actual execution.
Monitor the slaves (possibly taking them online and offline as required).
Recording and presenting the build results.
A Master instance of Jenkins can also execute build jobs directly.
Jenkins Slave
A Slave is a Java executable that runs on a remote machine. Following are the characteristics of Jenkins Slaves:
It hears requests from the Jenkins Master instance.
Slaves can run on a variety of operating systems.
The job of a Slave is to do as they are told to, which involves executing build jobs dispatched by the Master.
You can configure a project to always run on a particular Slave machine or a particular type of Slave machine, or simply let Jenkins pick the next available Slave.
Pre-requisites to setup Jenkins master & slave
Create one EC2 instance for Master node.
Create two separate EC2 instance for two slaves nodes.
Install JDK 17 on master & on both slaves.
Install Docker on master and both slaves and add user to docker group.
Caution : Do not install Jenkins on slaves ever then they will behave as master.
- Connect both slaves using SSH Key.
- Update all packages on both slaves
Copy
$ sudo apt update
- Final setup of master and slaves
Now try to access slaves 1 & 2 from master
Hence, we need to generate key pairs on Jenkins master and then copy master's public key and paste it in the Slaves 1 & 2 authorized keys.
- Let's generate SSH key pairs on Master
Copy
$ ssh-keygen -t rsa
# -t : type of algorithm
# rsa : Encryption algorithm
Copy Public key of master and paste it in the authorized key file of both slaves.
-> Slave 1
-> Slave 2
Now, try to perform SSH from master to slaves.
Login success on Slave-1.
Login success on Slave-2.
- Install JDK 17 on both slaves
Slave-1
Slave-2
Above SSH connection done only for server to server not for Jenkins to Server.
Installation of Jenkins on master instance
- As we created AWS instance for master in previous step, now install Jenkins on master.
- Connect instance using SSH key
# Mandatory step **
# Update all packages
$ sudo apt update
- Install Docker on Jenkins-Master
- Install Jenkins on Jenkins-Master instance
- Install Java(Pre-requisite for Jenkins):
$ sudo apt install fontconfig openjdk-17-jre
- Now, Install Jenkins
Create script called jenkins.sh and add below commands.
$ sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key
$ echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc]" \
https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
/etc/apt/sources.list.d/jenkins.list > /dev/null
$ sudo apt-get update
$ sudo apt-get install jenkins
Then give executable permission to script jenkins.sh,
$ chmod 700 jenkins.sh
Execute script,
$ ./jenkins.sh
Access Jenkins using UI, Before that allow port 8080 in security group of instance.
Access of Jenkins server should be accessible from OWN IP only initially.
After above step you will get like,
Go to the location which is mentioned in Unlock Jenkins page at system level and copy the Admin password then paste in box.
$ sudo cat /var/lib/jenkins/secrets/initialAdminPassword
e8e6f212c37742b491de027910dc****
Click on Installed suggested plugins,
Now, create Admin user
Default Jenkins server URL,
Now, Jenkins is ready.
- Install Docker compose on Jenkins-Master
$ sudo apt-get install docker-compose-v2
Additionally, we need to add users to the Docker group to grant them appropriate permissions. Run the following commands to add users to the Docker group:
Copy
# On Jenkins-Master
$ sudo usermod -aG docker $USER # Currently logged-in User
$ sudo init 6 # Reboot Jenkins server
Setup of nodes in Jenkins UI
Go to Dashboard -> Manage Jenkins -> Set up agent
- For Slave-1
For remote root directory, Go to Slave-1 and create one directory called 'apps' and copy complete path and paste it in remote root directory tab.
- Give label as
'dev'.
Select Launch method
'via SSH'In Host, Put Public IP of Slave-1 and Create credentials.
- Go to Credentials -> Add -> Jenkins
- Select Kind : SSH Username and Private key
- In Private key -> Select Enter directly and put private key of master.
- Select verification strategy as
'non'.
Click on
'save'.Above steps are for Slave-1 and follow same steps for Slave-2.
After setup of Slave-2, Refresh the page and both Slaves should in Sync with master.
Creation of Job to run on slave
Below is an Welcome page of Jenkins after fresh installation,
From the Jenkins Dashboard, Click on “New Item” to create Job.
Item Name : django-notes-app
Select : Pipeline
Click 'OK'.
Put GitHub repository URL, django-notes-app
Sample Pipeline,
pipeline{
agent any;
stages{
stage("Code"){
steps{
echo "Code Clone done"
}
}
stage("Build"){
steps{
echo "Build done"
}
}
stage("Test"){
steps{
echo "Testing done"
}
}
stage("Deploy"){
steps{
echo "Deployment done"
}
}
}
}
- Clink on "Save" and then run the Job.
Running Job on slave
Running Job on Slave-1(dev),
First Go to pipeline and Add agent label as 'dev'.
pipeline{
agent{
node{
label "dev"
}
}
Click on 'save' and Build the Job. Job should be running on Slave-1.
Connect With Me
Thank you for reading. I hope you were able to understand and learn something new from my blog.
Happy Learning!