Jenkins pipeline to Automate Docker build and push

Jenkins is an automation server that helps automate software development by building, testing, and deploying in continuous integration, and continuous delivery/deployment (CI/CD).

Docker is a software platform used by developers to build, test, and deploy applications into packages called containers containing all the application's dependencies to run on.

Importance of automation in the CI/CD pipeline.

Automation uses machines to accomplish tasks and minimize human intervention—DevOps operations to achieve reliable systems before automation was very hectic. Writing the same codes repeatedly to achieve seamless operations has been time-consuming and costly. Therefore, automation was incorporated into these operations to achieve a more flexible and reliable system with minimal human intervention thus saving on labor and time.

The following are some of the importance of a well-developed CI/CD pipeline;

Accelerate software delivery and development cycles
Minimal errors as major and complex tasks are automated.
Faster review times and resolutions to test code as a large code base is reviewed.
Minimize high labor costs for building and testing code.
The software released is stable and of high quality.
Good time management since the automation process is triggered.

This article explains in detail how to automate docker image creation using Jenkins. Instead of reviewing and building images using docker manually, Jenkins plays a crucial role in automating the process and thus helping the user reduce errors and make time for other important tasks.

Prerequisites

Dockerfile

You will need to have your Dockerfile and its dependencies ready and upload the project to GitHub. To get started on how to create a Dockerfile, refer to this guide Dockerfile.

Before moving forward to automate the Docker Image creation with Jenkins, make sure the Dockerfile can be executed manually to create an image.

Jenkins installations

As explained above, Jenkins is an automation server used to automate building, testing, and deploying software.

In this article, we will install our Jenkins on Windows OS. Visit this link Jenkins to get started on Jenkins installation.

After successfully installing Jenkins, you should be able to access Jenkins via localhost:8080 on your browser as shown in the image below.

Docker installation

Docker, a platform for containerized applications can comfortably be installed and run on Windows OS. Refer to this guide to get started with Docker.

After successfully installing Docker Desktop, your Docker Desktop app will be similar to the image below.

Docker Plugin for Jenkins

Docker plugin for Jenkins is a Jenkins plugin developed to support Docker that automates Docker image creation and containerization when Jenkins jobs are triggered. Use this guide Docker-Jenkins plugin to learn more.

Follow the steps below to install the Docker plugin from the Jenkins plugin manager.

Access the Jenkins dashboard in a web browser by using the default URL http://localhost:8080 or your configured URL and type in your username and password to log in to your administrator account.

On the Jenkins dashboard, navigate to the left side menu and click Manage Jenkins.

Under system configuration, click Plugins.

Then, in the Plugins section, click Available Plugins (displays all the available plugins). There is a search box for searching all suitable plugins that match your needs.

On the search box, search Docker to output all Docker-related plugins.
Look for these plugins; Docker Pipeline Plugin, CloudBees Docker Build and Push plugin, Docker Build Step, and Docker Plugin. Check all their respective checkboxes and click Install.

The plugins will start to download immediately and after successful installation, the respective plugins will show a success message on the right with a green tick.

Restart Jenkins for changes to take effect.
After restarting, navigate back to Manage Plugins to check if the Docker plugins are listed in the Installed Plugins tab. Type Docker on the search box to display all installed Docker plugins.

Setting up Docker for Jenkins

Configuring Docker Host in Jenkins

After successfully installing the Docker plugins and verifying their installation, you will need to configure Jenkins to use Docker as a building tool.

On the Jenkins Dashboard, click on Manage Jenkins, and under system configuration, click on Cloud.

Under Cloud, select Add New Cloud.

Choose Docker as your cloud provider and click create.

Jenkins has to communicate with Docker Desktop via a URL.
Open Docker Desktop >, Settings >, General.
Check the Expose Daemon on tcp://localhost:2375 without TLS. Ignore the warning.
Apply and restart Docker.

Go back to Jenkins Docker settings and paste the link tcp://localhost:2375.

Click Test connection to ensure Jenkins can Communicate with Docker.
If the connection is successful, it will display the Docker version and API. Click Save.

Adding Docker Credentials to Jenkins

This step is crucial for pushing the Docker image to Dockerhub. Jenkins will use the docker logins to gain access to the Docker Hub.

On the Jenkins Dashboard click Manage Jenkins, under Security, click Credentials.

Add your Docker Registry credentials that is, set your Docker hub username and password and add a suitable ID then click Create.

Adding other necessary tools in Jenkins

For the Jenkins pipeline to run smoothly, install tools such as Maven, JDK, and Docker on the Jenkins server.

On the Jenkins Dashboard > Manage Jenkins > system configuration, click Tools.

On JDK installations, add JDK and give it a name, for example, OpenJDK8.

Check the installation automatically. Then choose the installer as install for adoptium.net and select the latest version of JDK.

Scroll down to Maven installations and add Maven. Check the installation automatically checkbox and choose the installer as install from Apache and select the latest version of Maven.

Finally, scroll down to Docker installations and select Add Docker.

Choose a suitable name, check the install automatically checkbox, and choose the installer as a download from docker.com. Choose the latest version.

Configuring Maven POM file.

We will use Maven as a build agent for Jenkins.

On your GitHub repository, create a new file and name it POM.xml.

Paste the below code and save it. (This is a sample Maven Project. Learn more here)

<project xmlns="maven.apache.org/POM/4.0.0"
xmlns:xsi="w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="maven.apache.org/POM/4.0.0 maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>


<groupId>com.example</groupId>
<artifactId>my-project</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging> 

<name>My Maven Project</name>
<description>A simple Maven project.</description>
<url>example.com</url>


<dependencies>

<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
<version>3.1.4</version>
</dependency>
</dependencies>


<build>
<plugins>

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.10.1</version>
<configuration>
<source>11</source> 
<target>11</target> 
</configuration>
</plugin>
</plugins>
</build>
</project>

The above code is a Maven Project, a build automation tool for Java projects.

Code Breakdown

XML Declaration and Schema

This code declares the XML namespace for the POM schema and ensures the file conforms to Maven’s POM structure version 4.0.0.

Project Data.

<groupId>com.example</groupId>
<artifactId>my-project</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging> 

<name>My Maven Project</name>
<description>A simple Maven project.</description>
<url>example.com</url>

groupId: A unique project group identifier.

artifactId: Unique name of the project within the same group.

version: This has the project version.

packaging: This part shows the type of artifact to produce for example jar for libraries.

name: Name of the Project.

description: Contains description of the project.

url: Project’s website.

Dependencies

<dependencies>

<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
<version>3.1.4</version>
</dependency>
</dependencies>

This part declares the libraries required for the project. It specifies Spring Boot Starter dependency version 3.1.4.

Build Configuration

<build>
<plugins>

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.10.1</version>
<configuration>
<source>11</source> 
<target>11</target> 
</configuration>
</plugin>
</plugins>
</build>

This code snippet configures the project build process. The plugin adds build plugins to extend Maven’s functionality targeting Java 11.

Creating a Jenkins pipeline

Jenkins pipelines are a set of plugins that contain automated steps that Jenkins executes to build, test, and deploy applications. It is from these pipelines that continuous delivery and integration are implemented to achieve highly flexible and robust applications.

Writing a Jenkins file

Below is a sample of a Jenkins pipeline designed to automate the process of checking the code, building a Docker image, and pushing the created image to a Docker hub.

Sample Jenkins Code.

pipeline {
agent any

stages {
stage ('Checkout Code') {
steps {
checkout scmGit(branches: [[name: '*/main']], extensions: [], userRemoteConfigs: [[url: 'github.com/joshwizard/Docker-Image-Build-Au..)
bat 'mvn clean install'
}
}

stage('Build Docker Image') {
steps {
script{
bat 'docker build -t joshmurih/flaskapp:latest .'
}
}
}

stage('Push Image to Hub') {
steps {
script {
withCredentials([usernamePassword(credentialsId: 'dockerhubpassword', usernameVariable: 'DOCKER_USER', passwordVariable: 'DOCKER_PASS')]) {
bat """
echo $DOCKER_PASS | docker login -u $DOCKER_USER --password-stdin
docker push joshmurih/flaskapp:latest
"""
}
}
}
}

}
}

Explanation of each stage in the pipeline.

Pipeline Overview

pipeline {
agent any
...
}

Pipeline: This is what defines the entire Jenkins pipeline.

Agent any: Communicates with Jenkins to run the pipeline on any available agents. You can specify the containers if need be.

Stages and Steps

The stages in the pipeline show phases that the Jenkins pipeline follows to accomplish specific tasks.

Stage 1: Checkout Code

stage ('Checkout Code') {
steps {
checkout scmGit(branches: [[name: '*/main']], extensions: [], userRemoteConfigs: [[url: 'github.com/joshwizard/Docker-Image-Build-Au..)
bat 'mvn clean install'
}
}

Checkout Code: This part retrieves the latest code from a Git Repository and builds it using Maven.

steps: Contains the tasks to be executed in this stage.

Checkout scmGit: Checks out the code from the specified Git repository branch (Main).
- branches: Specifies the branch to fetch the code.
- userRemoteConfigs: Contains the GitHub repository URL.
bat ‘mvn clean install’ : Runs Maven command to clean and build the project. bat is a command for Windows, sh is built for Unix/Linux platforms.

Stage 2: Build Docker Image

stage('Build Docker Image') {
steps {
script{
bat 'docker build -t joshmurih/flaskapp:latest .'
}
}
}

Build Docker Image: This stage will build a Docker image using the project’s Dockerfile.

script: specifies code to run for execution.

bat ‘docker build -t joshmurih/flaskapp:latest .’: This command executes a Docker build image command.

Stage 3: Push Docker Image

stage('Push Image to Hub') {
steps {
script {
withCredentials([usernamePassword(credentialsId: 'dockerhubpassword', usernameVariable: 'DOCKER_USER', passwordVariable: 'DOCKER_PASS')]) {
bat """
echo $DOCKER_PASS | docker login -u $DOCKER_USER --password-stdin
docker push joshmurih/flaskapp:latest
"""
}
}
}

Push Docker Image: After building the Docker image, this stage pushes the Docker image to the Docker Hub.

withCredentials: Securely injects credentials from the Jenkins credentials system server.

credentialsId: This contains Docker Hub credentials (username and password).

usernameVariable: This step binds the Docker Hub username to DOCKER_USER.

passwordVariable: This step binds the Docker Hub password to DOCKER_PASS.

The code runs under bat command, logs into the Docker Hub.

docker push joshmurih/flaskapp:latest: This command pushes the Docker image to Docker Hub successfully.

Running the pipeline Job

Creating the pipeline job in Jenkins

Now after everything is set, a job must be created to enable the build and push to take place effectively.

When a job is created, you can schedule build and push by setting a trigger of your choice or do it manually.

To create a Jenkins pipeline;

A new page is displayed and asks for an item name. Enter a suitable name for your Jenkins pipeline.

On the Select Item Type, choose Pipeline and click OK.

A new configuration page is displayed where all the settings of the pipeline take place.

Type a short and brief description of your project in the text area provided.

Check the Discard old builds to prevent accumulating old builds that have no use in your dashboard.

Put a maximum of 2 builds (It depends on your preference.)

Under Build Triggers

Check the checkbox for the GitHub hook trigger for GITScm polling.

Note: To understand better the build triggers, at the end of every command there is a question mark that contains more detailed information of what that particular command does.

On the Pipeline part, select pipeline script.

Copy and paste the sample Pipeline Jenkins code on the script area click apply to capture changes and save to close the window.

A page is displayed and on the left side of the page, click Build Now to start building your project.

When building the project, open your pipeline project, click on the project building number starting with #, and click Console Output to monitor project's progress. (It also contains error logs if the project fails to execute properly).

If the process is successful, open Docker Desktop and confirm the docker image. Thereafter, open Docker Hub on your browser and check if your image has been published.

Best practices

Security Considerations

Publishing Docker images to Docker Hub can be vulnerable especially if Docker credentials are not securely configured.

Use Jenkins credentials to manage sensitive information like APIs, usernames and passwords, and Docker Hub credentials.

pipeline {
environment {
DOCKER_CREDS = credentials('docker-registry-credentials-id')
}
stages {
stage('Login to Docker Registry') {
steps {
sh 'echo $DOCKER_CREDS_PSW | docker login -u $DOCKER_CREDS_USR --password-stdin'
}
}
}
}

Pipeline Simplicity

Use clean and structured code syntax which is easier to debug in case an error occurs.

Make sure to maintain clean indentation and flow of code syntax which can also be used in future projects by simply changing some commands, steps, or stages.

Docker Image Management

Ensure a proper workspace cleanup to free up space to avoid disk space-related issues.

Cleaning the Docker workspace creates an efficient platform to run other newly created images without taking into consideration space issues.

Use the code below in your pipeline script to clean up space after the image has been built.

stage('Clean Up') {
steps {
sh 'docker system prune -f'
}
}

Conclusion

This article has covered all the relevant steps to build and push the created Docker image to Docker Hub using well defined Jenkins Pipeline from checkout code, building the image, and pushing to Docker Hub.

The pace of technological innovation is accelerating and as a result, always counter-check for new updates on plugins to avoid unnecessary errors.

Also, save the pipeline script as Jenkinsfile in your GitHub Repository to use later by running the Pipeline Script from SCM on the Jenkins pipeline instead of writing a new script.

Explore other automation possibilities with Jenkins and Docker to make work easier and pave the way for other project priorities.

How to Automate Docker Build and Push with Jenkins Pipeline