Jenkins Pipelines Explained: Declarative Syntax, Stages and Agents

A Jenkins Pipeline defines an automated delivery workflow as code. Instead of manually configuring every build action in the Jenkins interface, teams store a Jenkinsfile with the application source code and review pipeline changes through version control.

This Jenkins pipeline tutorial focuses on Declarative Pipeline syntax, stage execution, agent selection and practical troubleshooting. It assumes that Jenkins is installed and connected to a Git repository.

What is a Jenkins Pipeline?

A Jenkins Pipeline is a sequence of automated steps used to build, test and deliver software. Jenkins reads these steps from a Jenkinsfile, schedules them on an available agent and records the result of each stage.

A basic delivery flow can be visualised as:

Developer pushes code
        |
        v
Jenkins detects the change
        |
        v
Checkout -> Build -> Test -> Package -> Publish
        |
        v
Success or failure notification

Pipeline as code provides several practical benefits:

• The build process is stored with the application.
• Pipeline changes can be reviewed through pull requests.
• Teams can reproduce the same workflow across Jenkins environments.
• Jenkins keeps stage logs, execution duration and build history.
• Failed stages can be identified without reading one large build script.

Jenkins supports two Pipeline styles: Declarative and Scripted.

This article uses Declarative Pipeline because its predictable structure is suitable for maintainable team projects.

How is a Declarative Jenkinsfile structured?

A Declarative Jenkinsfile starts with a top-level pipeline block. Inside it, the agent, stages, steps and optional post sections describe where the pipeline runs, what it does and what happens after execution.

Here is the smallest useful example:

pipeline {
    agent any

    stages {
        stage('Build') {
            steps {
                echo 'Building the application'
            }
        }

        stage('Test') {
            steps {
                echo 'Running tests'
            }
        }
    }

    post {
        success {
            echo 'Pipeline completed successfully'
        }
        failure {
            echo 'Pipeline failed'
        }
    }
}

The main blocks have different responsibilities:

Declarative syntax is strict. For example, commands such as sh and echo must normally be placed inside a steps block.

What does an agent mean in Jenkins?

A Jenkins agent is the system that executes Pipeline steps. The Jenkins controller schedules work, while a permanent machine, virtual machine or container provides the CPU, memory, tools and workspace required by the job.

Think of the architecture as:

Jenkins controller
  |-- chooses an executor
  |-- sends the job
  v
Agent labelled "linux-docker"
  |-- checks out source code
  |-- runs Maven and Docker
  |-- returns logs and status

The simplest declaration allows Jenkins to use any available agent:

agent any

A label expression restricts execution to a suitable agent:

agent {
    label 'linux && docker'
}

The selected node must have both linux and docker labels. It must also have the required commands installed and accessible to the Jenkins service account.

For different environments, use no global agent and select one per stage:

pipeline {
    agent none

    stages {
        stage('Build on Linux') {
            agent { label 'linux' }
            steps {
                sh 'make build'
            }
        }

        stage('Windows Test') {
            agent { label 'windows' }
            steps {
                bat 'run-tests.bat'
            }
        }
    }
}

Jenkins can also create a temporary Docker-based build environment when the Docker Pipeline plugin is installed:

stage('Maven Build') {
    agent {
        docker {
            image 'maven:3.9-eclipse-temurin-17'
            reuseNode true
        }
    }
    steps {
        sh 'mvn -B clean verify'
    }
}

The Jenkins agent still needs access to a working Docker daemon. For container fundamentals before using this pattern, see Docker images, containers and volumes.

How do stages and steps execute?

Stages run sequentially by default, while the steps inside each stage run in the order written. Parallel branches can be used for independent checks, but each branch should avoid modifying the same files or shared environment at the same time.

A common sequence is:

Checkout
   |
Build
   |
Unit Test
   |
Package
   |
Publish only from main

Sequential stages are easy to read:

stages {
    stage('Checkout') {
        steps {
            checkout scm
        }
    }

    stage('Build') {
        steps {
            sh 'mvn -B compile'
        }
    }
}

Independent tests can run in parallel:

stage('Quality Checks') {
    parallel {
        stage('Unit Tests') {
            steps {
                sh 'mvn -B test'
            }
        }

        stage('Static Check') {
            steps {
                sh './scripts/static-check.sh'
            }
        }
    }
}

Jenkins waits for both parallel branches before moving to the next stage. If either branch fails, the parent stage normally fails.

The when directive provides conditional execution:

stage('Publish') {
    when {
        branch 'main'
    }
    steps {
        echo 'Publishing a release from the main branch'
    }
}

The branch condition is mainly intended for Multibranch Pipeline jobs, where Jenkins sets BRANCH_NAME. In a basic Pipeline job, use a parameter or an environment expression instead.

How do you build a practical Declarative Pipeline?

A practical Pipeline should check out code, run repeatable tests, create an artefact and publish only from an approved branch. Credentials must be stored in Jenkins and referenced by credential ID rather than written directly in the Jenkinsfile.

The following example builds a Java application and creates a Docker image. It assumes the agent has Git, Maven and Docker, and that Jenkins contains a username-and-password credential with the ID registry-login.

pipeline {
    agent {
        label 'linux && docker'
    }

    options {
        timestamps()
        timeout(time: 20, unit: 'MINUTES')
        disableConcurrentBuilds()
    }

    environment {
        IMAGE_NAME = 'registry.example.com/training/sample-api'
    }

    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }

        stage('Build and Test') {
            steps {
                sh 'mvn -B clean verify'
            }
        }

        stage('Build Image') {
            steps {
                sh 'docker build -t "$IMAGE_NAME:$BUILD_NUMBER" .'
            }
        }

        stage('Push Image') {
            when {
                branch 'main'
            }
            environment {
                REGISTRY_AUTH = credentials('registry-login')
            }
            steps {
                sh '''
                    printf '%s' "$REGISTRY_AUTH_PSW" | \
                      docker login registry.example.com \
                      --username "$REGISTRY_AUTH_USR" \
                      --password-stdin

                    docker push "$IMAGE_NAME:$BUILD_NUMBER"
                    docker logout registry.example.com
                '''
            }
        }
    }

    post {
        always {
            junit testResults: 'target/surefire-reports/*.xml',
                  allowEmptyResults: true
        }
        success {
            archiveArtifacts artifacts: 'target/*.jar',
                             fingerprint: true
        }
        cleanup {
            deleteDir()
        }
    }
}

Important details in this Jenkinsfile include:

1. BUILD_NUMBER is a Jenkins-provided environment variable.
2. credentials('registry-login') creates variables ending in _USR and _PSW for a username-and-password credential.
3. The shell script is enclosed by triple single quotes, preventing Groovy from interpolating secrets before the shell runs.
4. --password-stdin avoids placing the registry password directly in the command arguments.
5. junit requires the JUnit plugin and reads Maven Surefire XML reports.
6. archiveArtifacts stores the generated JAR in Jenkins; it is not a replacement for a proper artefact repository.

Credential permissions should follow least privilege. The registry account should be allowed to push only to the required repository, similar to the access-control principles described in AWS IAM users, roles and policies.

How do you create and run the Pipeline in Jenkins?

Store the Jenkinsfile in the repository root, commit it and configure Jenkins to load the script from source control. A Multibranch Pipeline is usually suitable when each Git branch or pull request needs an independent build.

Create the file and commit it with Git:

touch Jenkinsfile
git add Jenkinsfile
git commit -m "Add Declarative Jenkins pipeline"
git push origin main

In Jenkins, the general workflow is:

1. Create a Pipeline or Multibranch Pipeline job.
2. Select the appropriate Git source configuration.
3. Add the repository URL and credentials if required.
4. Keep the script path as Jenkinsfile, unless it is stored elsewhere.
5. Save the job and select Build Now or scan the repository branches.
6. Open Console Output to inspect commands and errors.

You can validate Declarative syntax from the Jenkins CLI when CLI access is enabled:

java -jar jenkins-cli.jar \
  -s http://jenkins.example.com:8080/ \
  -auth username:api-token \
  declarative-linter < Jenkinsfile

A successful validation produces a message similar to:

Jenkinsfile successfully validated.

Validation checks Pipeline structure, but it cannot confirm that an agent has Docker, a credential ID exists or a shell command will succeed.

How do you troubleshoot common Jenkins Pipeline failures?

Start with the first failed stage and the earliest meaningful error in Console Output. Then determine whether the problem comes from Declarative syntax, agent availability, missing tools, workspace files, credentials or the application command itself.

On a Linux system running Jenkins as a systemd service, inspect its status and recent controller logs:

sudo systemctl status jenkins
sudo journalctl -u jenkins -n 100 --no-pager

Check tools from the agent using a temporary diagnostic stage:

stage('Diagnostics') {
    steps {
        sh '''
            whoami
            pwd
            java -version
            mvn -version
            docker version
        '''
    }
}

Do not print all environment variables in production because output may reveal sensitive operational information. Jenkins masks many bound secrets, but masking should not be treated as the only security control.

For a permission error such as:

permission denied while trying to connect to the Docker daemon socket

confirm which user runs the agent and inspect the socket:

id
ls -l /var/run/docker.sock

Adding a user to the Docker group grants powerful host-level access. Evaluate that risk before changing group membership; a dedicated isolated agent is often safer than giving the controller direct Docker access.

What are good Jenkinsfile practices?

Keep the Jenkinsfile readable, deterministic and free of embedded secrets. Use stages for major outcomes, place complex reusable logic in scripts or trusted shared libraries, and fail early when required tools or inputs are missing.

Recommended practices include:

• Keep the Jenkins controller focused on scheduling rather than running builds.
• Use clearly labelled and isolated agents.
• Pin important build-tool or container versions instead of relying on latest.
• Add timeouts so stalled commands do not occupy executors indefinitely.
• Prevent concurrent execution when builds share mutable resources.
• Store secrets in Jenkins Credentials and restrict their scope.
• Use post blocks for reports, artefacts and cleanup.
• Avoid putting large shell programs directly inside the Jenkinsfile.
• Test pipeline changes on a feature branch before merging them.
• Review plugins and remove unused integrations.

Developing these skills through repeatable labs is part of learning production-oriented automation. The AWS DevOps course connects Pipeline fundamentals with Git, Linux, containers, AWS services and infrastructure automation.

Summary

A Declarative Jenkins Pipeline uses a structured Jenkinsfile to define automated work. The agent chooses the execution environment, stages organise the workflow, steps run commands, when controls conditions and post handles results and cleanup.

Begin with a small build-and-test pipeline before adding image publishing or deployment. When a failure occurs, validate the syntax, inspect the first useful console error and confirm the selected agent has the required tools, permissions and credentials.

Reviewed by Network Rhinos DevOps trainers.

For guided Jenkins, Git, Docker and AWS automation labs, enquire about AWS DevOps course batch details.

Related reading: Cloud Engineer Career Path in Bangalore