Deploying a Jenkins CI/CD Pipeline

This page describes how to deploy a MettleCI CI/CD Pipeline using Jenkins. It assumes you have created and configured Jenkins Pipelines before. Consult with your local Jenkins resources for additional detail and guidance if necessary.

MettleCI ships with a template Git repository which includes ajenkinsfiletemplate file (located in ./pipelines/jenkins in the MettleCI sample repository) which you can modify as required. You can take a copy from the template repository and place it in your project repository (typically located in the root folder) by committing it using your preferred Git interface.

The template git repository can be found in the repositories section of the MettleCI software distribution as mettleci-jenkins-template.zip. See Accessing the MettleCI Software Distribution.

The sample pipeline is annotated ( /* this is a comment */ ) to make it easy to understand where you need to place your environment-specific values, such as hostnames and credentials references. The sample pipeline has three stages as follows (each with a number of steps):

• Deploy: Fetch the latest version of the DataStage project assets from your repo, create or validate a CI project on your DataStage environment, perform some substitutions in configuration/parameter files and transfer them to the engine, then deploy all the changed assets (DataStage jobs, sequences, etc, and configuration files) to the project, compiling the changed and dependent jobs after import, and do some clean up

• Test: Two subsets of steps:
  - in the Static Analysis subset, perform compliance checking by fetching the latest version of the compliance rules, and applying them against each changed job or other relevant asset in the project, then archive the JUnit results, then clean up
  - in the Unit Tests subset, fetch any unit test specifications for changed assets, and execute them, generating unit test reports, then archive the JUnit results and clean up

• Promote: Three identical subsets of steps, each of which performs the tasks outlined for Deploy but for a different environment. The first subset runs without intervention but the second two prompt the user whether to deploy to that environment or not.

The sample pipeline is parameterised with 4 parameters, which are set and used multiple times, once for each environment. You should edit all the occurrences of these 4 parameters to match your values, and commit the change.

• domainName: the hostname and port of the service tier to which you are deploying the project.

• serverName: the hostname of the engine tier to which you are deploying the CI project.

• projectName: the ‘base name’ of your project without the environmentId appended

• environmentId : An abbreviation of the name of the environment to which you are deploying. This is CI for the first deployment. For subsequent promotions you should specify environments that make sense for your process. The sample uses GA (quality assurance), PERF (performance) and PROD (production).

When creating a new pipeline, select “Pipeline script from SCM”, and give the repository URL of your project, For script path give the relative path (if you didn’t place it at the root) and name. This will reference the copy of the template you extracted, customised and committed earlier.

If you want the pipeline to run automatically whenever there is a commit to your repo, enable “Poll SCM” and provide a schedule in cron notation. For instance, every minute would be “* * * * *“

After saving, launch the pipeline as normal (manually or via a commit if so configured).

Here’s what a customised version of the sample pipeline might look like in Blue Ocean when run. All stages ran to a successful completion. The results of the Deploy step are shown below the graphical pipeline representation. Twisties by each step let you review the results in further detail.

Tests

DataStage job compilation, MettleCI Compliance testing, and MettleCI Unit testing are all exposed as xUnit (JUnit-compatible) XML files, and so appear as tests in the Jenkins Tests tab, with expansion icons which let you view additional details, where appropriate. Tests might be shown as “skipped” when they were run in a previous build and there has been no change in the underlying assets to merit a retest.

For additional details on the pipeline structure, see Understanding your Jenkins CI/CD Pipeline

Triggering your build

See Pipeline Trigger Strategies