Introduction to GitLab CI/CD

This tutorial is adapted from the Web Age course https://www.webagesolutions.com/courses/WA3243-advanced-gitlab-administration.

1.1 Continuous Integration (CI)

Every time a developer pushes code, CI ensures quality of the software by
running a few tasks:

• Compile and build the software to ensure no compilation and build error were introduced.
• Run static analysis checker (lint) to make sure common best practices are not being violated.
• Run a suite of automated tests to verify that old features are not broken and any new features are working properly.
• Run code coverage tools to make sure a minimum percentage of the code is being tested.
• You can choose to run all or some of these tasks in all branches or only on the main branch.

1.2 Continuous Delivery/Deployment (CD)

Continuous Delivery prepares the application for deployment. For example, the Docker image for the application built and pushed to a registry. Actual deployment is done manually. Continuous Deployment goes one step further and automatically deploys the application in Docker or Kubernetes. These steps can be done in a staging environment or in production.

1.3 GitLab and CI/CD

GitLab provides an extensive infrastructure to define the CI/CD tasks. For each project, you can create a pipeline of jobs. These jobs can build, test and deploy your application. Basically, GitLab CI/CD is an alternative to Jenkins and CircleCI. Except that GitLab comes with the CI/CD infrastructure built-in. There’s no need to install extra software.

1.4 A Typical CI/CD Workflow

This is a basic starter flow. You need to customize it for your need with additional branches and jobs.
The developer pushes changes to a feature branch. This kicks off the CI pipeline of jobs, such as build, lint check, and tests. If the CI pipeline finishes successfully then a merge request is created to merge the code into the main branch. The code is reviewed by the peers and senior developers. If the merge request is approved then the changes are merged into the main branch. The CD pipeline starts. It takes the code from the main branch, builds a Docker image, and pushes it to the image registry. Finally, deployment can be done manually or automatically. If the software is an NPM package or Java JAR file then it can also be pushed to the appropriate package repository from here.

1.5 Defining the CI/CD Pipeline

The pipeline is defined in the .gitlab-ci.yml file in the root folder of a project.
Sample file:

my_build_job:
stage: build
script:

- echo Shell commands here
- echo Doing a build

test_job_1:
stage: test
script:

- echo Testing should start after build finishes

You can edit this file in your local machine and push it to the project. The pipeline will start immediately.
GitLab website also provides an editor to make changes to the file. This is good for rapid experimentation.

1.6 The Anatomy of a Pipeline

• A pipeline contains all the jobs (or, tasks) needed to implement CI/CD.
• A project can have only one pipeline definition in the .gitlab-ci.yml file.
• A pipeline has several stages, such as, build, test and deploy. These stages are run sequentially. For example, the test stage runs only after the build stage finishes successfully.
• Each stage has a collection of jobs. These jobs are run in parallel.
• Each job has a sequence of script commands.
• Each job is run in a separate container. GitLab uses a separate software called GitLab Runner to manage the execution of these jobs.

1.7 About Stages

GitLab has a few predefined stages that are run in sequence.

• .pre
• build
• test
• deploy
• .post

You can override the default stage configuration by defining your own. The .pre and .post stages will still be run before and after all stages.
stages:

• my-build
• run-tests
• deploy

When defining a job refer to it’s stage.
job 1:
stage: build
script: make build dependencies

1.8 About Jobs

Runs a sequence of scripts. When a job starts a container image is booted up, the code repository is mounted as a volume and then the scripts are executed in that folder.
A job has status:

• pending – hasn’t started yet,
• running
• failed – a script of the job has failed.

You can set conditions that will restrict when a job can be run. For example, run a job only when code is pushed into the main branch.
If no stage is specified then test stage is used.

1.9 Writing Scripts

You can add one or more scripts to a job. Example:
build_job_1:
stage: build
script:

echo This is a simple build task
echo Jobs run in $(pwd) folder

If the script has special characters, it is safe to wrap it in single quotes. In the example below “:” can cause problem in YAML and needs to be wrapped.
script:

'echo Name: "Daffy Duck"'

If any script exits with a non-zero value the whole job will terminate as a failure.

1.10 Script Setup and Cleanup

You can run scripts before every job starts and after any job ends.
Example:

default:
before_script:

echo Global before script

echo Global after script
echo OK1

echo Local before script

echo Local after script

1.11 Choosing the Container Image

You need to select the right Docker container image to run the jobs. For example, to build and test a Node.js application use the “node:latest” image.
You can supply an image for all jobs by using the “image” top level property.

image: node:4.2.2

Individual jobs can override this and choose to run in a different image.

Default image
image: node:4.2.2
Per job image
build-app:
image: adoptopenjdk/openjdk11
script:
echo building app

1.12 GitLab Runner

GitLab Runner is a separate software that is used to execute the pipelines.
A runner uses a specific platform such as Docker or Kubernetes to run the jobs.

Basic steps to setup a runner:

• Install GitLab runner.
• Register the runner with your GitLab server.

1.13 Installing GitLab Runner

You have a couple options for installing GitLab Runner.

• Direct install – Directly install the software on a OS. For example, in Ubuntu you can run:

sudo apt-get install gitlab-runner

• In a Docker image – Sample command.

docker run -d --name gitlab-runner --restart always \
-v /srv/gitlab-runner/config:/etc/gitlab-runner \
-v /var/run/docker.sock:/var/run/docker.sock \
gitlab/gitlab-runner:latest

• In a Kubernetes cluster using Helm

1.14 Registering a Runner

First, acquire the registration token. Log into GitLab web console as a root admin user. Click Runners in the navigation menu.
Copy the registration token.

In GitLab, the runner machine runs this command to start the registration process.

sudo gitlab-runner register

You will be prompted to enter:

• The URL of the GitLab server.
• The registration token.
• The executor platform, such as Docker, Kubernetes or SS

1.15 Starting a Pipeline

Once the .gitlab-ci.yml file is added, every time you push anything to the project the pipeline is started.
You can also start the pipeline manually.

• In the GitLab console login as a project admin.
• Go to CI/CD > Pipelines.
• Click Run Pipeline.

1.16 Monitoring a Pipeline

In GitLab web console, go to the project. Go to CI/CD > Pipelines.
You can see the current status of the pipeline:

Click the status of the pipeline to see the status of the jobs.

Click a job to see the log output from the job.

1.17 Summary

• GitLab comes with CI/CD capabilities built-in.
• To define a CI/CD pipeline in GitLab just add the .gitlab-ci.yml file to the root of a project.
• Add stages, jobs and scripts to the file.
• To start a pipeline simply push any changes to the project.
• You need to install a runner software and register it with the GitLab server. Runner actually executes the jobs in a pipeline.