Skip to main content

CI/CD with GitLab

GitLab CI/CD is a powerful built-in continuous integration and continuous deployment tool that automates the software development lifecycle. It enables teams to build, test, and deploy code efficiently with automated pipelines defined in code.

Why GitLab CI/CD?

GitLab CI/CD offers an all-in-one DevOps platform with integrated version control, CI/CD, security scanning, and container registry - eliminating the need for multiple tools.

1. Overview

GitLab CI/CD provides:

  • Integrated Platform: Built directly into GitLab, no external tools needed
  • Pipeline as Code: Define pipelines using .gitlab-ci.yml in your repository
  • Docker Support: Native container and Docker integration
  • Kubernetes Integration: Deploy directly to Kubernetes clusters
  • Parallel Execution: Run jobs concurrently to speed up pipelines
  • Auto DevOps: Automated CI/CD configuration for common use cases
  • Security Scanning: Built-in SAST, DAST, dependency, and container scanning

Key Concepts

ConceptDescription
PipelineA collection of jobs organized in stages
StageA logical grouping of jobs (e.g., build, test, deploy)
JobIndividual tasks that execute scripts (e.g., compile code, run tests)
RunnerAgent that executes CI/CD jobs
ArtifactFiles generated by jobs and passed between stages
CacheDependencies stored to speed up subsequent pipeline runs

2. Architecture

┌─────────────────────────────────────────────────────────────┐
│ GitLab Server │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Git Repo │ │ Pipeline │ │ Registry │ │
│ │ │ │ Scheduler │ │ (Docker) │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
└─────────────────────────────────────────────────────────────┘


┌─────────────────────────┐
│ GitLab Runners │
│ ┌────────┐ ┌────────┐ │
│ │ Shell │ │ Docker │ │
│ └────────┘ └────────┘ │
│ ┌────────┐ ┌────────┐ │
│ │ K8s │ │ Custom │ │
│ └────────┘ └────────┘ │
└─────────────────────────┘


┌─────────────────────────┐
│ Deployment Targets │
│ • Servers │
│ • Kubernetes │
│ • Cloud Platforms │
└─────────────────────────┘

3. Getting Started

3.1 Prerequisites

  • GitLab account (GitLab.com or self-hosted)
  • Git repository
  • GitLab Runner (shared runners available on GitLab.com)

3.2 Basic Pipeline Configuration

Create a .gitlab-ci.yml file in your repository root:

.gitlab-ci.yml
# Define pipeline stages
stages:
- build
- test
- deploy

# Build job
build-job:
stage: build
script:
- echo "Building the application..."
- npm install
- npm run build
artifacts:
paths:
- dist/
expire_in: 1 hour

# Test job
test-job:
stage: test
script:
- echo "Running tests..."
- npm test

# Deploy job
deploy-job:
stage: deploy
script:
- echo "Deploying application..."
- ./deploy.sh
only:
- main

3.3 Pipeline Execution Flow

Commit/Push → Pipeline Triggered → Build Stage → Test Stage → Deploy Stage
↓ ↓ ↓
Build Job Test Jobs Deploy Job
↓ ↓ ↓
Artifacts Test Results Deployment

4. GitLab Runner Setup

4.1 Install GitLab Runner

# Download the binary
curl -L --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64

# Give it execute permissions
chmod +x /usr/local/bin/gitlab-runner

# Create GitLab CI user
useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash

# Install and run as service
gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner
gitlab-runner start

4.2 Register Runner

# Register runner interactively
gitlab-runner register

# Or use command line arguments
gitlab-runner register \
--non-interactive \
--url "https://gitlab.com/" \
--registration-token "YOUR_TOKEN" \
--executor "docker" \
--docker-image "alpine:latest" \
--description "docker-runner" \
--tag-list "docker,linux" \
--run-untagged="true" \
--locked="false"

4.3 Runner Executors

ExecutorUse CaseProsCons
ShellSimple scripts, direct host accessFast, simpleNo isolation, security risk
DockerContainerized buildsIsolated, reproducibleDocker dependency
KubernetesCloud-native, scalableAuto-scaling, efficientComplex setup
SSHRemote server deploymentFlexibleRequires SSH setup

5. Advanced Pipeline Configurations

5.1 Multi-Stage Pipeline with Dependencies

.gitlab-ci.yml
stages:
- build
- test
- security
- deploy

variables:
DOCKER_IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
KUBECONFIG: /etc/deploy/kubeconfig

# Build Docker image
build:
stage: build
image: docker:latest
services:
- docker:dind
before_script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
script:
- docker build -t $DOCKER_IMAGE .
- docker push $DOCKER_IMAGE
only:
- main
- develop
- tags

# Unit tests
unit-test:
stage: test
image: node:18
script:
- npm install
- npm run test:unit
coverage: '/Coverage: \d+\.\d+%/'
artifacts:
reports:
junit: test-results.xml
coverage_report:
coverage_format: cobertura
path: coverage/cobertura-coverage.xml

# Integration tests
integration-test:
stage: test
image: node:18
services:
- postgres:14
- redis:7
variables:
POSTGRES_DB: test_db
POSTGRES_USER: test_user
POSTGRES_PASSWORD: test_pass
script:
- npm install
- npm run test:integration

# SAST security scanning
sast:
stage: security
image: docker:latest
variables:
SAST_EXCLUDED_PATHS: spec,test,tests,tmp
script:
- echo "Running SAST scan..."
allow_failure: true

# Container scanning
container-scanning:
stage: security
image: docker:latest
services:
- docker:dind
script:
- docker pull $DOCKER_IMAGE
- echo "Scanning container for vulnerabilities..."
allow_failure: true

# Deploy to staging
deploy-staging:
stage: deploy
image: bitnami/kubectl:latest
environment:
name: staging
url: https://staging.example.com
script:
- kubectl config use-context staging
- kubectl set image deployment/myapp myapp=$DOCKER_IMAGE
- kubectl rollout status deployment/myapp
only:
- develop

# Deploy to production
deploy-production:
stage: deploy
image: bitnami/kubectl:latest
environment:
name: production
url: https://example.com
script:
- kubectl config use-context production
- kubectl set image deployment/myapp myapp=$DOCKER_IMAGE
- kubectl rollout status deployment/myapp
when: manual
only:
- main
- tags

5.2 Parallel Jobs

# Run tests in parallel
test:unit:
stage: test
parallel: 4
script:
- npm run test:unit -- --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL

test:e2e:
stage: test
parallel:
matrix:
- BROWSER: [chrome, firefox, safari]
PLATFORM: [linux, macos]
script:
- npm run test:e2e -- --browser=$BROWSER --platform=$PLATFORM

5.3 Dynamic Child Pipelines

# Parent pipeline
generate-config:
stage: build
script:
- python generate-pipeline.py > generated-pipeline.yml
artifacts:
paths:
- generated-pipeline.yml

trigger-child:
stage: test
trigger:
include:
- artifact: generated-pipeline.yml
job: generate-config
strategy: depend

5.4 Caching Strategies

# Global cache configuration
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- node_modules/
- .npm/
policy: pull

# Build job - push cache
build:
stage: build
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- node_modules/
policy: pull-push
script:
- npm ci
- npm run build

# Test jobs - pull only
test:
stage: test
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- node_modules/
policy: pull
script:
- npm test

6. Docker Integration

6.1 Building Docker Images

build-docker:
stage: build
image: docker:latest
services:
- docker:dind
variables:
DOCKER_TLS_CERTDIR: "/certs"
DOCKER_DRIVER: overlay2
before_script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
script:
# Build image
- docker build --cache-from $CI_REGISTRY_IMAGE:latest -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA .
- docker tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA $CI_REGISTRY_IMAGE:latest

# Push to registry
- docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
- docker push $CI_REGISTRY_IMAGE:latest
only:
- main

6.2 Multi-Stage Docker Build

Dockerfile
# Build stage
FROM node:18-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
RUN npm run build

# Production stage
FROM nginx:alpine
COPY --from=builder /app/dist /usr/share/nginx/html
COPY nginx.conf /etc/nginx/nginx.conf
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
build-optimized:
stage: build
image: docker:latest
services:
- docker:dind
script:
- docker build --target builder -t $CI_REGISTRY_IMAGE:builder .
- docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA .
- docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA

7. Kubernetes Deployment

7.1 Basic Kubernetes Deployment

deploy-k8s:
stage: deploy
image: bitnami/kubectl:latest
environment:
name: production
kubernetes:
namespace: myapp
before_script:
- echo $KUBE_CONFIG | base64 -d > $KUBECONFIG
script:
# Apply Kubernetes manifests
- kubectl apply -f k8s/namespace.yaml
- kubectl apply -f k8s/configmap.yaml
- kubectl apply -f k8s/secrets.yaml
- kubectl apply -f k8s/deployment.yaml
- kubectl apply -f k8s/service.yaml
- kubectl apply -f k8s/ingress.yaml

# Wait for rollout
- kubectl rollout status deployment/myapp -n myapp

# Verify deployment
- kubectl get pods -n myapp
only:
- main

7.2 Helm Chart Deployment

deploy-helm:
stage: deploy
image: alpine/helm:latest
environment:
name: production
before_script:
- echo $KUBE_CONFIG | base64 -d > $KUBECONFIG
script:
# Add Helm repository
- helm repo add myrepo https://charts.example.com
- helm repo update

# Deploy using Helm
- |
helm upgrade --install myapp myrepo/myapp \
--namespace myapp \
--create-namespace \
--set image.tag=$CI_COMMIT_SHA \
--set ingress.host=myapp.example.com \
--values values-prod.yaml \
--wait \
--timeout 5m
only:
- main

7.3 GitOps with ArgoCD

trigger-argocd:
stage: deploy
image: curlimages/curl:latest
script:
# Update image tag in Git repository
- git clone https://gitlab.com/myorg/gitops-repo.git
- cd gitops-repo
- sed -i "s|image:.*|image: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA|" apps/myapp/deployment.yaml
- git add apps/myapp/deployment.yaml
- git commit -m "Update myapp to $CI_COMMIT_SHA"
- git push

# Trigger ArgoCD sync (optional)
- |
curl -X POST \
-H "Authorization: Bearer $ARGOCD_TOKEN" \
https://argocd.example.com/api/v1/applications/myapp/sync

8. Environment Management

8.1 Multiple Environments

variables:
STAGING_URL: "https://staging.example.com"
PRODUCTION_URL: "https://example.com"

.deploy-template: &deploy-template
image: bitnami/kubectl:latest
script:
- kubectl set image deployment/myapp myapp=$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
- kubectl rollout status deployment/myapp

deploy-dev:
<<: *deploy-template
stage: deploy
environment:
name: development
url: https://dev.example.com
only:
- develop

deploy-staging:
<<: *deploy-template
stage: deploy
environment:
name: staging
url: $STAGING_URL
on_stop: stop-staging
only:
- develop

stop-staging:
stage: deploy
environment:
name: staging
action: stop
script:
- kubectl delete deployment myapp
when: manual

deploy-production:
<<: *deploy-template
stage: deploy
environment:
name: production
url: $PRODUCTION_URL
when: manual
only:
- main

8.2 Review Apps

review-app:
stage: deploy
image: bitnami/kubectl:latest
environment:
name: review/$CI_COMMIT_REF_NAME
url: https://$CI_ENVIRONMENT_SLUG.review.example.com
on_stop: stop-review
auto_stop_in: 1 week
script:
- |
kubectl create namespace review-$CI_ENVIRONMENT_SLUG || true
helm upgrade --install $CI_ENVIRONMENT_SLUG ./helm-chart \
--namespace review-$CI_ENVIRONMENT_SLUG \
--set image.tag=$CI_COMMIT_SHA \
--set ingress.host=$CI_ENVIRONMENT_SLUG.review.example.com
only:
- merge_requests

stop-review:
stage: deploy
image: bitnami/kubectl:latest
environment:
name: review/$CI_COMMIT_REF_NAME
action: stop
script:
- helm uninstall $CI_ENVIRONMENT_SLUG --namespace review-$CI_ENVIRONMENT_SLUG
- kubectl delete namespace review-$CI_ENVIRONMENT_SLUG
when: manual
only:
- merge_requests

9. Security and Secrets Management

9.1 Using CI/CD Variables

# Protected and masked variables in GitLab UI:
# Settings > CI/CD > Variables
# - DATABASE_URL (protected, masked)
# - API_KEY (protected, masked)
# - KUBE_CONFIG (file type, protected)

deploy:
stage: deploy
script:
- echo $DATABASE_URL | base64 > database-url.txt
- kubectl create secret generic app-secrets \
--from-literal=database-url=$DATABASE_URL \
--from-literal=api-key=$API_KEY \
--dry-run=client -o yaml | kubectl apply -f -
only:
- main

9.2 HashiCorp Vault Integration

.vault-secrets:
before_script:
- export VAULT_ADDR=https://vault.example.com
- export VAULT_TOKEN=$(vault write -field=token auth/gitlab/login role=myapp jwt=$CI_JOB_JWT)
- export DB_PASSWORD=$(vault kv get -field=password secret/myapp/database)
- export API_KEY=$(vault kv get -field=key secret/myapp/api)

deploy:
extends: .vault-secrets
stage: deploy
script:
- kubectl create secret generic app-secrets \
--from-literal=db-password=$DB_PASSWORD \
--from-literal=api-key=$API_KEY

9.3 SAST and Dependency Scanning

include:
- template: Security/SAST.gitlab-ci.yml
- template: Security/Dependency-Scanning.gitlab-ci.yml
- template: Security/Secret-Detection.gitlab-ci.yml
- template: Security/Container-Scanning.gitlab-ci.yml

# Customize SAST
sast:
variables:
SAST_EXCLUDED_PATHS: "spec,test,tests"
artifacts:
reports:
sast: gl-sast-report.json

# Container scanning
container_scanning:
variables:
CI_APPLICATION_REPOSITORY: $CI_REGISTRY_IMAGE
CI_APPLICATION_TAG: $CI_COMMIT_SHA

10. Monitoring and Notifications

10.1 Slack Notifications

.notify-slack: &notify-slack
after_script:
- |
if [ "$CI_JOB_STATUS" == "success" ]; then
COLOR="#36a64f"
STATUS="✅ Success"
else
COLOR="#ff0000"
STATUS="❌ Failed"
fi
curl -X POST -H 'Content-type: application/json' \
--data "{\"attachments\":[{\"color\":\"$COLOR\",\"title\":\"$STATUS: $CI_PROJECT_NAME\",\"text\":\"Pipeline $CI_PIPELINE_ID - Job: $CI_JOB_NAME\",\"fields\":[{\"title\":\"Branch\",\"value\":\"$CI_COMMIT_REF_NAME\",\"short\":true},{\"title\":\"Commit\",\"value\":\"$CI_COMMIT_SHORT_SHA\",\"short\":true}]}]}" \
$SLACK_WEBHOOK_URL

deploy-production:
<<: *notify-slack
stage: deploy
script:
- ./deploy.sh

10.2 Email Notifications

Configure in GitLab UI:

Settings > Integrations > Emails on push
Settings > Notifications > Pipeline events

10.3 Pipeline Badges

Add to your README.md:

[![Pipeline Status](https://gitlab.com/username/project/badges/main/pipeline.svg)](https://gitlab.com/username/project/-/commits/main)
[![Coverage](https://gitlab.com/username/project/badges/main/coverage.svg)](https://gitlab.com/username/project/-/commits/main)

11. Performance Optimization

11.1 Caching Best Practices

# Use separate cache for different jobs
variables:
CACHE_KEY: "$CI_COMMIT_REF_SLUG"

.cache-template:
cache:
key: "$CACHE_KEY-$CI_JOB_NAME"
paths:
- .npm/
- node_modules/

build:
extends: .cache-template
cache:
policy: pull-push
script:
- npm ci --cache .npm
- npm run build

test:
extends: .cache-template
cache:
policy: pull
script:
- npm test

11.2 Artifacts Optimization

build:
stage: build
script:
- npm run build
artifacts:
paths:
- dist/
# Only keep artifacts for 1 day
expire_in: 1 day
# Exclude unnecessary files
exclude:
- dist/**/*.map
- dist/**/*.test.js

11.3 Pipeline Efficiency

# Skip pipeline for documentation changes
workflow:
rules:
- if: $CI_COMMIT_MESSAGE =~ /\[skip ci\]/
when: never
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

# Run jobs only when needed
test:unit:
rules:
- changes:
- src/**/*
- tests/**/*
- package.json
script:
- npm test

12. Complete Example: Node.js Application

Here's a complete real-world example for a Node.js application:

.gitlab-ci.yml
image: node:18

stages:
- install
- lint
- test
- build
- security
- deploy

variables:
NPM_CONFIG_CACHE: "$CI_PROJECT_DIR/.npm"
DOCKER_IMAGE: "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA"

# Install dependencies
install-deps:
stage: install
cache:
key: "$CI_COMMIT_REF_SLUG"
paths:
- .npm/
- node_modules/
policy: pull-push
script:
- npm ci --cache .npm --prefer-offline
artifacts:
paths:
- node_modules/
expire_in: 1 hour

# Code linting
lint:
stage: lint
dependencies:
- install-deps
script:
- npm run lint

# Unit tests
test:unit:
stage: test
dependencies:
- install-deps
script:
- npm run test:unit
coverage: '/All files[^|]*\|[^|]*\s+([\d\.]+)/'
artifacts:
reports:
junit: junit.xml
coverage_report:
coverage_format: cobertura
path: coverage/cobertura-coverage.xml

# Integration tests
test:integration:
stage: test
services:
- postgres:14
- redis:7
variables:
POSTGRES_DB: test_db
POSTGRES_USER: test_user
POSTGRES_PASSWORD: test_pass
REDIS_URL: redis://redis:6379
dependencies:
- install-deps
script:
- npm run test:integration

# Build application
build:
stage: build
dependencies:
- install-deps
script:
- npm run build
artifacts:
paths:
- dist/
expire_in: 1 week

# Build Docker image
build:docker:
stage: build
image: docker:latest
services:
- docker:dind
before_script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
script:
- docker build -t $DOCKER_IMAGE .
- docker tag $DOCKER_IMAGE $CI_REGISTRY_IMAGE:latest
- docker push $DOCKER_IMAGE
- docker push $CI_REGISTRY_IMAGE:latest
only:
- main
- develop

# SAST scanning
sast:
stage: security
image: returntocorp/semgrep
script:
- semgrep --config=auto --json --output=gl-sast-report.json .
artifacts:
reports:
sast: gl-sast-report.json
allow_failure: true

# Deploy to staging
deploy:staging:
stage: deploy
image: bitnami/kubectl:latest
environment:
name: staging
url: https://staging.example.com
before_script:
- echo $KUBE_CONFIG_STAGING | base64 -d > $KUBECONFIG
script:
- kubectl set image deployment/myapp myapp=$DOCKER_IMAGE -n staging
- kubectl rollout status deployment/myapp -n staging
only:
- develop

# Deploy to production
deploy:production:
stage: deploy
image: bitnami/kubectl:latest
environment:
name: production
url: https://example.com
before_script:
- echo $KUBE_CONFIG_PROD | base64 -d > $KUBECONFIG
script:
- kubectl set image deployment/myapp myapp=$DOCKER_IMAGE -n production
- kubectl rollout status deployment/myapp -n production
when: manual
only:
- main

13. Troubleshooting

Common Issues

IssueCauseSolution
No runners availableNo runners registered/taggedRegister runner or use shared runners
Pipeline stuckRunner offline/busyCheck runner status, add more runners
Docker in Docker failsPrivileged mode not enabledEnable privileged mode in runner config
Cache not workingWrong cache keyUse consistent cache keys across jobs
Artifacts not foundJob dependency missingAdd dependencies or needs keyword
Slow pipelinesNo caching, sequential jobsAdd caching, use parallel jobs

Debugging Commands

debug:
stage: test
script:
- echo "CI_COMMIT_SHA=$CI_COMMIT_SHA"
- echo "CI_COMMIT_REF_NAME=$CI_COMMIT_REF_NAME"
- echo "CI_PROJECT_DIR=$CI_PROJECT_DIR"
- echo "CI_PIPELINE_ID=$CI_PIPELINE_ID"
- printenv | grep CI_
- ls -la
- pwd

14. Best Practices

✅ Do's

  1. Use specific image tags - Avoid latest tag for reproducibility
  2. Cache dependencies - Speed up pipeline execution
  3. Use artifacts wisely - Set expiration times
  4. Implement security scanning - Include SAST, dependency scanning
  5. Use protected variables - For sensitive data
  6. Define pipeline stages - Organize jobs logically
  7. Use manual gates - For production deployments
  8. Monitor pipeline metrics - Track success rate and duration

❌ Don'ts

  1. Don't hardcode secrets - Use CI/CD variables
  2. Don't use root user - In Docker containers
  3. Don't skip tests - On main/production branches
  4. Don't run everything on every commit - Use rules and changes
  5. Don't ignore failed jobs - Investigate and fix
  6. Don't deploy untagged images - Always use version tags
  7. Don't overcomplicate - Start simple, add complexity as needed

15. Migration from Other CI/CD Tools

From Jenkins

# Jenkins Pipeline equivalent
# Jenkinsfile:
# stage('Build') { sh 'npm run build' }
# stage('Test') { sh 'npm test' }
# stage('Deploy') { sh './deploy.sh' }

# GitLab CI/CD:
stages:
- build
- test
- deploy

build:
stage: build
script: npm run build

test:
stage: test
script: npm test

deploy:
stage: deploy
script: ./deploy.sh

From GitHub Actions

# GitHub Actions equivalent
# jobs:
# build:
# runs-on: ubuntu-latest
# steps:
# - uses: actions/checkout@v2
# - run: npm install
# - run: npm test

# GitLab CI/CD:
test:
image: node:18
script:
- npm install
- npm test

16. Resources

17. Next Steps

After mastering GitLab CI/CD basics, explore:

  • Auto DevOps - Automated CI/CD configuration
  • GitLab Container Registry - Built-in Docker registry
  • GitLab Kubernetes Agent - GitOps workflows
  • Merge Trains - Prevent merge conflicts
  • Review Apps - Preview changes before merging
  • Feature Flags - Progressive feature rollout
  • A/B Testing - Test different versions

Conclusion

GitLab CI/CD provides a comprehensive platform for automating your software delivery pipeline. By implementing the patterns and practices outlined in this guide, you can:

  • Automate build, test, and deployment workflows
  • Ensure code quality with automated testing and scanning
  • Deploy confidently to multiple environments
  • Scale your development process efficiently

Start with a simple pipeline and gradually add features as your team's needs evolve. Remember to:

  • Keep pipelines fast and efficient
  • Secure secrets and credentials properly
  • Monitor pipeline performance
  • Continuously improve based on team feedback

Happy automating! 🚀


Last Updated: 2025-10-19