CI/CD with GitHub Actions
Learn how to automate your software development workflow with GitHub Actions, from testing and building to deploying applications.
Overview
GitHub Actions is a powerful CI/CD platform integrated directly into GitHub that allows you to automate your build, test, and deployment pipeline. You can create workflows that build and test every pull request to your repository, or deploy merged pull requests to production.
Key Benefits
- Native GitHub Integration: No third-party service needed
- Matrix Builds: Test across multiple OS and language versions simultaneously
- Extensive Marketplace: Thousands of pre-built actions
- Self-hosted Runners: Run workflows on your own infrastructure
- Free Tier: 2,000 minutes/month for private repositories (unlimited for public)
Core Concepts
1. Workflows
A workflow is an automated process defined in a YAML file within .github/workflows/ directory. Workflows can be triggered by various events.
2. Events
Events are specific activities that trigger a workflow:
push: When code is pushed to the repositorypull_request: When a PR is opened, synchronized, or reopenedschedule: On a cron scheduleworkflow_dispatch: Manual triggerrelease: When a release is created
3. Jobs
A workflow contains one or more jobs that run in parallel by default (or sequentially if configured). Each job runs in a fresh virtual environment.
4. Steps
Steps are individual tasks that execute commands or actions within a job.
5. Actions
Actions are reusable units of code that can be combined into steps. You can create custom actions or use community actions from the GitHub Marketplace.
6. Runners
Runners are servers that execute your workflows. GitHub provides hosted runners (Ubuntu, Windows, macOS) or you can host your own.
Getting Started
Basic Workflow Structure
Create .github/workflows/ci.yml:
name: CI Pipeline
on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Run a one-line script
run: echo "Hello, GitHub Actions!"
- name: Run a multi-line script
run: |
echo "Building the project..."
echo "Current directory: $(pwd)"
ls -la
Language-Specific Examples
Node.js CI/CD Pipeline
name: Node.js CI
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [18.x, 20.x, 22.x]
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run linter
run: npm run lint
- name: Run tests
run: npm test
- name: Build project
run: npm run build
- name: Upload coverage reports
uses: codecov/codecov-action@v3
with:
file: ./coverage/coverage-final.json
flags: unittests
name: codecov-umbrella
Python CI/CD Pipeline
name: Python CI
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.9', '3.10', '3.11', '3.12']
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install pytest pytest-cov flake8
- name: Lint with flake8
run: |
# Stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# Exit-zero treats all errors as warnings
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Run tests with pytest
run: |
pytest --cov=./ --cov-report=xml
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v3
with:
file: ./coverage.xml
Docker Build and Push
name: Docker Build & Push
on:
push:
branches: [ main ]
tags:
- 'v*'
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: myusername/myapp
tags: |
type=ref,event=branch
type=ref,event=pr
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
- name: Build and push Docker image
uses: docker/build-push-action@v5
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=registry,ref=myusername/myapp:buildcache
cache-to: type=registry,ref=myusername/myapp:buildcache,mode=max
Advanced Workflows
Multi-Job Pipeline with Dependencies
name: Advanced Pipeline
on:
push:
branches: [ main ]
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run linter
run: npm run lint
test:
runs-on: ubuntu-latest
needs: lint # Wait for lint job to complete
strategy:
matrix:
node-version: [18.x, 20.x]
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
- run: npm ci
- run: npm test
build:
runs-on: ubuntu-latest
needs: [lint, test] # Wait for both lint and test
steps:
- uses: actions/checkout@v4
- run: npm ci
- run: npm run build
- name: Upload build artifacts
uses: actions/upload-artifact@v4
with:
name: build-output
path: dist/
retention-days: 7
deploy:
runs-on: ubuntu-latest
needs: build
if: github.ref == 'refs/heads/main'
steps:
- name: Download build artifacts
uses: actions/download-artifact@v4
with:
name: build-output
path: dist/
- name: Deploy to production
run: |
echo "Deploying to production..."
# Add your deployment commands here
Conditional Execution
name: Conditional Workflow
on: [push, pull_request]
jobs:
check-changes:
runs-on: ubuntu-latest
outputs:
backend: ${{ steps.filter.outputs.backend }}
frontend: ${{ steps.filter.outputs.frontend }}
steps:
- uses: actions/checkout@v4
- uses: dorny/paths-filter@v2
id: filter
with:
filters: |
backend:
- 'backend/**'
frontend:
- 'frontend/**'
test-backend:
needs: check-changes
if: needs.check-changes.outputs.backend == 'true'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Test backend
run: |
cd backend
npm test
test-frontend:
needs: check-changes
if: needs.check-changes.outputs.frontend == 'true'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Test frontend
run: |
cd frontend
npm test
Matrix Builds with Multiple Dimensions
name: Matrix Build
on: [push]
jobs:
test:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
node-version: [18, 20, 22]
include:
- os: ubuntu-latest
node-version: 22
experimental: true
exclude:
- os: macos-latest
node-version: 18
steps:
- uses: actions/checkout@v4
- name: Setup Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
- name: Install and test
run: |
npm ci
npm test
continue-on-error: ${{ matrix.experimental == true }}
Scheduled Workflows
name: Nightly Build
on:
schedule:
# Runs at 02:00 UTC every day
- cron: '0 2 * * *'
workflow_dispatch: # Allow manual trigger
jobs:
nightly:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run nightly tests
run: npm run test:integration
- name: Notify on failure
if: failure()
uses: actions/github-script@v7
with:
script: |
github.rest.issues.create({
owner: context.repo.owner,
repo: context.repo.repo,
title: 'Nightly build failed',
body: 'The nightly build failed. Please investigate.',
labels: ['bug', 'automated']
})
Deployment Examples
Deploy to GitHub Pages
name: Deploy to GitHub Pages
on:
push:
branches: [ main ]
permissions:
contents: read
pages: write
id-token: write
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Build
run: npm run build
- name: Upload artifact
uses: actions/upload-pages-artifact@v3
with:
path: ./dist
deploy:
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
needs: build
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
Deploy to AWS
name: Deploy to AWS
on:
push:
branches: [ main ]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-1
- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v2
- name: Build, tag, and push image to Amazon ECR
env:
ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
ECR_REPOSITORY: my-app
IMAGE_TAG: ${{ github.sha }}
run: |
docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
- name: Deploy to ECS
run: |
aws ecs update-service \
--cluster my-cluster \
--service my-service \
--force-new-deployment
Deploy to Vercel
name: Deploy to Vercel
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install Vercel CLI
run: npm install --global vercel@latest
- name: Pull Vercel Environment Information
run: vercel pull --yes --environment=production --token=${{ secrets.VERCEL_TOKEN }}
- name: Build Project Artifacts
run: vercel build --prod --token=${{ secrets.VERCEL_TOKEN }}
- name: Deploy Project Artifacts to Vercel
run: vercel deploy --prebuilt --prod --token=${{ secrets.VERCEL_TOKEN }}
Working with Secrets and Environment Variables
Using Secrets
name: Use Secrets
on: [push]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- name: Use secret in command
run: |
echo "Deploying with API key"
curl -H "Authorization: Bearer ${{ secrets.API_KEY }}" https://api.example.com
- name: Use secret in action
uses: some-action@v1
with:
api-key: ${{ secrets.API_KEY }}
Environment Variables
name: Environment Variables
on: [push]
env:
# Global environment variables
NODE_ENV: production
API_URL: https://api.example.com
jobs:
build:
runs-on: ubuntu-latest
env:
# Job-level environment variables
BUILD_ENV: staging
steps:
- name: Use environment variables
env:
# Step-level environment variables
CUSTOM_VAR: custom-value
run: |
echo "Node environment: $NODE_ENV"
echo "API URL: $API_URL"
echo "Build environment: $BUILD_ENV"
echo "Custom variable: $CUSTOM_VAR"
Using Environments
name: Deployment with Environments
on:
push:
branches: [ main ]
jobs:
deploy-staging:
runs-on: ubuntu-latest
environment:
name: staging
url: https://staging.example.com
steps:
- name: Deploy to staging
run: |
echo "Deploying to staging"
echo "Using secret: ${{ secrets.STAGING_API_KEY }}"
deploy-production:
runs-on: ubuntu-latest
needs: deploy-staging
environment:
name: production
url: https://example.com
steps:
- name: Deploy to production
run: |
echo "Deploying to production"
echo "Using secret: ${{ secrets.PRODUCTION_API_KEY }}"
Caching Dependencies
Cache Node Modules
name: Cache Node Modules
on: [push]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm' # Built-in caching
- name: Install dependencies
run: npm ci
Custom Cache Configuration
name: Custom Cache
on: [push]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Cache dependencies
uses: actions/cache@v3
with:
path: |
~/.npm
node_modules
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-
- name: Install dependencies
run: npm ci
Cache Python Dependencies
name: Cache Python
on: [push]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: 'pip' # Built-in caching
- name: Install dependencies
run: pip install -r requirements.txt
Security Best Practices
1. Use Pinned Action Versions
# ❌ Bad - uses latest version
- uses: actions/checkout@v4
# ✅ Better - uses commit SHA
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
2. Minimize Secret Exposure
jobs:
deploy:
runs-on: ubuntu-latest
steps:
# ❌ Bad - exposes secret in environment
- name: Bad practice
env:
SECRET: ${{ secrets.MY_SECRET }}
run: |
echo $SECRET # Never echo secrets!
./deploy.sh
# ✅ Good - passes secret directly
- name: Good practice
run: |
./deploy.sh --token "${{ secrets.MY_SECRET }}"
3. Limit Workflow Permissions
name: Secure Workflow
on: [push]
permissions:
contents: read # Only read access to repository
pull-requests: write # Write access to PRs
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npm test
4. Use Environments for Protection Rules
name: Protected Deployment
on:
push:
branches: [ main ]
jobs:
deploy:
runs-on: ubuntu-latest
environment:
name: production
# Environment protection rules:
# - Required reviewers
# - Wait timer
# - Allowed branches
steps:
- name: Deploy
run: ./deploy.sh
5. Scan for Vulnerabilities
name: Security Scan
on: [push, pull_request]
jobs:
security:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@master
with:
scan-type: 'fs'
scan-ref: '.'
format: 'sarif'
output: 'trivy-results.sarif'
- name: Upload Trivy results to GitHub Security tab
uses: github/codeql-action/upload-sarif@v2
with:
sarif_file: 'trivy-results.sarif'
Debugging and Troubleshooting
Enable Debug Logging
Add these secrets to your repository:
ACTIONS_RUNNER_DEBUG:true(for runner diagnostic logs)ACTIONS_STEP_DEBUG:true(for step debug logs)
Using tmate for Interactive Debugging
name: Debug with tmate
on: [push]
jobs:
debug:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup tmate session
if: failure() # Only on failure
uses: mxschmitt/action-tmate@v3
timeout-minutes: 30
Inspecting Context
name: Debug Context
on: [push]
jobs:
debug:
runs-on: ubuntu-latest
steps:
- name: Dump GitHub context
run: echo '${{ toJSON(github) }}'
- name: Dump runner context
run: echo '${{ toJSON(runner) }}'
- name: Dump job context
run: echo '${{ toJSON(job) }}'
Reusable Workflows
Creating a Reusable Workflow
.github/workflows/reusable-test.yml:
name: Reusable Test Workflow
on:
workflow_call:
inputs:
node-version:
required: true
type: string
environment:
required: false
type: string
default: 'development'
secrets:
NPM_TOKEN:
required: true
outputs:
test-result:
description: "Test execution result"
value: ${{ jobs.test.outputs.result }}
jobs:
test:
runs-on: ubuntu-latest
outputs:
result: ${{ steps.test.outcome }}
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ inputs.node-version }}
- name: Install dependencies
env:
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
run: npm ci
- name: Run tests
id: test
run: npm test
Using a Reusable Workflow
name: Call Reusable Workflow
on: [push]
jobs:
call-reusable:
uses: ./.github/workflows/reusable-test.yml
with:
node-version: '20'
environment: 'staging'
secrets:
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
Composite Actions
Creating a Composite Action
.github/actions/setup-node-cache/action.yml:
name: 'Setup Node with Cache'
description: 'Setup Node.js with dependency caching'
inputs:
node-version:
description: 'Node.js version'
required: true
default: '20'
runs:
using: 'composite'
steps:
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ inputs.node-version }}
cache: 'npm'
- name: Install dependencies
shell: bash
run: npm ci
- name: Print Node version
shell: bash
run: node --version
Using a Composite Action
name: Use Composite Action
on: [push]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Node with caching
uses: ./.github/actions/setup-node-cache
with:
node-version: '20'
- name: Build
run: npm run build
Performance Optimization
1. Use Concurrency Controls
name: Optimized Workflow
on: [push, pull_request]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true # Cancel old runs
jobs:
test:
runs-on: ubuntu-latest
steps:
- run: npm test
2. Parallelize Jobs
name: Parallel Execution
jobs:
unit-tests:
runs-on: ubuntu-latest
steps:
- run: npm run test:unit
integration-tests:
runs-on: ubuntu-latest
steps:
- run: npm run test:integration
e2e-tests:
runs-on: ubuntu-latest
steps:
- run: npm run test:e2e
3. Use Matrix Strategy Efficiently
name: Optimized Matrix
jobs:
test:
strategy:
fail-fast: false # Don't cancel other jobs on first failure
max-parallel: 4 # Limit concurrent jobs
matrix:
node-version: [18, 20, 22]
os: [ubuntu-latest, windows-latest]
Best Practices
1. Use Semantic Versioning for Actions
# ✅ Good - uses specific version
- uses: actions/checkout@v4.1.1
# ⚠️ Acceptable - uses major version
- uses: actions/checkout@v4
# ❌ Bad - uses latest
- uses: actions/checkout@main
2. Keep Workflows DRY (Don't Repeat Yourself)
name: DRY Workflow
on: [push]
jobs:
setup:
runs-on: ubuntu-latest
outputs:
node-version: ${{ steps.versions.outputs.node }}
steps:
- id: versions
run: echo "node=20" >> $GITHUB_OUTPUT
test:
needs: setup
runs-on: ubuntu-latest
steps:
- uses: actions/setup-node@v4
with:
node-version: ${{ needs.setup.outputs.node-version }}
3. Use Descriptive Names
# ✅ Good
- name: Install Node.js dependencies
run: npm ci
# ❌ Bad
- name: Install
run: npm ci
4. Add Timeout to Jobs
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 10 # Prevent hanging jobs
steps:
- run: npm test
5. Use Path Filters for Monorepos
name: Monorepo CI
on:
push:
paths:
- 'packages/api/**'
- 'packages/shared/**'
jobs:
api-tests:
runs-on: ubuntu-latest
steps:
- name: Test API
run: npm test --workspace=packages/api
Monitoring and Notifications
Slack Notification
name: Notify on Slack
on: [push]
jobs:
notify:
runs-on: ubuntu-latest
steps:
- name: Send Slack notification
uses: slackapi/slack-github-action@v1
with:
payload: |
{
"text": "Deployment to production completed!",
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "Build: <${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}|#${{ github.run_number }}>"
}
}
]
}
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
Email Notification on Failure
name: Email on Failure
on: [push]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- name: Deploy
run: ./deploy.sh
- name: Send email on failure
if: failure()
uses: dawidd6/action-send-mail@v3
with:
server_address: smtp.gmail.com
server_port: 465
username: ${{ secrets.EMAIL_USERNAME }}
password: ${{ secrets.EMAIL_PASSWORD }}
subject: Deployment Failed
to: team@example.com
from: GitHub Actions
body: The deployment workflow failed. Check the logs for details.
Common Pitfalls and Solutions
1. Workflow Not Triggering
Problem: Workflow doesn't run after push.
Solutions:
- Ensure workflow file is in
.github/workflows/ - Check YAML syntax with a validator
- Verify branch names match trigger conditions
- Check if workflow is disabled in repository settings
2. Permission Denied Errors
Problem: Permission denied when accessing resources.
Solution: Configure proper permissions:
permissions:
contents: write
pull-requests: write
issues: write
3. Secrets Not Working
Problem: Secrets are undefined in workflow.
Solutions:
- Verify secret names match exactly (case-sensitive)
- Check if running in a forked PR (secrets not available)
- Ensure secrets are set at correct level (repo/organization/environment)
4. Rate Limiting
Problem: GitHub API rate limits exceeded.
Solution: Use GITHUB_TOKEN for authentication:
- name: Use GitHub token
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: gh api /user
Summary
This tutorial covered:
- Core Concepts: Workflows, jobs, steps, actions, and runners
- Language Examples: Node.js, Python, Docker CI/CD pipelines
- Advanced Features: Matrix builds, reusable workflows, composite actions
- Deployment: GitHub Pages, AWS, Vercel
- Security: Best practices for secrets, permissions, and vulnerability scanning
- Optimization: Caching, parallelization, and concurrency controls
- Monitoring: Notifications and debugging techniques
Next Steps
- Explore the GitHub Actions Marketplace
- Set up self-hosted runners
- Implement workflow templates for your organization
- Learn about GitHub Actions expressions
- Integrate with third-party CI/CD tools
Additional Resources
- GitHub Actions Documentation
- GitHub Actions Changelog
- Awesome Actions - Curated list of awesome actions
- GitHub Actions Toolkit - Build your own actions