CI/CD Pipelines

Automate everything from code to production, GitHub Actions, GitLab CI, Jenkins, and beyond

From Manual to Automated

Remember manually SSHing into servers, running tests, building Docker images, and deploying? That was yesterday. Modern development uses CI/CD (Continuous Integration / Continuous Deployment) to automate the entire journey from commit to production. Every push triggers automated tests, builds, security scans, and deployments, making releases faster, safer, and repeatable. This lesson covers the three major platforms (GitHub Actions, GitLab CI, Jenkins) and patterns that power modern software delivery.

CI/CD Fundamentals, The Pipeline

CI/CD is about automating the software delivery pipeline. Let's break down what this actually means.

Continuous Integration vs Continuous Deployment

Phase	What It Does	Goal
Continuous Integration (CI)	Automatically build and test every code change	Catch bugs early, ensure code quality
Continuous Delivery (CD)	Automatically prepare releases, manual deploy button	Always ready to deploy to production
Continuous Deployment (CD)	Automatically deploy to production after tests pass	Zero human intervention, maximum velocity

Typical Pipeline Stages

┌─────────────────────────────────────────────────────────────────────┐
│                     CI/CD PIPELINE FLOW                             │
└─────────────────────────────────────────────────────────────────────┘

1. CODE CHANGE
   Developer: git push
   ↓

2. BUILD STAGE
   ├─ Checkout code
   ├─ Install dependencies
   └─ Compile/build artifacts
   ↓

3. TEST STAGE
   ├─ Unit tests
   ├─ Integration tests
   ├─ Linting & code quality
   └─ Security scanning
   ↓

4. PACKAGE STAGE
   ├─ Build Docker image
   ├─ Push to registry
   └─ Tag with version
   ↓

5. DEPLOY STAGE (Staging)
   ├─ Deploy to staging environment
   ├─ Run smoke tests
   └─ Run E2E tests
   ↓

6. DEPLOY STAGE (Production)
   ├─ Deploy to production
   ├─ Health checks
   └─ Notify team

Total time: 5-15 minutes (faster = better)

Why CI/CD?

Speed

Deploy multiple times per day (vs weekly)
Automated = consistent, no human errors
Parallel testing reduces wait time
Fast feedback on broken builds

Quality

Every change is tested automatically
Security scans catch vulnerabilities
Code quality gates (coverage, linting)
Rollback is easy and automated

Collaboration

Shared pipeline visible to whole team
Blame-free: CI catches mistakes early
Documentation as code (pipeline config)
Standardized deployment process

Reliability

Smaller, frequent changes = less risk
Environment parity (dev = staging = prod)
Automated rollback on failures
Comprehensive test coverage

GitHub Actions, Native CI/CD for GitHub

GitHub Actions is built into GitHub. It's YAML-based, has a huge marketplace of reusable actions, and generous free tier for public repos.

Your First Workflow

# .github/workflows/ci.yml

name: CI Pipeline

# When to run
on:
  push:
    branches: [ main, develop ]
  pull_request:
    branches: [ main ]

# Jobs to run
jobs:
  test:
    runs-on: ubuntu-latest

    steps:
    # Checkout code
    - name: Checkout repository
      uses: actions/checkout@v4

    # Setup Node.js
    - name: Setup Node.js
      uses: actions/setup-node@v4
      with:
        node-version: '20'
        cache: 'npm'

    # Install dependencies
    - name: Install dependencies
      run: npm ci

    # Run tests
    - name: Run tests
      run: npm test

    # Run linter
    - name: Lint code
      run: npm run lint

What happens:
1. Push to main/develop or open PR
2. GitHub spins up Ubuntu VM
3. Runs each step sequentially
4. Reports success/failure on PR

Complete CI/CD Pipeline

# .github/workflows/deploy.yml

name: Build, Test, and Deploy

on:
  push:
    branches: [ main ]

env:
  DOCKER_IMAGE: mycompany/myapp
  KUBE_NAMESPACE: production

jobs:
  # Job 1: Build and Test
  build-and-test:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v4

    - name: Setup Node.js
      uses: actions/setup-node@v4
      with:
        node-version: '20'
        cache: 'npm'

    - name: Install dependencies
      run: npm ci

    - name: Run unit tests
      run: npm test -- --coverage

    - name: Upload coverage to Codecov
      uses: codecov/codecov-action@v5
      with:
        files: ./coverage/coverage-final.json

    - name: Run linter
      run: npm run lint

    - name: Build application
      run: npm run build

    - name: Upload build artifacts
      uses: actions/upload-artifact@v4
      with:
        name: dist
        path: dist/

  # Job 2: Security Scanning
  security-scan:
    runs-on: ubuntu-latest
    needs: build-and-test

    steps:
    - uses: actions/checkout@v4

    - name: Run Trivy vulnerability scanner
      uses: aquasecurity/trivy-action@master
      with:
        scan-type: 'fs'
        scan-ref: '.'
        severity: 'CRITICAL,HIGH'

  # Job 3: Build and Push Docker Image
  docker:
    runs-on: ubuntu-latest
    needs: [build-and-test, security-scan]

    steps:
    - uses: actions/checkout@v4

    - name: Set up Docker Buildx
      uses: docker/setup-buildx-action@v3

    - name: Login to Docker Hub
      uses: docker/login-action@v3
      with:
        username: ${{ secrets.DOCKER_USERNAME }}
        password: ${{ secrets.DOCKER_PASSWORD }}

    - name: Build and push
      uses: docker/build-push-action@v6
      with:
        context: .
        push: true
        tags: |
          ${{ env.DOCKER_IMAGE }}:latest
          ${{ env.DOCKER_IMAGE }}:${{ github.sha }}
        cache-from: type=registry,ref=${{ env.DOCKER_IMAGE }}:buildcache
        cache-to: type=registry,ref=${{ env.DOCKER_IMAGE }}:buildcache,mode=max

  # Job 4: Deploy to Kubernetes
  deploy:
    runs-on: ubuntu-latest
    needs: docker

    steps:
    - uses: actions/checkout@v4

    - name: Setup kubectl
      uses: azure/setup-kubectl@v3

    - name: Configure kubectl
      run: |
        echo "${{ secrets.KUBE_CONFIG }}" | base64 -d > kubeconfig
        export KUBECONFIG=./kubeconfig

    - name: Deploy to Kubernetes
      run: |
        kubectl set image deployment/myapp \
          myapp=${{ env.DOCKER_IMAGE }}:${{ github.sha }} \
          -n ${{ env.KUBE_NAMESPACE }}

        kubectl rollout status deployment/myapp \
          -n ${{ env.KUBE_NAMESPACE }}

    - name: Notify Slack
      uses: slackapi/slack-github-action@v1
      with:
        payload: |
          {
            "text": "Deployed ${{ github.sha }} to production!"
          }
      env:
        SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK }}

This pipeline:
1. Builds and tests code
2. Uploads coverage reports
3. Scans for security vulnerabilities
4. Builds Docker image with layer caching
5. Deploys to Kubernetes
6. Sends Slack notification

Matrix Builds (Test Multiple Versions)

# Test against multiple Node versions and OSes

name: Matrix Tests

on: [push]

jobs:
  test:
    runs-on: ${{ matrix.os }}

    strategy:
      matrix:
        os: [ubuntu-latest, windows-latest, macos-latest]
        node-version: [18, 20, 21]

    steps:
    - uses: actions/checkout@v4

    - name: Setup Node.js ${{ matrix.node-version }}
      uses: actions/setup-node@v4
      with:
        node-version: ${{ matrix.node-version }}

    - run: npm ci
    - run: npm test

# This creates 9 jobs (3 OS × 3 Node versions) running in parallel

Reusable Workflows

# .github/workflows/reusable-deploy.yml

name: Reusable Deploy

on:
  workflow_call:
    inputs:
      environment:
        required: true
        type: string
      image_tag:
        required: true
        type: string
    secrets:
      kube_config:
        required: true

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: ${{ inputs.environment }}

    steps:
    - name: Deploy to ${{ inputs.environment }}
      run: |
        kubectl set image deployment/myapp \
          myapp=mycompany/myapp:${{ inputs.image_tag }}

# .github/workflows/main.yml (uses the reusable workflow)

name: Deploy to Environments

on:
  push:
    branches: [main]

jobs:
  deploy-staging:
    uses: ./.github/workflows/reusable-deploy.yml
    with:
      environment: staging
      image_tag: ${{ github.sha }}
    secrets:
      kube_config: ${{ secrets.STAGING_KUBE_CONFIG }}

  deploy-production:
    needs: deploy-staging
    uses: ./.github/workflows/reusable-deploy.yml
    with:
      environment: production
      image_tag: ${{ github.sha }}
    secrets:
      kube_config: ${{ secrets.PROD_KUBE_CONFIG }}

GitHub Actions Features:
• 2,000 free minutes/month for private repos
• Unlimited for public repos
• Self-hosted runners for custom environments
• 20,000+ actions in marketplace
• Native GitHub integration (PR status checks, etc.)

GitLab CI, All-in-One DevOps Platform

GitLab CI is tightly integrated with GitLab's DevOps platform. It uses .gitlab-ci.yml and has powerful features like dynamic environments and review apps.

Basic Pipeline

# .gitlab-ci.yml

# Define stages (run sequentially)
stages:
  - build
  - test
  - deploy

# Define default behavior
default:
  image: node:20
  cache:
    paths:
      - node_modules/

# Build job
build:
  stage: build
  script:
    - npm ci
    - npm run build
  artifacts:
    paths:
      - dist/
    expire_in: 1 hour

# Test job
test:
  stage: test
  script:
    - npm ci
    - npm test
  coverage: '/Lines\s*:\s*(\d+\.\d+)%/'

# Lint job (runs in parallel with test)
lint:
  stage: test
  script:
    - npm ci
    - npm run lint

# Deploy to staging
deploy_staging:
  stage: deploy
  script:
    - kubectl set image deployment/myapp myapp=$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
  environment:
    name: staging
    url: https://staging.myapp.com
  only:
    - develop

# Deploy to production (manual trigger)
deploy_production:
  stage: deploy
  script:
    - kubectl set image deployment/myapp myapp=$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
  environment:
    name: production
    url: https://myapp.com
  when: manual
  only:
    - main

Production Pipeline with Docker

# .gitlab-ci.yml

stages:
  - build
  - test
  - security
  - package
  - deploy

variables:
  DOCKER_DRIVER: overlay2
  DOCKER_TLS_CERTDIR: "/certs"

# Build application
build_app:
  stage: build
  image: node:20
  script:
    - npm ci
    - npm run build
  artifacts:
    paths:
      - dist/
    expire_in: 1 day
  cache:
    key: ${CI_COMMIT_REF_SLUG}
    paths:
      - node_modules/

# Unit tests
unit_test:
  stage: test
  image: node:20
  script:
    - npm ci
    - npm run test:unit -- --coverage
  coverage: '/Lines\s*:\s*(\d+\.\d+)%/'
  artifacts:
    reports:
      coverage_report:
        coverage_format: cobertura
        path: coverage/cobertura-coverage.xml

# Integration tests
integration_test:
  stage: test
  image: node:20
  services:
    - postgres:15
    - redis:7
  variables:
    POSTGRES_DB: testdb
    POSTGRES_USER: testuser
    POSTGRES_PASSWORD: testpass
    DATABASE_URL: "postgresql://testuser:testpass@postgres:5432/testdb"
    REDIS_URL: "redis://redis:6379"
  script:
    - npm ci
    - npm run test:integration

# Security scanning
security_scan:
  stage: security
  image: aquasec/trivy:latest
  script:
    - trivy fs --severity HIGH,CRITICAL .
  allow_failure: true

# SAST (Static Application Security Testing)
sast:
  stage: security
  image: returntocorp/semgrep
  script:
    - semgrep --config=auto .
  allow_failure: true

# Build and push Docker image
docker_build:
  stage: package
  image: docker:24
  services:
    - docker:24-dind
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA .
    - docker tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA $CI_REGISTRY_IMAGE:latest
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
    - docker push $CI_REGISTRY_IMAGE:latest
  only:
    - main
    - develop

# Deploy to staging
deploy_staging:
  stage: deploy
  image: bitnami/kubectl:latest
  script:
    - kubectl config use-context ${KUBE_CONTEXT}
    - |
      kubectl set image deployment/myapp \
        myapp=$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA \
        -n staging
    - kubectl rollout status deployment/myapp -n staging
  environment:
    name: staging
    url: https://staging.myapp.com
    on_stop: stop_staging
  only:
    - develop

# Stop staging environment
stop_staging:
  stage: deploy
  image: bitnami/kubectl:latest
  script:
    - kubectl scale deployment/myapp --replicas=0 -n staging
  when: manual
  environment:
    name: staging
    action: stop

# Deploy to production
deploy_production:
  stage: deploy
  image: bitnami/kubectl:latest
  script:
    - kubectl config use-context ${KUBE_CONTEXT}
    - |
      kubectl set image deployment/myapp \
        myapp=$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA \
        -n production
    - kubectl rollout status deployment/myapp -n production
  environment:
    name: production
    url: https://myapp.com
  when: manual
  only:
    - main

Review Apps (Dynamic Environments)

# Create temporary environment for each merge request

review_app:
  stage: deploy
  script:
    - |
      kubectl create namespace review-$CI_MERGE_REQUEST_IID || true
      kubectl apply -f k8s/ -n review-$CI_MERGE_REQUEST_IID
      kubectl set image deployment/myapp \
        myapp=$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA \
        -n review-$CI_MERGE_REQUEST_IID
  environment:
    name: review/$CI_MERGE_REQUEST_IID
    url: https://review-$CI_MERGE_REQUEST_IID.myapp.com
    on_stop: stop_review
  only:
    - merge_requests

stop_review:
  stage: deploy
  script:
    - kubectl delete namespace review-$CI_MERGE_REQUEST_IID
  environment:
    name: review/$CI_MERGE_REQUEST_IID
    action: stop
  when: manual
  only:
    - merge_requests

Review Apps: Each merge request gets its own environment atreview-123.myapp.com. QA can test changes before merging. Environment auto-deleted when MR is closed.

Jenkins, The OG (Original Gangster) of CI/CD

Jenkins is the oldest and most flexible CI/CD tool. It's self-hosted, has 1000+ plugins, and uses Groovy-based Jenkinsfile for pipeline-as-code.

Declarative Pipeline

# Jenkinsfile

pipeline {
    agent any

    environment {
        DOCKER_IMAGE = "mycompany/myapp"
        DOCKER_REGISTRY = "https://registry.hub.docker.com"
        KUBE_NAMESPACE = "production"
    }

    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }

        stage('Build') {
            agent {
                docker {
                    image 'node:20'
                    reuseNode true
                }
            }
            steps {
                sh 'npm ci'
                sh 'npm run build'
                stash includes: 'dist/**', name: 'dist'
            }
        }

        stage('Test') {
            parallel {
                stage('Unit Tests') {
                    agent {
                        docker {
                            image 'node:20'
                            reuseNode true
                        }
                    }
                    steps {
                        sh 'npm ci'
                        sh 'npm test'
                    }
                    post {
                        always {
                            junit 'test-results/**/*.xml'
                            publishHTML([
                                reportDir: 'coverage',
                                reportFiles: 'index.html',
                                reportName: 'Coverage Report'
                            ])
                        }
                    }
                }

                stage('Lint') {
                    agent {
                        docker {
                            image 'node:20'
                            reuseNode true
                        }
                    }
                    steps {
                        sh 'npm ci'
                        sh 'npm run lint'
                    }
                }

                stage('Security Scan') {
                    steps {
                        sh 'trivy fs --severity HIGH,CRITICAL .'
                    }
                }
            }
        }

        stage('Build Docker Image') {
            steps {
                script {
                    docker.build("${DOCKER_IMAGE}:${BUILD_NUMBER}")
                    docker.build("${DOCKER_IMAGE}:latest")
                }
            }
        }

        stage('Push Docker Image') {
            steps {
                script {
                    docker.withRegistry(DOCKER_REGISTRY, 'docker-credentials') {
                        docker.image("${DOCKER_IMAGE}:${BUILD_NUMBER}").push()
                        docker.image("${DOCKER_IMAGE}:latest").push()
                    }
                }
            }
        }

        stage('Deploy to Staging') {
            steps {
                script {
                    kubernetesDeploy(
                        configs: 'k8s/staging/*.yaml',
                        kubeconfigId: 'staging-kubeconfig'
                    )
                }
            }
        }

        stage('Deploy to Production') {
            when {
                branch 'main'
            }
            steps {
                input message: 'Deploy to production?', ok: 'Deploy'
                script {
                    kubernetesDeploy(
                        configs: 'k8s/production/*.yaml',
                        kubeconfigId: 'prod-kubeconfig'
                    )
                }
            }
        }
    }

    post {
        success {
            slackSend(
                color: 'good',
                message: "Build #${BUILD_NUMBER} succeeded: ${BUILD_URL}"
            )
        }
        failure {
            slackSend(
                color: 'danger',
                message: "Build #${BUILD_NUMBER} failed: ${BUILD_URL}"
            )
        }
        always {
            cleanWs()
        }
    }
}

Scripted Pipeline (More Flexible)

# Jenkinsfile (Scripted)

node {
    def app
    def imageTag = "mycompany/myapp:${env.BUILD_NUMBER}"

    try {
        stage('Checkout') {
            checkout scm
        }

        stage('Build') {
            docker.image('node:20').inside {
                sh 'npm ci'
                sh 'npm run build'
            }
        }

        stage('Test') {
            docker.image('node:20').inside {
                sh 'npm test'
            }
        }

        stage('Docker Build') {
            app = docker.build(imageTag)
        }

        stage('Push to Registry') {
            docker.withRegistry('https://registry.hub.docker.com', 'docker-creds') {
                app.push("${env.BUILD_NUMBER}")
                app.push("latest")
            }
        }

        stage('Deploy') {
            if (env.BRANCH_NAME == 'main') {
                sh """
                    kubectl set image deployment/myapp \
                        myapp=${imageTag} \
                        -n production
                    kubectl rollout status deployment/myapp -n production
                """
            }
        }

        currentBuild.result = 'SUCCESS'
    } catch (Exception e) {
        currentBuild.result = 'FAILURE'
        throw e
    } finally {
        // Send notifications
        notifyBuild(currentBuild.result)
    }
}

def notifyBuild(String buildStatus = 'STARTED') {
    buildStatus = buildStatus ?: 'SUCCESS'

    def color = buildStatus == 'SUCCESS' ? 'good' : 'danger'
    def message = "${buildStatus}: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]'"

    slackSend(color: color, message: message)
}

Shared Libraries (DRY Pipelines)

# vars/buildNodeApp.groovy (shared library)

def call(Map config) {
    pipeline {
        agent any

        stages {
            stage('Build') {
                steps {
                    script {
                        docker.image('node:20').inside {
                            sh 'npm ci'
                            sh 'npm run build'
                        }
                    }
                }
            }

            stage('Test') {
                steps {
                    script {
                        docker.image('node:20').inside {
                            sh 'npm test'
                        }
                    }
                }
            }

            stage('Deploy') {
                when {
                    branch 'main'
                }
                steps {
                    script {
                        deployToKubernetes(
                            namespace: config.namespace,
                            deployment: config.deployment
                        )
                    }
                }
            }
        }
    }
}

# Jenkinsfile (uses shared library)

@Library('my-shared-library') _

buildNodeApp(
    namespace: 'production',
    deployment: 'myapp'
)

Jenkins Strengths:
• Self-hosted = full control
• 1000+ plugins for any integration
• Highly customizable with Groovy
• Mature ecosystem, battle-tested

Weaknesses:
• Requires maintenance (server, plugins)
• Steeper learning curve
• UI feels dated

Automated Testing, The Safety Net

CI/CD is worthless without comprehensive testing. Here's how to structure your test suite.

The Testing Pyramid

           ╱╲
          ╱  ╲ E2E Tests (5%)
         ╱────╲ Slow, expensive, brittle
        ╱      ╲ Test entire user flows
       ╱────────╲
      ╱          ╲ Integration Tests (15%)
     ╱────────────╲ Test component interactions
    ╱              ╲ Database, APIs, external services
   ╱────────────────╲
  ╱                  ╲ Unit Tests (80%)
 ╱────────────────────╲ Fast, cheap, reliable
╱______________________╲ Test individual functions

Goal: More unit tests, fewer E2E tests
Unit: milliseconds, E2E: minutes

Unit Tests

# Example: Jest (JavaScript)

// sum.js
export function sum(a, b) {
    return a + b;
}

// sum.test.js
import { sum } from './sum';

describe('sum', () => {
    test('adds 1 + 2 to equal 3', () => {
        expect(sum(1, 2)).toBe(3);
    });

    test('adds negative numbers', () => {
        expect(sum(-1, -2)).toBe(-3);
    });

    test('handles zero', () => {
        expect(sum(0, 5)).toBe(5);
    });
});

// Run: npm test
// CI: npm test -- --coverage --ci

Integration Tests

# Example: Testing API with database

import { describe, test, expect, beforeAll, afterAll } from '@jest/globals';
import request from 'supertest';
import app from './app';
import { db } from './database';

describe('User API Integration Tests', () => {
    beforeAll(async () => {
        await db.connect();
        await db.migrate();
    });

    afterAll(async () => {
        await db.cleanup();
        await db.disconnect();
    });

    test('POST /users creates user', async () => {
        const response = await request(app)
            .post('/users')
            .send({
                name: 'John Doe',
                email: 'john@example.com'
            });

        expect(response.status).toBe(201);
        expect(response.body).toHaveProperty('id');
        expect(response.body.name).toBe('John Doe');
    });

    test('GET /users/:id retrieves user', async () => {
        // Create user
        const createRes = await request(app)
            .post('/users')
            .send({ name: 'Jane', email: 'jane@example.com' });

        const userId = createRes.body.id;

        // Get user
        const getRes = await request(app).get(`/users/${userId}`);

        expect(getRes.status).toBe(200);
        expect(getRes.body.name).toBe('Jane');
    });
});

End-to-End Tests

# Example: Playwright (E2E testing)

import { test, expect } from '@playwright/test';

test.describe('User Registration Flow', () => {
    test('user can register and login', async ({ page }) => {
        // Go to registration page
        await page.goto('https://myapp.com/register');

        // Fill form
        await page.fill('[name="email"]', 'test@example.com');
        await page.fill('[name="password"]', 'SecurePass123!');
        await page.fill('[name="confirmPassword"]', 'SecurePass123!');

        // Submit
        await page.click('button[type="submit"]');

        // Verify redirect to dashboard
        await expect(page).toHaveURL('https://myapp.com/dashboard');

        // Verify welcome message
        await expect(page.locator('h1')).toContainText('Welcome');

        // Logout
        await page.click('[data-testid="logout-button"]');

        // Login again
        await page.goto('https://myapp.com/login');
        await page.fill('[name="email"]', 'test@example.com');
        await page.fill('[name="password"]', 'SecurePass123!');
        await page.click('button[type="submit"]');

        // Verify logged in
        await expect(page).toHaveURL('https://myapp.com/dashboard');
    });
});

// Run: npx playwright test
// CI: npx playwright test --reporter=junit

Testing in CI Pipeline

# GitHub Actions example

name: Test Suite

on: [push, pull_request]

jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
      - run: npm ci
      - run: npm run test:unit -- --coverage
      - uses: codecov/codecov-action@v5

  integration-tests:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:15
        env:
          POSTGRES_PASSWORD: testpass
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
      - run: npm ci
      - run: npm run test:integration
        env:
          DATABASE_URL: postgresql://postgres:testpass@postgres:5432/testdb

  e2e-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
      - run: npm ci
      - run: npx playwright install --with-deps
      - run: npm run build
      - run: npm run test:e2e
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: playwright-report
          path: playwright-report/

Testing Best Practices:
• Run fast tests first (fail fast)
• Parallelize tests across runners
• Keep E2E tests minimal and stable
• Require 80%+ code coverage
• Mock external dependencies in unit tests
• Use real services in integration tests

Deployment Strategies

How you deploy matters. Different strategies balance risk, speed, and complexity.

Strategy	How It Works	Pros	Cons
Rolling	Gradually replace old instances with new	Default K8s, zero downtime	Slow rollback, mixed versions live
Blue-Green	Run two identical envs, switch traffic	Instant rollback, test before switch	2x resources, DB migrations tricky
Canary	Roll out to small % of users first	Risk mitigation, gradual rollout	Complex, requires metrics
Recreate	Shut down old, start new	Simple, clean	Downtime

Automated Canary with Flagger

# Canary deployment with automated rollback

apiVersion: flagger.app/v1beta1
kind: Canary
metadata:
  name: myapp
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: myapp
  service:
    port: 80
  analysis:
    interval: 1m
    threshold: 5
    maxWeight: 50
    stepWeight: 10
    metrics:
    # Success rate must be > 99%
    - name: request-success-rate
      thresholdRange:
        min: 99
      interval: 1m
    # Latency must be < 500ms
    - name: request-duration
      thresholdRange:
        max: 500
      interval: 1m
  webhooks:
  # Run load test during canary
  - name: load-test
    url: http://loadtester/
    timeout: 5s
    metadata:
      cmd: "hey -z 1m -q 10 -c 2 http://myapp/"

# Automated flow:
# 1. Deploy new version
# 2. Route 10% traffic to canary
# 3. Measure metrics for 1 min
# 4. If success rate > 99% and latency < 500ms:
#    Increase to 20%, repeat
# 5. If metrics fail: automatic rollback
# 6. If reaches 50%: promote to primary

Feature Flags (Decouple Deploy from Release)

# Deploy code with feature disabled, enable later

// Using LaunchDarkly (feature flag service)
import LaunchDarkly from 'launchdarkly-node-server-sdk';

const client = LaunchDarkly.init(process.env.LAUNCHDARKLY_SDK_KEY);

app.get('/checkout', async (req, res) => {
    const user = { key: req.user.id };

    // Check if new checkout flow is enabled for this user
    const useNewCheckout = await client.variation(
        'new-checkout-flow',
        user,
        false  // default value
    );

    if (useNewCheckout) {
        return renderNewCheckout(req, res);
    } else {
        return renderOldCheckout(req, res);
    }
});

// Benefits:
// 1. Deploy new code safely (disabled)
// 2. Enable for 5% of users via dashboard
// 3. Monitor metrics
// 4. Gradual rollout: 5% → 25% → 50% → 100%
// 5. Instant rollback (flip flag off)
// 6. A/B testing built-in

Production strategy: Use rolling deployments as default, canary for risky changes, feature flags for new features. This gives you speed + safety + control.

CI/CD Best Practices

Lessons learned from running pipelines at scale.

1. Security & Secrets Management

# ❌ NEVER commit secrets
DATABASE_URL=postgresql://user:password@host:5432/db

# ✅ Use CI/CD secrets
# GitHub: Settings → Secrets → New repository secret
# GitLab: Settings → CI/CD → Variables
# Jenkins: Credentials → Add credentials

# Access in pipeline:
steps:
  - name: Deploy
    env:
      DATABASE_URL: ${{ secrets.DATABASE_URL }}
      API_KEY: ${{ secrets.API_KEY }}
    run: ./deploy.sh

# Even better: Use secret managers
# - AWS Secrets Manager
# - HashiCorp Vault
# - Google Secret Manager

# Scan for leaked secrets:
- name: Secret scanning
  run: |
    docker run --rm -v "$(pwd):/src" trufflesecurity/trufflehog \
      filesystem /src --json

2. Speed Optimization

Cache Dependencies

# GitHub Actions
- uses: actions/setup-node@v4
  with:
    cache: 'npm'

# Saves 1-2 minutes per build

Parallelize Tests

strategy:
  matrix:
    test: [unit, integration, e2e]

# 3 jobs run simultaneously

Docker Layer Caching

# Copy package.json first
COPY package*.json ./
RUN npm ci
COPY . .

# npm layer cached if no pkg changes

Skip Unnecessary Jobs

# Only run on relevant changes
on:
  paths:
    - 'src/**'
    - '!**.md'

3. Pipeline Monitoring

# Track key metrics:
- Build duration (target: < 10 minutes)
- Deployment frequency (target: multiple times/day)
- Change failure rate (target: < 15%)
- Mean time to recovery (target: < 1 hour)

# Tools:
- Datadog CI Visibility
- Honeycomb for distributed tracing
- Custom metrics to Prometheus

# Alert on:
- Pipeline failures
- Slow builds (> 15 min)
- Failed deployments
- High test failure rate

4. Quality Gates

# Example: SonarQube integration

quality_gate:
  stage: test
  script:
    - sonar-scanner
    - |
      # Wait for quality gate result
      status=$(curl -s -u ${SONAR_TOKEN}: \
        "${SONAR_URL}/api/qualitygates/project_status?projectKey=myapp" \
        | jq -r '.projectStatus.status')

      if [ "$status" != "OK" ]; then
        echo "Quality gate failed!"
        exit 1
      fi

# Quality gate criteria:
# - Coverage > 80%
# - No critical bugs
# - Technical debt < 5%
# - No code smells (high severity)

Golden rules:
1. Keep pipelines fast (< 10 min ideal)
2. Fail fast (run cheap tests first)
3. Make failures obvious (Slack, email)
4. Keep main branch always deployable
5. Automate everything you run twice
6. Use feature flags for risky changes

Platform Comparison

Feature	GitHub Actions	GitLab CI	Jenkins
Hosting	Cloud (GitHub)	Cloud or Self-hosted	Self-hosted only
Configuration	YAML workflows	.gitlab-ci.yml	Jenkinsfile (Groovy)
Learning Curve	Easy	Easy-Medium	Medium-Hard
Marketplace	20,000+ actions	Limited	1,000+ plugins
Free Tier	2,000 min/month	400 min/month	N/A (self-hosted)
Best For	GitHub projects, startups	Full DevOps platform	Complex, custom needs

Bonus: Production-Grade Python CI/CD Blueprint

A ready-to-clone GitLab CI/CD template for Python services. It provides a five-stage pipeline (lint, build, test, security, compliance) wired up from day one, so you can focus on business logic instead of pipeline boilerplate.

The blueprint covers:

Multi-tool linting - ruff (linter + formatter), ty (Astral's fast type checker), mypy, pyright, and pylint all run in parallel on every push
Docker BuildKit secrets - private GitLab dependencies are installed via --mount=type=secret; the token is injected at build time and never stored in any image layer
100% test coverage enforcement - tests run inside the built container; pytest fails the job if coverage drops below 100%
Security scanning suite - bandit (Python SAST), pip-audit (CVE scanning against OSV database), Trivy (container image CVEs), kics (IaC misconfigurations), GitLab Secret Detection, and pip-licenses (license compliance)
Compliance gate - a final stage aggregates every security report with jq and fails the pipeline on any Critical, High, or Medium finding, or on disallowed licenses (GPL, AGPL, LGPL)

Explore the blueprint:gitlab.com/bytecode-solutions/templates/blueprint-python-cicd

Key Takeaways

CI/CD Pipeline: Build → Test → Package → Deploy, automated on every commit
GitHub Actions: YAML workflows, huge marketplace, free for public repos
GitLab CI: .gitlab-ci.yml, integrated platform, review apps for MRs
Jenkins: Jenkinsfile (Groovy), most flexible, requires self-hosting
Testing Pyramid: 80% unit, 15% integration, 5% E2E
Deployment Strategies: Rolling (default), blue-green (instant rollback), canary (gradual)
Security: Never commit secrets, scan for vulnerabilities, use secret managers
Speed: Cache dependencies, parallelize tests, Docker layer caching
Quality Gates: Enforce coverage, code quality, security standards
Feature Flags: Decouple deployment from feature release, safe rollouts

Back to "Advanced Kubernetes"

Infrastructure for EngineersLesson 7

Next: "Infrastructure as Code"