Day 21: Docker in CI/CD Pipelines

calendar_today July 12, 2025 schedule 12 min read

Learning Objectives: By the end of this lesson, you will be able to:

Integrate Docker into continuous integration workflows

Build automated testing strategies for containerized applications

Implement automated image building and deployment pipelines

Configure multi-stage deployments with Docker containers

Design efficient CI/CD workflows for container-based applications

1. Welcome and Introduction (5 minutes)

Icebreaker Activity

“Imagine you’re part of a development team that pushes code changes multiple times per day. Each change needs to be tested, built into a container, and deployed across multiple environments. Doing this manually would take hours and be error-prone. How would you automate this entire process to deploy changes safely and quickly? Today we’ll learn how Docker revolutionizes CI/CD pipelines.”

Lesson Overview

Duration: 90 minutes
Prerequisites: Docker fundamentals, basic CI/CD concepts, version control (Git)
Materials Needed: Docker Desktop, GitHub/GitLab account, CI/CD platform access

2. Learning Objectives and Session Outline (5 minutes)

What We’ll Cover Today:

CI/CD Fundamentals - Understanding continuous integration and deployment
Dockerized Testing - Running tests in consistent container environments
Automated Image Building - Building and tagging images in pipelines
Deployment Strategies - Blue-green, rolling, and canary deployments
Pipeline Optimization - Caching strategies and performance improvements

Success Criteria:

By the end of this session, you should be able to design and implement a complete CI/CD pipeline that automatically builds, tests, and deploys containerized applications with proper quality gates and deployment strategies.

3. Foundation Knowledge (15-20 minutes)

Core Concepts

CI/CD Pipeline Stages

A typical CI/CD pipeline for containerized applications includes:

Source: Code changes trigger the pipeline
Build: Compile application and create Docker images
Test: Run unit, integration, and security tests
Package: Tag and push images to registries
Deploy: Deploy containers to target environments
Monitor: Track deployment success and application health

Real-World Analogy: Think of a CI/CD pipeline like an automated factory assembly line. Raw materials (code) enter one end, go through various quality control stations (tests), get packaged (Docker images), and emerge as finished products (deployed applications) ready for customers.

Why This Matters: Automated pipelines reduce human error, increase deployment frequency, and provide consistent, repeatable processes for delivering software.

Benefits of Docker in CI/CD

Consistency: Same environment from development to production
Isolation: Tests run in clean, predictable environments
Portability: Deploy the same container to any environment
Scalability: Easily scale build agents and deployment targets
Rollback: Quick rollback using previous image versions

Pipeline Patterns

Common CI/CD patterns with Docker:

Branch-based Workflows: Different branches trigger different pipeline behaviors
GitFlow Integration: Automated releases based on Git workflow
Multi-environment Promotion: Progressive deployment through environments
Feature Toggles: Deploy features that can be enabled/disabled

Key Terminology

Continuous Integration (CI): Automatically building and testing code changes
Continuous Deployment (CD): Automatically deploying verified changes to production
Pipeline as Code: Defining CI/CD pipelines in version-controlled files
Artifact: Build output (Docker images, binaries, etc.)
Blue-Green Deployment: Running two identical production environments

4. Detailed Explanation and Examples (25-30 minutes)

Understanding CI/CD with Docker

Docker transforms CI/CD pipelines by providing consistent environments and simplifying deployment processes. Instead of deploying application files and configuring environments, you deploy pre-built, tested containers.

Pipeline Architecture

A typical Docker CI/CD architecture includes:

Source Control: Git repositories with Dockerfile and application code
CI/CD Platform: Jenkins, GitLab CI, GitHub Actions, or cloud services
Container Registry: Docker Hub, AWS ECR, Google GCR, or Azure ACR
Deployment Targets: Kubernetes clusters, Docker Swarm, or container services

Testing Strategies

Docker enables sophisticated testing strategies:

Unit Tests: Run in lightweight containers with minimal dependencies
Integration Tests: Use Docker Compose to test multi-service interactions
Security Tests: Scan images for vulnerabilities and configuration issues
Performance Tests: Load test containerized applications
Contract Tests: Verify API contracts between services

Image Building Strategies

Multi-stage Builds for CI/CD

Optimize builds for both development and production:

# Build stage
FROM node:16-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production

# Test stage
FROM builder AS tester
RUN npm ci
COPY . .
RUN npm test
RUN npm run lint
RUN npm audit --audit-level high

# Production stage
FROM node:16-alpine AS production
RUN addgroup -g 1001 -S nodejs && \
    adduser -S nextjs -u 1001 -G nodejs
WORKDIR /app
COPY --from=builder --chown=nextjs:nodejs /app/node_modules ./node_modules
COPY --chown=nextjs:nodejs . .
USER nextjs
CMD ["npm", "start"]

Image Tagging Strategies

Consistent tagging enables traceability and rollbacks:

Git Commit SHA: myapp:a1b2c3d for exact version tracking
Semantic Versioning: myapp:1.2.3 for release versions
Environment Tags: myapp:staging, myapp:production
Feature Branches: myapp:feature-auth for development

Deployment Patterns

Blue-Green Deployment

Maintain two identical production environments:

# Blue environment (current production)
version: '3.8'
services:
  app-blue:
    image: myapp:v1.0.0
    labels:
      - "environment=blue"
      - "version=v1.0.0"
    networks:
      - production

# Green environment (new version)
  app-green:
    image: myapp:v1.1.0
    labels:
      - "environment=green"
      - "version=v1.1.0"
    networks:
      - production
    deploy:
      replicas: 0  # Initially not serving traffic

networks:
  production:

Common Misconceptions

Misconception: CI/CD pipelines should be as fast as possible Reality: Quality gates and thorough testing are more important than speed
Misconception: Docker eliminates the need for environment-specific configuration Reality: Configuration management is still crucial for different environments

5. Live Coding Demonstration (20-25 minutes)

Example 1: GitHub Actions CI/CD Pipeline

# .github/workflows/ci-cd.yml
name: CI/CD Pipeline

on:
  push:
    branches: [ main, develop ]
  pull_request:
    branches: [ main ]

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}

jobs:
  test:
    runs-on: ubuntu-latest
    
    steps:
    - name: Checkout code
      uses: actions/checkout@v3

    - name: Set up Docker Buildx
      uses: docker/setup-buildx-action@v2

    - name: Build test image
      uses: docker/build-push-action@v3
      with:
        context: .
        target: tester
        load: true
        tags: ${{ env.IMAGE_NAME }}:test
        cache-from: type=gha
        cache-to: type=gha,mode=max

    - name: Run tests
      run: |
        docker run --rm ${{ env.IMAGE_NAME }}:test

    - name: Run security scan
      uses: aquasecurity/trivy-action@master
      with:
        image-ref: ${{ env.IMAGE_NAME }}:test
        format: 'sarif'
        output: 'trivy-results.sarif'

    - name: Upload security scan results
      uses: github/codeql-action/upload-sarif@v2
      with:
        sarif_file: 'trivy-results.sarif'

  build-and-push:
    needs: test
    runs-on: ubuntu-latest
    if: github.event_name == 'push'
    
    permissions:
      contents: read
      packages: write

    steps:
    - name: Checkout code
      uses: actions/checkout@v3

    - name: Set up Docker Buildx
      uses: docker/setup-buildx-action@v2

    - name: Log in to Container Registry
      uses: docker/login-action@v2
      with:
        registry: ${{ env.REGISTRY }}
        username: ${{ github.actor }}
        password: ${{ secrets.GITHUB_TOKEN }}

    - name: Extract metadata
      id: meta
      uses: docker/metadata-action@v4
      with:
        images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
        tags: |
          type=ref,event=branch
          type=ref,event=pr
          type=sha,prefix={{branch}}-
          type=raw,value=latest,enable={{is_default_branch}}

    - name: Build and push image
      uses: docker/build-push-action@v3
      with:
        context: .
        target: production
        push: true
        tags: ${{ steps.meta.outputs.tags }}
        labels: ${{ steps.meta.outputs.labels }}
        cache-from: type=gha
        cache-to: type=gha,mode=max

  deploy-staging:
    needs: build-and-push
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/develop'
    
    environment:
      name: staging
      url: https://staging.myapp.com

    steps:
    - name: Deploy to staging
      run: |
        echo "Deploying to staging environment"
        # Add deployment commands here

  deploy-production:
    needs: build-and-push
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'
    
    environment:
      name: production
      url: https://myapp.com

    steps:
    - name: Deploy to production
      run: |
        echo "Deploying to production environment"
        # Add deployment commands here

Example 2: GitLab CI/CD Pipeline

# .gitlab-ci.yml
stages:
  - test
  - build
  - deploy

variables:
  DOCKER_DRIVER: overlay2
  DOCKER_TLS_CERTDIR: "/certs"
  IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
  LATEST_TAG: $CI_REGISTRY_IMAGE:latest

# Test stage
test:
  stage: test
  image: docker:20.10.16
  services:
    - docker:20.10.16-dind
  before_script:
    - docker info
  script:
    - docker build --target tester -t $IMAGE_TAG-test .
    - docker run --rm $IMAGE_TAG-test
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

# Security scanning
security_scan:
  stage: test
  image: 
    name: aquasec/trivy:latest
    entrypoint: [""]
  script:
    - trivy image --exit-code 1 --severity HIGH,CRITICAL $IMAGE_TAG-test
  allow_failure: false
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

# Build and push image
build:
  stage: build
  image: docker:20.10.16
  services:
    - docker:20.10.16-dind
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - docker build --target production -t $IMAGE_TAG -t $LATEST_TAG .
    - docker push $IMAGE_TAG
    - docker push $LATEST_TAG
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

# Deploy to staging
deploy_staging:
  stage: deploy
  image: alpine:latest
  before_script:
    - apk add --no-cache curl
  script:
    - echo "Deploying $IMAGE_TAG to staging"
    - |
      curl -X POST \
        -H "Authorization: Bearer $DEPLOY_TOKEN" \
        -H "Content-Type: application/json" \
        -d "{\"image\": \"$IMAGE_TAG\", \"environment\": \"staging\"}" \
        $DEPLOY_WEBHOOK_URL
  environment:
    name: staging
    url: https://staging.myapp.com
  rules:
    - if: $CI_COMMIT_BRANCH == "develop"

# Deploy to production
deploy_production:
  stage: deploy
  image: alpine:latest
  before_script:
    - apk add --no-cache curl
  script:
    - echo "Deploying $IMAGE_TAG to production"
    - |
      curl -X POST \
        -H "Authorization: Bearer $DEPLOY_TOKEN" \
        -H "Content-Type: application/json" \
        -d "{\"image\": \"$IMAGE_TAG\", \"environment\": \"production\"}" \
        $DEPLOY_WEBHOOK_URL
  environment:
    name: production
    url: https://myapp.com
  when: manual
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

Example 3: Jenkins Pipeline with Docker

// Jenkinsfile
pipeline {
    agent any
    
    environment {
        DOCKER_REGISTRY = 'your-registry.com'
        IMAGE_NAME = 'myapp'
        DOCKER_CREDENTIALS = credentials('docker-registry-credentials')
    }
    
    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }
        
        stage('Test') {
            steps {
                script {
                    def testImage = docker.build("${IMAGE_NAME}:test", "--target tester .")
                    testImage.inside {
                        sh 'npm test'
                        sh 'npm run lint'
                    }
                }
            }
            post {
                always {
                    publishTestResults testResultsPattern: 'test-results.xml'
                    publishHTML([
                        allowMissing: false,
                        alwaysLinkToLastBuild: true,
                        keepAll: true,
                        reportDir: 'coverage',
                        reportFiles: 'index.html',
                        reportName: 'Coverage Report'
                    ])
                }
            }
        }
        
        stage('Security Scan') {
            steps {
                script {
                    sh """
                        docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
                        -v \$(pwd):/tmp aquasecurity/trivy:latest image \
                        --exit-code 1 --severity HIGH,CRITICAL ${IMAGE_NAME}:test
                    """
                }
            }
        }
        
        stage('Build Production Image') {
            when {
                anyOf {
                    branch 'main'
                    branch 'develop'
                }
            }
            steps {
                script {
                    def image = docker.build("${DOCKER_REGISTRY}/${IMAGE_NAME}:${env.BUILD_NUMBER}", "--target production .")
                    docker.withRegistry("https://${DOCKER_REGISTRY}", env.DOCKER_CREDENTIALS) {
                        image.push()
                        image.push("latest")
                    }
                }
            }
        }
        
        stage('Deploy to Staging') {
            when {
                branch 'develop'
            }
            steps {
                script {
                    sh """
                        docker-compose -f docker-compose.staging.yml pull
                        docker-compose -f docker-compose.staging.yml up -d
                    """
                }
            }
        }
        
        stage('Deploy to Production') {
            when {
                branch 'main'
            }
            steps {
                input message: 'Deploy to production?', ok: 'Deploy'
                script {
                    sh """
                        docker-compose -f docker-compose.production.yml pull
                        docker-compose -f docker-compose.production.yml up -d
                    """
                }
            }
        }
    }
    
    post {
        always {
            sh 'docker system prune -f'
        }
        failure {
            emailext (
                subject: "Pipeline Failed: ${env.JOB_NAME} - ${env.BUILD_NUMBER}",
                body: "Build failed. Check console output at ${env.BUILD_URL}",
                to: "${env.CHANGE_AUTHOR_EMAIL}"
            )
        }
    }
}

Example 4: Docker Compose for Testing

# docker-compose.test.yml
version: '3.8'

services:
  app:
    build:
      context: .
      target: tester
    environment:
      - NODE_ENV=test
      - DATABASE_URL=postgresql://test:test@database:5432/testdb
    depends_on:
      - database
      - redis
    volumes:
      - ./coverage:/app/coverage
    networks:
      - test-network

  database:
    image: postgres:13-alpine
    environment:
      - POSTGRES_DB=testdb
      - POSTGRES_USER=test
      - POSTGRES_PASSWORD=test
    tmpfs:
      - /var/lib/postgresql/data
    networks:
      - test-network

  redis:
    image: redis:6-alpine
    tmpfs:
      - /data
    networks:
      - test-network

  integration-tests:
    build:
      context: .
      dockerfile: Dockerfile.integration
    environment:
      - API_URL=http://app:3000
    depends_on:
      - app
    networks:
      - test-network
    volumes:
      - ./test-results:/app/test-results

networks:
  test-network:
    driver: bridge

6. Hands-On Practice (30-40 minutes)

Exercise 1: Build a Complete CI/CD Pipeline (20 minutes)

Objective: Create a full CI/CD pipeline for a Node.js application

Instructions:

Set up a GitHub repository with a sample Node.js application
Create a multi-stage Dockerfile with test and production targets
Configure GitHub Actions to run tests, build images, and deploy
Implement proper image tagging and security scanning
Set up deployment to staging and production environments

Starter Application Structure:

myapp/
├── package.json
├── package-lock.json
├── src/
│   ├── app.js
│   └── routes/
├── tests/
│   ├── unit/
│   └── integration/
├── Dockerfile
├── docker-compose.yml
├── docker-compose.test.yml
└── .github/
    └── workflows/
        └── ci-cd.yml

Sample package.json:

{
  "name": "myapp",
  "version": "1.0.0",
  "scripts": {
    "start": "node src/app.js",
    "test": "jest",
    "test:integration": "jest --config jest.integration.config.js",
    "lint": "eslint src/",
    "coverage": "jest --coverage"
  },
  "dependencies": {
    "express": "^4.18.0"
  },
  "devDependencies": {
    "jest": "^28.0.0",
    "eslint": "^8.0.0",
    "supertest": "^6.2.0"
  }
}

Exercise 2: Implement Blue-Green Deployment (15 minutes)

Objective: Set up automated blue-green deployment with Docker Compose

Requirements:

Two identical environments (blue and green)
Load balancer that can switch traffic between environments
Health checks to verify deployment success
Rollback mechanism in case of deployment failure

Hints:

Use Docker Compose override files for different environments
Implement health check endpoints in your application
Use labels to identify active/inactive environments

Exercise 3: Advanced Pipeline Optimization (5 minutes)

Objective: Optimize pipeline performance and reliability

Challenge: Improve an existing pipeline by:

Implementing intelligent caching strategies
Parallelizing independent build steps
Adding comprehensive monitoring and alerting
Implementing canary deployment patterns

Extension Ideas:

Add performance testing and benchmarking
Implement feature toggle integration
Create rollback automation based on metrics

7. Common Pitfalls and Troubleshooting (10 minutes)

Frequent Pipeline Issues

Problem: Docker builds are slow and consume too much time Cause: Inefficient Dockerfile layer ordering and lack of caching Solution: Optimize Dockerfile layers, use multi-stage builds, and implement proper caching
Problem: Tests fail in CI but pass locally Cause: Environment differences between local and CI systems Solution: Use identical Docker environments for local development and CI
Problem: Image vulnerabilities discovered after deployment Cause: Lack of security scanning in the pipeline Solution: Integrate vulnerability scanning as a mandatory pipeline step

Pipeline Debugging Techniques

# Debug failing builds
docker build --no-cache -t debug-image .
docker run -it debug-image /bin/sh

# Check build context size
docker build --no-cache --progress=plain .

# Analyze image layers
docker history image-name
dive image-name

# Debug pipeline secrets and environment variables
docker run --rm -e ENV_VAR=value image-name env

# Test pipeline steps locally
act -P ubuntu-latest=nektos/act-environments-ubuntu:18.04

Performance Optimization

Build Caching: Use Docker layer caching and external cache mounts
Parallel Execution: Run independent pipeline stages in parallel
Resource Management: Optimize runner resources and container limits
Image Size: Use minimal base images and multi-stage builds

8. Best Practices and Professional Tips (5 minutes)

Pipeline Design Principles

Fail Fast: Put quick, high-confidence tests first
Idempotency: Ensure pipeline runs are reproducible and consistent
Observability: Add comprehensive logging and monitoring
Security First: Integrate security checks throughout the pipeline
Environment Parity: Keep development, staging, and production environments identical

Image Management Strategy

Semantic Versioning: Use consistent versioning strategy
Registry Organization: Structure images logically in registries
Cleanup Policies: Implement automatic cleanup of old images
Security Scanning: Regular vulnerability scanning and updates

Deployment Best Practices

Rolling Updates: Minimize downtime with rolling deployments
Health Checks: Implement comprehensive health checking
Monitoring: Monitor deployment success and application metrics
Rollback Strategy: Plan and test rollback procedures
Blue-Green/Canary: Use advanced deployment patterns for critical applications

9. Summary and Review (5 minutes)

Key Takeaways

Automation: CI/CD pipelines eliminate manual deployment errors and speed up delivery
Consistency: Docker ensures identical environments across the entire pipeline
Quality Gates: Automated testing and security scanning prevent issues from reaching production
Deployment Strategies: Advanced deployment patterns enable safe, reliable releases

Quick Review Questions

What are the key stages of a Docker CI/CD pipeline?
How would you implement a rollback strategy for containerized applications?
What security considerations are important for CI/CD pipelines?

For Instructors

Teaching Tips

Use real CI/CD platforms and show live pipeline execution
Demonstrate both successful deployments and failure scenarios
Emphasize the importance of testing and quality gates
Show how CI/CD practices integrate with team workflows

Assessment Ideas

Build complete CI/CD pipelines for provided applications
Design deployment strategies for different application types
Troubleshoot broken pipeline configurations

Adaptation Notes

Beginners: Focus on basic pipeline concepts and simple GitHub Actions
Advanced: Cover advanced deployment patterns and enterprise CI/CD tools
Time Management: Balance hands-on pipeline building with conceptual understanding
Platform Choice: Adapt examples to the CI/CD platforms available in your environment