Day 15: Introduction to Docker Compose

calendar_today July 6, 2025 schedule 13 min read

Learning Objectives: By the end of this lesson, you will be able to:

Understand what Docker Compose is and why it’s essential for multi-container applications

Write basic docker-compose.yml files to define application stacks

Use Docker Compose commands to manage application lifecycles

Convert existing multi-container setups to Docker Compose configurations

1. Welcome and Introduction (5 minutes)

Icebreaker Activity

Remember yesterday when we manually created networks, set up containers one by one, and carefully configured all the connections? It felt like assembling furniture without instructions - lots of pieces, lots of steps, and lots of room for error. Today, we’ll learn Docker Compose, which is like having a detailed instruction manual that lets you build complex applications with a single command!

Lesson Overview

Duration: 50-60 minutes
Prerequisites: Completion of Day 14, understanding of multi-container applications and Docker networking
Materials Needed: Docker Desktop (includes Docker Compose), text editor, yesterday’s task manager application

2. Learning Objectives and Session Outline (5 minutes)

What We’ll Cover Today:

Docker Compose Fundamentals - What it is and how it simplifies container orchestration
YAML Configuration - Writing and understanding docker-compose.yml files
Service Definitions - Defining containers, networks, and volumes declaratively
Compose Commands - Managing application lifecycles with simple commands
Migration Practice - Converting yesterday’s manual setup to Compose

Success Criteria:

By the end of this session, you should be able to define, deploy, and manage complex multi-container applications using Docker Compose with confidence and best practices.

3. Foundation Knowledge (15-20 minutes)

What is Docker Compose?

Docker Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services, networks, and volumes. Then, with a single command, you create and start all the services from your configuration.

Real-World Analogy: If Docker containers are like individual musicians, Docker Compose is like a conductor’s sheet music. It tells each musician (container) when to start, what to play (configuration), and how to harmonize with others (networking), all orchestrated from a single score (docker-compose.yml).

Key Benefits:

Declarative Configuration: Define your entire application stack in code
Single Command Deployment: Start/stop entire applications with one command
Environment Consistency: Same configuration works across dev, staging, and production
Version Control: Your infrastructure as code alongside your application code

Compose vs Manual Container Management

Manual Approach (Yesterday’s Method)

# Create network
docker network create task-manager-net

# Start database
docker run -d --name task-db --network task-manager-net \
  -e POSTGRES_DB=taskmanager postgres:13

# Start backend
docker run -d --name task-api --network task-manager-net \
  -e DB_HOST=task-db backend:latest

# Start frontend
docker run -d --name task-frontend --network task-manager-net \
  -p 8080:80 frontend:latest

Docker Compose Approach

version: '3.8'
services:
  database:
    image: postgres:13
    environment:
      POSTGRES_DB: taskmanager
  
  backend:
    build: ./backend
    environment:
      DB_HOST: database
  
  frontend:
    build: ./frontend
    ports:
      - "8080:80"

# Single command to start everything
docker-compose up

Docker Compose Architecture

Core Components

Services: Container definitions (what to run)
Networks: How containers communicate
Volumes: Where to store persistent data
Secrets: Sensitive configuration data
Configs: Non-sensitive configuration files

File Structure

version: '3.8'  # Compose file format version

services:       # Container definitions
  service1:
    # Service configuration
  service2:
    # Service configuration

networks:       # Custom network definitions (optional)
  network1:
    # Network configuration

volumes:        # Named volume definitions (optional)
  volume1:
    # Volume configuration

Key Terminology

Service: A container definition in docker-compose.yml
Stack: The complete set of services defined in a Compose file
Project: A group of services defined by a Compose file (default: directory name)
Compose File: The YAML file defining services, networks, and volumes
Override File: Additional Compose files that modify the base configuration

4. Detailed Explanation and Examples (25-30 minutes)

Basic Service Definitions

Simple Web Application

version: '3.8'

services:
  web:
    image: nginx:alpine
    ports:
      - "80:80"
    volumes:
      - ./html:/usr/share/nginx/html:ro
    restart: unless-stopped

Database Service with Persistence

version: '3.8'

services:
  database:
    image: postgres:13-alpine
    environment:
      POSTGRES_DB: myapp
      POSTGRES_USER: appuser
      POSTGRES_PASSWORD: securepass
    volumes:
      - postgres_data:/var/lib/postgresql/data
    restart: unless-stopped

volumes:
  postgres_data:

Build vs Image Services

Using Pre-built Images

services:
  redis:
    image: redis:alpine
    command: redis-server --appendonly yes
    volumes:
      - redis_data:/data

Building from Dockerfile

services:
  api:
    build:
      context: ./backend
      dockerfile: Dockerfile
      args:
        NODE_ENV: production
    environment:
      - DATABASE_URL=postgresql://user:pass@database:5432/mydb

Advanced Build Configuration

services:
  app:
    build:
      context: .
      dockerfile: Dockerfile.prod
      target: production
      args:
        BUILD_VERSION: "1.2.3"
        ENVIRONMENT: production
      cache_from:
        - myapp:latest
    image: myapp:${TAG:-latest}

Environment Variables and Configuration

Environment Variables

services:
  web:
    image: myapp:latest
    environment:
      - NODE_ENV=production
      - API_KEY=secret123
      - DEBUG=false
    env_file:
      - .env
      - .env.production

Environment File (.env)

# .env file
NODE_ENV=development
DATABASE_URL=postgresql://localhost:5432/myapp
API_KEY=dev-api-key-123
LOG_LEVEL=debug

Networking in Docker Compose

Default Networking

# Docker Compose automatically creates a network
services:
  web:
    image: nginx
  api:
    image: myapi:latest
    # Can communicate with 'web' service by name

Custom Networks

version: '3.8'

services:
  web:
    image: nginx
    networks:
      - frontend
  
  api:
    image: myapi:latest
    networks:
      - frontend
      - backend
  
  database:
    image: postgres:13
    networks:
      - backend

networks:
  frontend:
    driver: bridge
  backend:
    driver: bridge
    internal: true  # No external access

Volume Management

Named Volumes

version: '3.8'

services:
  database:
    image: postgres:13
    volumes:
      - db_data:/var/lib/postgresql/data
  
  backup:
    image: postgres:13
    volumes:
      - db_data:/data:ro
      - ./backups:/backups
    command: pg_dump -h database -U postgres mydb > /backups/backup.sql

volumes:
  db_data:
    driver: local

Bind Mounts

services:
  development:
    image: node:18-alpine
    volumes:
      - .:/app                    # Current directory to /app
      - /app/node_modules         # Anonymous volume for node_modules
    working_dir: /app
    command: npm run dev

Service Dependencies and Startup Order

Basic Dependencies

services:
  web:
    image: nginx
    depends_on:
      - api
      - database
  
  api:
    build: ./backend
    depends_on:
      - database
  
  database:
    image: postgres:13

Health Check Dependencies

services:
  web:
    image: nginx
    depends_on:
      api:
        condition: service_healthy
  
  api:
    build: ./backend
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s

Common Misconceptions

Misconception: “depends_on waits for the service to be ready” Reality: depends_on only waits for the container to start, not for the application to be ready
Misconception: “Services can only communicate if they’re on the same network” Reality: Services in the same Compose file can communicate by default through the automatically created network
Misconception: “Docker Compose is only for development” Reality: Compose is widely used in production, especially with Docker Swarm and single-host deployments

5. Live Coding Demonstration (20-25 minutes)

Example 1: Converting Yesterday’s Task Manager to Compose

# docker-compose.yml
version: '3.8'

services:
  database:
    image: postgres:13-alpine
    environment:
      POSTGRES_DB: taskmanager
      POSTGRES_USER: taskuser
      POSTGRES_PASSWORD: taskpass
    volumes:
      - postgres_data:/var/lib/postgresql/data
      - ./database/init.sql:/docker-entrypoint-initdb.d/init.sql:ro
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U taskuser -d taskmanager"]
      interval: 10s
      timeout: 5s
      retries: 5
    restart: unless-stopped

  backend:
    build:
      context: ./backend
      dockerfile: Dockerfile
    environment:
      NODE_ENV: ${NODE_ENV:-development}
      PORT: 3001
      DB_HOST: database
      DB_PORT: 5432
      DB_NAME: taskmanager
      DB_USER: taskuser
      DB_PASSWORD: taskpass
    depends_on:
      database:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "node", "-e", "require('http').get('http://localhost:3001/health')"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 60s
    restart: unless-stopped

  frontend:
    build:
      context: ./frontend
      dockerfile: Dockerfile
    ports:
      - "${FRONTEND_PORT:-8080}:80"
    depends_on:
      - backend
    restart: unless-stopped

volumes:
  postgres_data:

networks:
  default:
    name: task-manager-net

# Create .env file for environment-specific configuration
cat > .env << 'EOF'
NODE_ENV=development
FRONTEND_PORT=8080
COMPOSE_PROJECT_NAME=taskmanager
EOF

# Start the entire application stack
docker-compose up -d

# Check status
docker-compose ps

# View logs
docker-compose logs -f backend

# Scale a service (if stateless)
docker-compose up -d --scale backend=2

Example 2: Development vs Production Configurations

Base Configuration (docker-compose.yml)

version: '3.8'

services:
  database:
    image: postgres:13-alpine
    environment:
      POSTGRES_DB: taskmanager
      POSTGRES_USER: taskuser
      POSTGRES_PASSWORD: taskpass
    volumes:
      - postgres_data:/var/lib/postgresql/data

  backend:
    build: ./backend
    environment:
      DB_HOST: database
      DB_NAME: taskmanager
      DB_USER: taskuser
      DB_PASSWORD: taskpass
    depends_on:
      - database

  frontend:
    build: ./frontend
    depends_on:
      - backend

volumes:
  postgres_data:

Development Override (docker-compose.override.yml)

version: '3.8'

services:
  backend:
    build:
      target: development
    environment:
      NODE_ENV: development
      LOG_LEVEL: debug
    volumes:
      - ./backend/src:/app/src:ro
    ports:
      - "3001:3001"  # Expose for debugging
    command: npm run dev

  frontend:
    ports:
      - "8080:80"
    volumes:
      - ./frontend/public:/usr/share/nginx/html:ro

  database:
    ports:
      - "5432:5432"  # Expose for local database tools

Production Override (docker-compose.prod.yml)

version: '3.8'

services:
  backend:
    build:
      target: production
    environment:
      NODE_ENV: production
      LOG_LEVEL: warn
    restart: unless-stopped
    deploy:
      resources:
        limits:
          memory: 512M
          cpus: '0.5'

  frontend:
    ports:
      - "80:80"
      - "443:443"
    restart: unless-stopped

  database:
    restart: unless-stopped
    deploy:
      resources:
        limits:
          memory: 1G
          cpus: '1.0'

# Development environment (uses override file automatically)
docker-compose up -d

# Production environment
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d

Example 3: Complex Application with Multiple Services

version: '3.8'

services:
  # Reverse proxy
  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
      - ./nginx/ssl:/etc/nginx/ssl:ro
    depends_on:
      - frontend
      - backend
    restart: unless-stopped

  # Frontend application
  frontend:
    build: ./frontend
    expose:
      - "80"
    restart: unless-stopped

  # Backend API
  backend:
    build: ./backend
    environment:
      DATABASE_URL: postgresql://taskuser:taskpass@database:5432/taskmanager
      REDIS_URL: redis://redis:6379
      JWT_SECRET: ${JWT_SECRET}
    expose:
      - "3001"
    depends_on:
      database:
        condition: service_healthy
      redis:
        condition: service_started
    restart: unless-stopped

  # Database
  database:
    image: postgres:13-alpine
    environment:
      POSTGRES_DB: taskmanager
      POSTGRES_USER: taskuser
      POSTGRES_PASSWORD: taskpass
    volumes:
      - postgres_data:/var/lib/postgresql/data
      - ./database/init.sql:/docker-entrypoint-initdb.d/init.sql:ro
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U taskuser -d taskmanager"]
      interval: 10s
      timeout: 5s
      retries: 5
    restart: unless-stopped

  # Redis cache
  redis:
    image: redis:alpine
    command: redis-server --appendonly yes
    volumes:
      - redis_data:/data
    restart: unless-stopped

  # Background job processor
  worker:
    build: ./backend
    environment:
      DATABASE_URL: postgresql://taskuser:taskpass@database:5432/taskmanager
      REDIS_URL: redis://redis:6379
    command: npm run worker
    depends_on:
      - database
      - redis
    restart: unless-stopped

volumes:
  postgres_data:
  redis_data:

networks:
  default:
    name: taskmanager

Common Compose Commands

# Start services
docker-compose up                 # Foreground
docker-compose up -d              # Background (detached)
docker-compose up --build         # Rebuild images before starting

# Stop services
docker-compose down               # Stop and remove containers
docker-compose down -v            # Also remove volumes
docker-compose stop               # Stop without removing

# View status and logs
docker-compose ps                 # Show running services
docker-compose logs               # Show logs from all services
docker-compose logs -f backend    # Follow logs from specific service

# Execute commands
docker-compose exec backend bash  # Open shell in backend container
docker-compose run backend npm test  # Run one-off command

# Scaling services
docker-compose up -d --scale backend=3

# Validate configuration
docker-compose config             # Parse and validate compose file
docker-compose config --services  # List all services

6. Hands-On Practice (25-30 minutes)

Exercise 1: Convert Manual Setup to Compose (15 minutes)

Objective: Take yesterday’s manually created task manager application and convert it to Docker Compose

Instructions:

Create a docker-compose.yml file that replicates yesterday’s setup
Include all three services: database, backend, frontend
Add proper health checks and dependencies
Test that the application works identically to the manual setup
Add environment variable configuration

Starter Template:

version: '3.8'

services:
  database:
    # TODO: Configure PostgreSQL service
    
  backend:
    # TODO: Configure backend API service
    
  frontend:
    # TODO: Configure frontend service

volumes:
  # TODO: Define named volumes

# Optional: Custom networks

Verification Commands:

docker-compose up -d
docker-compose ps
curl http://localhost:8080
docker-compose logs backend

Exercise 2: Multi-Environment Configuration (10 minutes)

Objective: Create development and production configurations using override files

Requirements:

Create a base docker-compose.yml with common configuration
Create docker-compose.override.yml for development with:
- Volume mounts for live code editing
- Exposed ports for debugging
- Development environment variables
Create docker-compose.prod.yml for production with:
- Resource limits
- Restart policies
- Production environment variables
- Security hardening

Test both configurations:

# Development (automatic override)
docker-compose up -d

# Production
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d

Exercise 3: Add Monitoring Stack (Optional Advanced)

Objective: Extend the application with monitoring services

Requirements: Add the following services to your Compose configuration:

Prometheus for metrics collection
Grafana for visualization
Node Exporter for system metrics
Configure the backend to expose metrics endpoint

Monitoring Services Template:

  prometheus:
    image: prom/prometheus:latest
    ports:
      - "9090:9090"
    volumes:
      - ./monitoring/prometheus.yml:/etc/prometheus/prometheus.yml:ro
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.path=/prometheus'

  grafana:
    image: grafana/grafana:latest
    ports:
      - "3000:3000"
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=admin
    volumes:
      - grafana_data:/var/lib/grafana

  node-exporter:
    image: prom/node-exporter:latest
    ports:
      - "9100:9100"

7. Common Pitfalls and Troubleshooting (10 minutes)

Frequent Issues and Solutions

Problem: Services can’t communicate with each other Cause: Incorrect service names or network configuration Solution: Use service names as hostnames and verify network connectivity

# Test service communication
docker-compose exec backend ping database

# Check network configuration
docker-compose exec backend nslookup database

Problem: Environment variables not working Cause: Incorrect .env file location or variable syntax Solution: Verify .env file is in same directory as docker-compose.yml

# Check environment variables
docker-compose config

# Debug specific service environment
docker-compose exec backend env | grep DB_

Problem: Volumes not persisting data Cause: Volume configuration or mounting issues Solution: Verify volume definitions and mount points

# Check volume configuration
docker-compose config | grep -A 5 volumes

# Inspect volume contents
docker-compose exec database ls -la /var/lib/postgresql/data

Problem: Build context issues Cause: Incorrect build context or Dockerfile location Solution: Verify build context and Dockerfile paths

# Correct build configuration
services:
  backend:
    build:
      context: ./backend     # Directory containing Dockerfile
      dockerfile: Dockerfile # Dockerfile name (optional if default)

Debugging Compose Applications

# Validate Compose file syntax
docker-compose config

# View effective configuration
docker-compose config --services
docker-compose config --volumes

# Check service dependencies
docker-compose ps
docker-compose top

# Debug startup issues
docker-compose up --no-deps backend  # Start without dependencies
docker-compose logs --follow backend

# Resource usage monitoring
docker-compose exec backend ps aux
docker stats $(docker-compose ps -q)

Performance Troubleshooting

# Check container resource usage
docker-compose exec backend top
docker-compose exec backend free -h

# Analyze build performance
docker-compose build --no-cache --progress=plain

# Network performance testing
docker-compose exec frontend ping backend
docker-compose exec backend curl -w "@curl-format.txt" http://database:5432

8. Best Practices and Professional Tips (10 minutes)

Industry Standards

Compose File Organization

# Use consistent ordering and grouping
version: '3.8'

# Services grouped by tier
services:
  # Frontend tier
  nginx:
    # Configuration
  frontend:
    # Configuration
  
  # Backend tier
  api:
    # Configuration
  worker:
    # Configuration
  
  # Data tier
  database:
    # Configuration
  redis:
    # Configuration

# Infrastructure
volumes:
  # Named volumes

networks:
  # Custom networks

Environment Management

# Use descriptive .env files
# .env.development
COMPOSE_PROJECT_NAME=taskmanager-dev
DATABASE_URL=postgresql://localhost:5432/taskmanager_dev
LOG_LEVEL=debug

# .env.production
COMPOSE_PROJECT_NAME=taskmanager-prod
DATABASE_URL=postgresql://prod-server:5432/taskmanager
LOG_LEVEL=warn

Security Best Practices

services:
  database:
    image: postgres:13-alpine
    environment:
      POSTGRES_PASSWORD_FILE: /run/secrets/db_password
    secrets:
      - db_password
    # Don't expose ports unless necessary
    # ports:
    #   - "5432:5432"  # Remove in production

secrets:
  db_password:
    file: ./secrets/db_password.txt

Health Checks and Monitoring

services:
  api:
    build: ./backend
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3001/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 60s
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"

Production Deployment Patterns

# Resource limits for production
services:
  api:
    build: ./backend
    deploy:
      resources:
        limits:
          memory: 512M
          cpus: '0.5'
        reservations:
          memory: 256M
          cpus: '0.25'
      restart_policy:
        condition: on-failure
        delay: 5s
        max_attempts: 3

Development Workflow Optimization

# Development-friendly configuration
services:
  api:
    build:
      context: ./backend
      target: development
    volumes:
      - ./backend:/app:ro
      - /app/node_modules  # Prevent host node_modules from overriding
    environment:
      - NODE_ENV=development
      - CHOKIDAR_USEPOLLING=true  # For file watching in containers

Next Steps in Your Learning Journey

Learn Docker Compose networking in depth
Explore Docker Compose with CI/CD pipelines
Study Docker Swarm mode for production orchestration
Practice with Kubernetes as next-level orchestration

9. Summary and Review (5 minutes)

Key Takeaways

Docker Compose simplifies multi-container orchestration - Single command deployment vs multiple manual steps
Declarative configuration is powerful - Infrastructure as code that’s version controlled and reproducible
Environment flexibility is crucial - Override files enable different configurations for different environments

Comparison: Before and After Compose

Manual Approach	Docker Compose
Multiple commands	Single command
Error-prone setup	Repeatable configuration
Hard to document	Self-documenting YAML
Manual networking	Automatic service discovery
Complex scaling	Simple scaling commands

Quick Review Questions

What’s the difference between docker-compose up and docker-compose up -d?
How do services communicate with each other in Compose?
What’s the purpose of override files in Docker Compose?

Answers:

-d runs containers in detached mode (background), without it runs in foreground showing logs
Services communicate using service names as hostnames through the automatically created network
Override files allow environment-specific configurations without duplicating the base configuration

For Instructors

Teaching Tips

Start with simple examples and gradually add complexity
Emphasize the comparison between manual and Compose approaches
Show real-time editing of Compose files and immediate effects
Use the task manager app from Day 14 as a consistent example

Assessment Ideas

Compose file writing exercises for different application types
Troubleshooting scenarios with broken Compose configurations
Environment migration projects (dev to production)
Performance optimization challenges using Compose features

Adaptation Notes

For beginners: Focus more on basic service definitions and simple commands
For advanced students: Introduce complex networking, secrets, and production patterns
Online delivery: Ensure students can follow YAML editing and command execution
Time management: YAML syntax can be tricky - allow extra time for debugging

Common Student Challenges

YAML indentation errors - Emphasize consistent spacing and validation
Service naming confusion - Clarify difference between container names and service names
Environment variable precedence - Explain the hierarchy of environment sources
Volume mounting paths - Common source of confusion between host and container paths