Rate Limiting and Security

calendar_today January 7, 2024 schedule 17 min read

Learning Objectives: By the end of this lesson, you will be able to:

Implement sophisticated rate limiting algorithms using Redis

Secure Redis deployments with authentication, encryption, and network controls

Build DDoS protection and abuse prevention systems

Design API throttling mechanisms for different user tiers

Implement security monitoring and intrusion detection with Redis

1. Welcome and Introduction (5 minutes)

Icebreaker Activity

Have you ever wondered how APIs like Twitter handle millions of requests per second without being overwhelmed? Or how financial systems prevent fraudulent transactions in real-time? The answer lies in intelligent rate limiting and security systems, and Redis provides the perfect foundation for building these critical protections!

Lesson Overview

Duration: 90 minutes
Prerequisites: Lessons 1-6 completed, understanding of HTTP APIs and security concepts
Materials Needed: Go environment, Redis server, basic cryptography knowledge

2. Learning Objectives and Session Outline (5 minutes)

What We’ll Cover Today:

Rate Limiting Algorithms - Token bucket, sliding window, and distributed rate limiting
API Security Patterns - Authentication, authorization, and abuse prevention
Redis Security Hardening - Configuration, network security, and access controls
DDoS Protection - Large-scale attack mitigation using Redis
Fraud Detection - Real-time pattern analysis and anomaly detection
Security Monitoring - Logging, alerting, and incident response

Success Criteria:

By the end of this session, you should be able to build production-ready security and rate limiting systems that protect applications from abuse while maintaining performance.

3. Foundation Knowledge (15-20 minutes)

Core Concepts

Rate Limiting Algorithms

1. Token Bucket Algorithm

Fixed capacity bucket filled with tokens at a constant rate
Each request consumes one token
When bucket is empty, requests are rejected or delayed

2. Sliding Window Algorithm

Track requests within a time window that “slides” with time
More accurate than fixed windows
Prevents burst traffic at window boundaries

3. Fixed Window Algorithm

Simple counter reset at fixed intervals
Can allow bursts at window boundaries
Easier to implement and understand

Security Threat Landscape

Common Attack Vectors

Brute Force: Repeated login attempts
API Abuse: Excessive requests to exhaust resources
DDoS: Distributed denial of service attacks
Data Scraping: Unauthorized data harvesting
Credential Stuffing: Using leaked credentials across services

Key Terminology

Rate Limiting: Controlling the frequency of requests from clients
Throttling: Slowing down requests rather than blocking them
Circuit Breaker: Automatically failing requests during system overload
IP Reputation: Scoring IP addresses based on historical behavior
Geofencing: Restricting access based on geographic location

4. Detailed Explanation and Examples (25-30 minutes)

Rate Limiting Architecture Patterns

1. Application-Level Rate Limiting

[Client] → [Load Balancer] → [Rate Limiter] → [Application] → [Database]
                                   ↑
                            [Redis Counter Store]

2. Distributed Rate Limiting

[Client] → [API Gateway] → [Multiple App Instances] → [Shared Redis Store]
              ↓                      ↓
        [Rate Limiter]         [Rate Limiter]
              ↓                      ↓
        [Shared State in Redis Cluster]

3. Multi-Layer Security

[Internet] → [CDN/WAF] → [Rate Limiter] → [Auth Layer] → [App] → [Redis]
     ↓            ↓            ↓             ↓
[DDoS Protect] [Geo Block] [API Limit] [User Auth] [Data Security]

5. Live Coding Demonstration (20-25 minutes)

Example 1: Advanced Rate Limiting System

package main

import (
    "context"
    "crypto/sha256"
    "encoding/hex"
    "fmt"
    "log"
    "net/http"
    "strconv"
    "strings"
    "time"

    "github.com/gin-gonic/gin"
    "github.com/redis/go-redis/v9"
)

// RateLimitAlgorithm defines different rate limiting strategies
type RateLimitAlgorithm int

const (
    TokenBucket RateLimitAlgorithm = iota
    SlidingWindow
    FixedWindow
)

// RateLimitConfig holds rate limiting configuration
type RateLimitConfig struct {
    Algorithm    RateLimitAlgorithm
    Limit        int           // requests per window
    Window       time.Duration // time window
    Burst        int           // burst capacity for token bucket
    KeyGenerator func(*gin.Context) string
}

// RateLimiter implements sophisticated rate limiting
type RateLimiter struct {
    client *redis.Client
    ctx    context.Context
    config RateLimitConfig
}

// RateLimitResult contains the result of rate limit check
type RateLimitResult struct {
    Allowed   bool
    Remaining int
    ResetTime time.Time
    RetryAfter time.Duration
}

func NewRateLimiter(client *redis.Client, config RateLimitConfig) *RateLimiter {
    return &RateLimiter{
        client: client,
        ctx:    context.Background(),
        config: config,
    }
}

// CheckLimit evaluates if request should be allowed
func (rl *RateLimiter) CheckLimit(c *gin.Context) (*RateLimitResult, error) {
    key := rl.config.KeyGenerator(c)
    
    switch rl.config.Algorithm {
    case TokenBucket:
        return rl.checkTokenBucket(key)
    case SlidingWindow:
        return rl.checkSlidingWindow(key)
    case FixedWindow:
        return rl.checkFixedWindow(key)
    default:
        return nil, fmt.Errorf("unknown algorithm: %v", rl.config.Algorithm)
    }
}

// Token Bucket Implementation
func (rl *RateLimiter) checkTokenBucket(key string) (*RateLimitResult, error) {
    bucketKey := fmt.Sprintf("bucket:%s", key)
    lastRefillKey := fmt.Sprintf("refill:%s", key)
    
    now := time.Now().Unix()
    
    // Lua script for atomic token bucket operation
    script := `
        local bucket_key = KEYS[1]
        local refill_key = KEYS[2]
        local limit = tonumber(ARGV[1])
        local burst = tonumber(ARGV[2])
        local window = tonumber(ARGV[3])
        local now = tonumber(ARGV[4])
        
        -- Get current tokens and last refill time
        local tokens = tonumber(redis.call('GET', bucket_key) or burst)
        local last_refill = tonumber(redis.call('GET', refill_key) or now)
        
        -- Calculate tokens to add based on time elapsed
        local elapsed = now - last_refill
        local tokens_to_add = math.floor(elapsed * limit / window)
        tokens = math.min(burst, tokens + tokens_to_add)
        
        -- Check if request can be served
        if tokens > 0 then
            tokens = tokens - 1
            redis.call('SET', bucket_key, tokens, 'EX', window * 2)
            redis.call('SET', refill_key, now, 'EX', window * 2)
            return {1, tokens, now + window}
        else
            -- Calculate retry after time
            local retry_after = math.ceil((1 - tokens) * window / limit)
            return {0, 0, now + retry_after}
        end
    `
    
    result, err := rl.client.Eval(rl.ctx, script, 
        []string{bucketKey, lastRefillKey},
        rl.config.Limit, rl.config.Burst, int(rl.config.Window.Seconds()), now).Result()
    
    if err != nil {
        return nil, fmt.Errorf("token bucket script failed: %w", err)
    }
    
    res := result.([]interface{})
    allowed := res[0].(int64) == 1
    remaining := int(res[1].(int64))
    resetTime := time.Unix(res[2].(int64), 0)
    
    return &RateLimitResult{
        Allowed:    allowed,
        Remaining:  remaining,
        ResetTime:  resetTime,
        RetryAfter: time.Until(resetTime),
    }, nil
}

// Sliding Window Implementation
func (rl *RateLimiter) checkSlidingWindow(key string) (*RateLimitResult, error) {
    windowKey := fmt.Sprintf("window:%s", key)
    now := time.Now().UnixMilli()
    windowStart := now - rl.config.Window.Milliseconds()
    
    // Lua script for sliding window
    script := `
        local key = KEYS[1]
        local now = tonumber(ARGV[1])
        local window_start = tonumber(ARGV[2])
        local limit = tonumber(ARGV[3])
        local window_ms = tonumber(ARGV[4])
        
        -- Remove old entries
        redis.call('ZREMRANGEBYSCORE', key, 0, window_start)
        
        -- Count current requests
        local current = redis.call('ZCARD', key)
        
        if current < limit then
            -- Add current request
            redis.call('ZADD', key, now, now)
            redis.call('EXPIRE', key, math.ceil(window_ms / 1000))
            return {1, limit - current - 1, now + window_ms}
        else
            return {0, 0, now + window_ms}
        end
    `
    
    result, err := rl.client.Eval(rl.ctx, script,
        []string{windowKey},
        now, windowStart, rl.config.Limit, rl.config.Window.Milliseconds()).Result()
    
    if err != nil {
        return nil, fmt.Errorf("sliding window script failed: %w", err)
    }
    
    res := result.([]interface{})
    allowed := res[0].(int64) == 1
    remaining := int(res[1].(int64))
    resetTime := time.UnixMilli(res[2].(int64))
    
    return &RateLimitResult{
        Allowed:    allowed,
        Remaining:  remaining,
        ResetTime:  resetTime,
        RetryAfter: time.Until(resetTime),
    }, nil
}

// Fixed Window Implementation
func (rl *RateLimiter) checkFixedWindow(key string) (*RateLimitResult, error) {
    windowKey := fmt.Sprintf("fixed:%s", key)
    now := time.Now()
    window := now.Truncate(rl.config.Window)
    windowEnd := window.Add(rl.config.Window)
    
    // Atomic increment with expiration
    pipe := rl.client.Pipeline()
    incrCmd := pipe.Incr(rl.ctx, windowKey)
    pipe.ExpireAt(rl.ctx, windowKey, windowEnd)
    
    _, err := pipe.Exec(rl.ctx)
    if err != nil {
        return nil, fmt.Errorf("fixed window operation failed: %w", err)
    }
    
    count := int(incrCmd.Val())
    allowed := count <= rl.config.Limit
    remaining := rl.config.Limit - count
    if remaining < 0 {
        remaining = 0
    }
    
    return &RateLimitResult{
        Allowed:    allowed,
        Remaining:  remaining,
        ResetTime:  windowEnd,
        RetryAfter: time.Until(windowEnd),
    }, nil
}

// Middleware for rate limiting
func (rl *RateLimiter) Middleware() gin.HandlerFunc {
    return func(c *gin.Context) {
        result, err := rl.CheckLimit(c)
        if err != nil {
            log.Printf("Rate limit check failed: %v", err)
            c.JSON(http.StatusInternalServerError, gin.H{"error": "Rate limit check failed"})
            c.Abort()
            return
        }

        // Set rate limit headers
        c.Header("X-RateLimit-Limit", strconv.Itoa(rl.config.Limit))
        c.Header("X-RateLimit-Remaining", strconv.Itoa(result.Remaining))
        c.Header("X-RateLimit-Reset", strconv.FormatInt(result.ResetTime.Unix(), 10))

        if !result.Allowed {
            c.Header("Retry-After", strconv.Itoa(int(result.RetryAfter.Seconds())))
            c.JSON(http.StatusTooManyRequests, gin.H{
                "error":   "Rate limit exceeded",
                "message": fmt.Sprintf("Too many requests. Try again in %v", result.RetryAfter.Round(time.Second)),
            })
            c.Abort()
            return
        }

        c.Next()
    }
}

// Key generation strategies
func IPBasedKey(c *gin.Context) string {
    return fmt.Sprintf("ip:%s", c.ClientIP())
}

func UserBasedKey(c *gin.Context) string {
    userID := c.GetHeader("X-User-ID")
    if userID == "" {
        userID = "anonymous"
    }
    return fmt.Sprintf("user:%s", userID)
}

func APIKeyBasedKey(c *gin.Context) string {
    apiKey := c.GetHeader("X-API-Key")
    if apiKey == "" {
        return IPBasedKey(c)
    }
    
    // Hash API key for privacy
    hash := sha256.Sum256([]byte(apiKey))
    return fmt.Sprintf("api:%s", hex.EncodeToString(hash[:8]))
}

func EndpointBasedKey(c *gin.Context) string {
    userKey := UserBasedKey(c)
    endpoint := c.FullPath()
    return fmt.Sprintf("%s:%s", userKey, endpoint)
}

// Demo application with multiple rate limiters
func main() {
    client := redis.NewClient(&redis.Options{
        Addr: "localhost:6379",
    })
    defer client.Close()

    r := gin.Default()

    // Global IP-based rate limiter (stricter)
    globalLimiter := NewRateLimiter(client, RateLimitConfig{
        Algorithm:    SlidingWindow,
        Limit:        100, // 100 requests per minute
        Window:       time.Minute,
        KeyGenerator: IPBasedKey,
    })

    // User-based rate limiter (more generous for authenticated users)
    userLimiter := NewRateLimiter(client, RateLimitConfig{
        Algorithm:    TokenBucket,
        Limit:        200,  // 200 requests per minute
        Window:       time.Minute,
        Burst:        50,   // Allow bursts up to 50
        KeyGenerator: UserBasedKey,
    })

    // API key-based rate limiter (highest limits)
    apiLimiter := NewRateLimiter(client, RateLimitConfig{
        Algorithm:    FixedWindow,
        Limit:        1000, // 1000 requests per minute
        Window:       time.Minute,
        KeyGenerator: APIKeyBasedKey,
    })

    // Apply global rate limiting to all routes
    r.Use(globalLimiter.Middleware())

    // Public endpoints (IP-based limiting only)
    r.GET("/public/status", func(c *gin.Context) {
        c.JSON(http.StatusOK, gin.H{"status": "ok", "timestamp": time.Now()})
    })

    // User endpoints (additional user-based limiting)
    userGroup := r.Group("/user")
    userGroup.Use(userLimiter.Middleware())
    {
        userGroup.GET("/profile", func(c *gin.Context) {
            c.JSON(http.StatusOK, gin.H{"message": "User profile data"})
        })
        
        userGroup.POST("/update", func(c *gin.Context) {
            c.JSON(http.StatusOK, gin.H{"message": "Profile updated"})
        })
    }

    // API endpoints (highest limits with API key)
    apiGroup := r.Group("/api/v1")
    apiGroup.Use(apiLimiter.Middleware())
    {
        apiGroup.GET("/data", func(c *gin.Context) {
            c.JSON(http.StatusOK, gin.H{"data": "API response"})
        })
        
        apiGroup.POST("/process", func(c *gin.Context) {
            c.JSON(http.StatusOK, gin.H{"message": "Processing completed"})
        })
    }

    log.Println("Rate limiting server starting on :8080...")
    r.Run(":8080")
}

Example 2: Security and Fraud Detection System

package main

import (
    "context"
    "crypto/rand"
    "encoding/hex"
    "encoding/json"
    "fmt"
    "log"
    "net"
    "net/http"
    "strconv"
    "strings"
    "time"

    "github.com/gin-gonic/gin"
    "github.com/redis/go-redis/v9"
)

// SecurityEvent represents a security-related event
type SecurityEvent struct {
    Type        string                 `json:"type"`
    IP          string                 `json:"ip"`
    UserID      string                 `json:"user_id,omitempty"`
    Timestamp   time.Time              `json:"timestamp"`
    Details     map[string]interface{} `json:"details"`
    RiskScore   int                    `json:"risk_score"`
    Action      string                 `json:"action"`
}

// ThreatDetector analyzes patterns and detects threats
type ThreatDetector struct {
    client *redis.Client
    ctx    context.Context
}

func NewThreatDetector(client *redis.Client) *ThreatDetector {
    return &ThreatDetector{
        client: client,
        ctx:    context.Background(),
    }
}

// AnalyzeRequest evaluates incoming requests for threats
func (td *ThreatDetector) AnalyzeRequest(c *gin.Context) *SecurityEvent {
    ip := c.ClientIP()
    userAgent := c.GetHeader("User-Agent")
    path := c.Request.URL.Path
    method := c.Request.Method
    
    event := &SecurityEvent{
        Type:      "request_analysis",
        IP:        ip,
        Timestamp: time.Now(),
        Details: map[string]interface{}{
            "user_agent": userAgent,
            "path":       path,
            "method":     method,
        },
        RiskScore: 0,
        Action:    "allow",
    }

    // Check various threat indicators
    event.RiskScore += td.checkIPReputation(ip)
    event.RiskScore += td.checkUserAgent(userAgent)
    event.RiskScore += td.checkRequestPattern(ip, path)
    event.RiskScore += td.checkGeolocation(ip)
    event.RiskScore += td.checkBruteForce(ip, c)

    // Determine action based on risk score
    if event.RiskScore >= 80 {
        event.Action = "block"
    } else if event.RiskScore >= 50 {
        event.Action = "challenge"
    } else if event.RiskScore >= 20 {
        event.Action = "monitor"
    }

    // Log the event
    td.logSecurityEvent(event)

    return event
}

// Check IP reputation
func (td *ThreatDetector) checkIPReputation(ip string) int {
    key := fmt.Sprintf("ip_reputation:%s", ip)
    
    reputation, err := td.client.Get(td.ctx, key).Int()
    if err == redis.Nil {
        return 0 // Unknown IP, neutral score
    } else if err != nil {
        log.Printf("Failed to check IP reputation: %v", err)
        return 0
    }

    // Convert reputation score (0-100) to risk score
    return 100 - reputation
}

// Update IP reputation based on behavior
func (td *ThreatDetector) UpdateIPReputation(ip string, delta int) {
    key := fmt.Sprintf("ip_reputation:%s", ip)
    
    // Use Lua script for atomic update with bounds
    script := `
        local key = KEYS[1]
        local delta = tonumber(ARGV[1])
        local current = tonumber(redis.call('GET', key) or 50)
        local new_score = math.max(0, math.min(100, current + delta))
        redis.call('SET', key, new_score, 'EX', 86400 * 7) -- 7 days expiry
        return new_score
    `
    
    _, err := td.client.Eval(td.ctx, script, []string{key}, delta).Result()
    if err != nil {
        log.Printf("Failed to update IP reputation: %v", err)
    }
}

// Check suspicious user agents
func (td *ThreatDetector) checkUserAgent(userAgent string) int {
    if userAgent == "" {
        return 30 // Missing user agent is suspicious
    }

    // Check for bot patterns
    botPatterns := []string{"bot", "crawler", "spider", "scraper", "curl", "wget"}
    lowerUA := strings.ToLower(userAgent)
    
    for _, pattern := range botPatterns {
        if strings.Contains(lowerUA, pattern) {
            return 40
        }
    }

    // Check for suspicious patterns
    if len(userAgent) < 10 || len(userAgent) > 500 {
        return 25
    }

    return 0
}

// Check request patterns for anomalies
func (td *ThreatDetector) checkRequestPattern(ip, path string) int {
    patternKey := fmt.Sprintf("pattern:%s", ip)
    now := time.Now().Unix()
    
    // Track request frequency and patterns
    script := `
        local key = KEYS[1]
        local now = tonumber(ARGV[1])
        local path = ARGV[2]
        local window = 300 -- 5 minutes
        local window_start = now - window
        
        -- Remove old entries
        redis.call('ZREMRANGEBYSCORE', key, 0, window_start)
        
        -- Add current request
        redis.call('ZADD', key, now, now .. ':' .. path)
        redis.call('EXPIRE', key, window)
        
        -- Count total requests and unique paths
        local total_requests = redis.call('ZCARD', key)
        local all_entries = redis.call('ZRANGE', key, 0, -1)
        local unique_paths = {}
        
        for i, entry in ipairs(all_entries) do
            local request_path = string.match(entry, ':(.+)')
            unique_paths[request_path] = true
        end
        
        local path_count = 0
        for _ in pairs(unique_paths) do
            path_count = path_count + 1
        end
        
        return {total_requests, path_count}
    `
    
    result, err := td.client.Eval(td.ctx, script, []string{patternKey}, now, path).Result()
    if err != nil {
        log.Printf("Failed to check request pattern: %v", err)
        return 0
    }

    res := result.([]interface{})
    totalRequests := int(res[0].(int64))
    uniquePaths := int(res[1].(int64))

    riskScore := 0
    
    // High frequency requests
    if totalRequests > 100 {
        riskScore += 50
    } else if totalRequests > 50 {
        riskScore += 25
    }

    // Too many different paths (scanning behavior)
    if uniquePaths > 20 {
        riskScore += 40
    } else if uniquePaths > 10 {
        riskScore += 20
    }

    return riskScore
}

// Simple geolocation check (normally would use GeoIP database)
func (td *ThreatDetector) checkGeolocation(ip string) int {
    // Check if IP is in suspicious ranges
    suspiciousRanges := []string{
        "10.0.0.0/8",     // Private networks (shouldn't be external)
        "172.16.0.0/12",  // Private networks
        "192.168.0.0/16", // Private networks
    }

    parsedIP := net.ParseIP(ip)
    if parsedIP == nil {
        return 30 // Invalid IP format
    }

    for _, cidr := range suspiciousRanges {
        _, network, err := net.ParseCIDR(cidr)
        if err != nil {
            continue
        }
        
        if network.Contains(parsedIP) {
            return 60 // Suspicious private IP in external request
        }
    }

    return 0
}

// Check for brute force attacks
func (td *ThreatDetector) checkBruteForce(ip string, c *gin.Context) int {
    // Only check login-related endpoints
    path := c.Request.URL.Path
    if !strings.Contains(path, "login") && !strings.Contains(path, "auth") {
        return 0
    }

    key := fmt.Sprintf("bruteforce:%s", ip)
    
    // Count failed attempts in the last hour
    attempts, err := td.client.Get(td.ctx, key).Int()
    if err == redis.Nil {
        return 0
    } else if err != nil {
        return 0
    }

    // Risk score based on failed attempts
    if attempts >= 20 {
        return 80
    } else if attempts >= 10 {
        return 50
    } else if attempts >= 5 {
        return 25
    }

    return 0
}

// Log failed login attempt
func (td *ThreatDetector) LogFailedLogin(ip string) {
    key := fmt.Sprintf("bruteforce:%s", ip)
    
    pipe := td.client.Pipeline()
    pipe.Incr(td.ctx, key)
    pipe.Expire(td.ctx, key, time.Hour)
    _, err := pipe.Exec(td.ctx)
    
    if err != nil {
        log.Printf("Failed to log brute force attempt: %v", err)
    }
}

// Log security events
func (td *ThreatDetector) logSecurityEvent(event *SecurityEvent) {
    eventJSON, err := json.Marshal(event)
    if err != nil {
        log.Printf("Failed to marshal security event: %v", err)
        return
    }

    // Store in Redis list for analysis
    eventsKey := "security_events"
    td.client.LPush(td.ctx, eventsKey, eventJSON)
    td.client.LTrim(td.ctx, eventsKey, 0, 9999) // Keep last 10k events

    // Store IP-specific events
    ipEventsKey := fmt.Sprintf("ip_events:%s", event.IP)
    td.client.LPush(td.ctx, ipEventsKey, eventJSON)
    td.client.Expire(td.ctx, ipEventsKey, 24*time.Hour)

    log.Printf("Security event: IP=%s, Risk=%d, Action=%s", 
               event.IP, event.RiskScore, event.Action)
}

// Security middleware
func (td *ThreatDetector) SecurityMiddleware() gin.HandlerFunc {
    return func(c *gin.Context) {
        event := td.AnalyzeRequest(c)

        switch event.Action {
        case "block":
            c.JSON(http.StatusForbidden, gin.H{
                "error": "Access denied",
                "message": "Your request has been blocked due to suspicious activity",
            })
            c.Abort()
            return

        case "challenge":
            // In a real system, this would trigger CAPTCHA or 2FA
            challengeToken := td.generateChallengeToken()
            c.Header("X-Challenge-Token", challengeToken)
            c.JSON(http.StatusUnauthorized, gin.H{
                "error": "Challenge required",
                "message": "Additional verification required",
                "challenge_token": challengeToken,
            })
            c.Abort()
            return

        case "monitor":
            // Add monitoring headers but allow request
            c.Header("X-Security-Level", "monitored")
        }

        // Store security context for downstream handlers
        c.Set("security_event", event)
        c.Next()
    }
}

func (td *ThreatDetector) generateChallengeToken() string {
    bytes := make([]byte, 16)
    rand.Read(bytes)
    return hex.EncodeToString(bytes)
}

// Demo application with security
func main() {
    client := redis.NewClient(&redis.Options{
        Addr: "localhost:6379",
    })
    defer client.Close()

    detector := NewThreatDetector(client)
    
    r := gin.Default()

    // Apply security middleware
    r.Use(detector.SecurityMiddleware())

    // Login endpoint with brute force protection
    r.POST("/login", func(c *gin.Context) {
        var credentials struct {
            Username string `json:"username"`
            Password string `json:"password"`
        }
        
        if err := c.ShouldBindJSON(&credentials); err != nil {
            c.JSON(http.StatusBadRequest, gin.H{"error": "Invalid request"})
            return
        }

        // Simulate authentication
        if credentials.Username != "admin" || credentials.Password != "password" {
            detector.LogFailedLogin(c.ClientIP())
            detector.UpdateIPReputation(c.ClientIP(), -10) // Decrease reputation
            c.JSON(http.StatusUnauthorized, gin.H{"error": "Invalid credentials"})
            return
        }

        // Successful login
        detector.UpdateIPReputation(c.ClientIP(), 5) // Increase reputation
        c.JSON(http.StatusOK, gin.H{"message": "Login successful"})
    })

    // Protected endpoint
    r.GET("/sensitive", func(c *gin.Context) {
        event := c.MustGet("security_event").(*SecurityEvent)
        c.JSON(http.StatusOK, gin.H{
            "message": "Sensitive data",
            "security_score": event.RiskScore,
        })
    })

    // Security dashboard endpoint
    r.GET("/security/events", func(c *gin.Context) {
        events, err := client.LRange(context.Background(), "security_events", 0, 99).Result()
        if err != nil {
            c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to fetch events"})
            return
        }

        c.JSON(http.StatusOK, gin.H{
            "events": events,
            "count":  len(events),
        })
    })

    log.Println("Security-enabled server starting on :8080...")
    r.Run(":8080")
}

6. Hands-On Practice (30-40 minutes)

Exercise 1: Multi-Tier Rate Limiting (15 minutes)

Objective: Implement different rate limits for different user tiers

Requirements:

Free tier: 100 requests/hour
Premium tier: 1000 requests/hour
Enterprise tier: 10000 requests/hour
Implement burst allowances and grace periods

Implementation Starter:

type UserTier string

const (
    FreeTier       UserTier = "free"
    PremiumTier    UserTier = "premium"
    EnterpriseTier UserTier = "enterprise"
)

type TierConfig struct {
    Limit      int
    Window     time.Duration
    Burst      int
    GracePeriod time.Duration
}

var TierConfigs = map[UserTier]TierConfig{
    FreeTier:       {Limit: 100, Window: time.Hour, Burst: 10, GracePeriod: 5 * time.Minute},
    PremiumTier:    {Limit: 1000, Window: time.Hour, Burst: 100, GracePeriod: time.Minute},
    EnterpriseTier: {Limit: 10000, Window: time.Hour, Burst: 500, GracePeriod: 30 * time.Second},
}

func TierBasedKey(c *gin.Context) (string, UserTier) {
    // TODO: Implement tier detection from API key or user session
    // TODO: Return appropriate key and tier
}

Exercise 2: DDoS Protection System (15 minutes)

Objective: Build automated DDoS detection and mitigation

Requirements:

Detect traffic spikes and unusual patterns
Implement automatic IP blocking
Create whitelist/blacklist management
Add captcha challenge system

Exercise 3: Fraud Detection Pipeline (10 minutes)

Objective: Real-time fraud scoring for transactions

Requirements:

Analyze transaction patterns
Detect velocity fraud (too many transactions)
Implement geolocation-based checks
Create risk scoring algorithm

7. Common Pitfalls and Troubleshooting (10 minutes)

Frequent Issues

False Positives in Rate Limiting
- Problem: Legitimate users getting blocked
- Solution: Implement grace periods and user feedback mechanisms
Redis Memory Issues with Security Data
- Problem: Security logs consuming too much memory
- Solution: Implement TTL and data rotation strategies
Performance Impact of Security Checks
- Problem: Security middleware slowing down requests
- Solution: Use Lua scripts and optimize Redis operations

Debugging Tips

// Monitor rate limiting effectiveness
func (rl *RateLimiter) GetStats(key string) map[string]interface{} {
    stats := make(map[string]interface{})
    
    // Get current state
    tokens, _ := rl.client.Get(rl.ctx, fmt.Sprintf("bucket:%s", key)).Int()
    stats["tokens"] = tokens
    
    // Get request history
    history, _ := rl.client.ZRange(rl.ctx, fmt.Sprintf("window:%s", key), 0, -1).Result()
    stats["recent_requests"] = len(history)
    
    return stats
}

// Security event analysis
func (td *ThreatDetector) AnalyzeIP(ip string) map[string]interface{} {
    analysis := make(map[string]interface{})
    
    // Get reputation
    reputation, _ := td.client.Get(td.ctx, fmt.Sprintf("ip_reputation:%s", ip)).Int()
    analysis["reputation"] = reputation
    
    // Get recent events
    events, _ := td.client.LRange(td.ctx, fmt.Sprintf("ip_events:%s", ip), 0, 9).Result()
    analysis["recent_events"] = len(events)
    
    return analysis
}

8. Best Practices and Professional Tips (5 minutes)

Production Security Checklist

✅ Multi-layer Rate Limiting: Different limits for different endpoints and user types
✅ Intelligent Blocking: Use reputation scoring instead of hard blocks
✅ Monitoring and Alerting: Track security metrics and anomalies
✅ Graceful Degradation: System should work even if Redis is down
✅ Regular Security Audits: Review and update security rules
✅ User Communication: Clear messages about rate limits and blocks
✅ Whitelist Management: Easy way to unblock legitimate users

Performance Optimization

// Batch security operations
func (td *ThreatDetector) BatchAnalyze(requests []*http.Request) []*SecurityEvent {
    pipe := td.client.Pipeline()
    
    // Queue all operations
    for _, req := range requests {
        // Add pipeline operations
    }
    
    // Execute all at once
    results, _ := pipe.Exec(td.ctx)
    
    // Process results
    return td.processResults(results)
}

// Use Redis Lua scripts for complex atomic operations
const securityAnalysisScript = `
    -- Complex security analysis in single Redis call
    local ip = ARGV[1]
    local user_agent = ARGV[2]
    -- ... security logic
    return risk_score
`

9. Summary and Review (5 minutes)

Key Takeaways

Rate limiting protects against abuse while maintaining good user experience
Multiple algorithms serve different needs - choose based on requirements
Security is layered - combine multiple detection methods
Redis enables real-time decisions with sub-millisecond performance
Monitoring and tuning are essential for effective security systems

Algorithm Selection Guide

Use Case	Best Algorithm	Characteristics
API Protection	Sliding Window	Smooth rate distribution
Burst Traffic	Token Bucket	Allows bursts, gradual refill
Simple Implementation	Fixed Window	Easy to understand and debug
High Precision	Sliding Window Counter	Most accurate, complex

Quick Review Questions

When would you choose token bucket over sliding window?
How do you prevent false positives in security systems?
What Redis data structures are best for rate limiting?
How do you handle distributed rate limiting across multiple servers?
What’s the difference between throttling and blocking?

For Instructors

Teaching Tips

Demonstrate actual attacks and how systems respond
Show rate limiting in action with load testing tools
Use real security event logs to illustrate patterns
Demonstrate the cost of security checks on performance
Show both over-aggressive and under-protective configurations

Common Student Questions

Q: “How do I handle false positives?” A: Implement reputation systems, grace periods, and manual override capabilities
Q: “What if Redis goes down?” A: Implement fallback mechanisms - fail open vs fail closed decision
Q: “How do I test rate limiting?” A: Use load testing tools like Apache Bench, wrk, or custom scripts

Assessment Ideas

Design complete API security architecture
Implement custom rate limiting algorithms
Build fraud detection system for e-commerce
Create DDoS protection for high-traffic application
Security audit of existing Redis-based systems

Adaptation Notes

Beginners: Focus on basic rate limiting concepts and simple implementations
Advanced: Cover sophisticated threat detection and distributed security
Enterprise: Emphasize compliance, monitoring, and incident response procedures