Advanced Topics and Best Practices
Learning Objectives: By the end of this lesson, you will be able to:
- Implement real-time communication with WebSockets and Server-Sent Events
- Build GraphQL APIs alongside REST endpoints
- Apply advanced architectural patterns (microservices, event sourcing, CQRS)
- Implement robust caching strategies and performance optimization
- Use advanced Go concurrency patterns for high-performance applications
- Apply security best practices and handle common vulnerabilities
- Design extensible and maintainable application architectures
- Implement observability patterns for complex distributed systems
1. Welcome and Introduction (5 minutes)
Icebreaker Activity
Have you ever wondered how applications like Slack handle thousands of simultaneous real-time messages, or how Netflix can serve millions of personalized recommendations instantly? Today, we’re diving into the advanced techniques and architectural patterns that separate good applications from great ones. We’ll explore the cutting-edge technologies and practices that power the world’s most successful web applications.
Lesson Overview
- Duration: 160 minutes
- Prerequisites: All previous lessons completed, solid understanding of Go concurrency, REST APIs, and distributed systems concepts
- Materials Needed: Go 1.25+ for Gin v1.12, Gin framework, Redis, WebSocket testing tools
2. Learning Objectives and Session Outline (5 minutes)
What We’ll Cover Today:
- Real-time Communication - WebSockets, SSE, and real-time architectures
- GraphQL Integration - Building flexible, efficient APIs
- Advanced Architecture - Microservices, event sourcing, and CQRS patterns
- Performance Optimization - Caching, concurrency, and resource management
- Security Excellence - Advanced security patterns and vulnerability mitigation
- System Design - Scalable, maintainable architecture patterns
- Observability - Advanced monitoring, tracing, and debugging techniques
Success Criteria:
By the end of this session, you should be able to architect and implement production-grade applications using advanced patterns and technologies.
3. Foundation Knowledge (15-20 minutes)
Core Concepts
Real-time Communication Patterns
Modern applications require various real-time communication methods:
- WebSockets: Full-duplex communication for interactive applications
- Server-Sent Events (SSE): Server-to-client streaming for live updates
- Long Polling: HTTP-based pseudo real-time communication
- Push Notifications: Out-of-band notifications to mobile/web clients
Real-World Analogy: Think of real-time communication like different types of conversations - WebSockets are like phone calls (two-way), SSE is like a radio broadcast (one-way), and push notifications are like text messages (asynchronous alerts).
API Design Evolution
- REST: Resource-oriented, standardized, cacheable
- GraphQL: Query-oriented, flexible, client-specified data fetching
- gRPC: High-performance, type-safe, binary protocol
- WebSocket APIs: Real-time, stateful, event-driven
Advanced Architectural Patterns
Microservices Architecture:
- Services are small, independent, and focused on specific business capabilities
- Communicate via well-defined APIs
- Can be developed, deployed, and scaled independently
Event Sourcing:
- Store events that represent state changes instead of current state
- Enables audit trails, replay capability, and temporal queries
- Complements CQRS (Command Query Responsibility Segregation)
CQRS (Command Query Responsibility Segregation):
- Separate models for reading and writing data
- Optimizes for different access patterns
- Enables independent scaling of read and write operations
Performance Optimization Strategies
- Caching Layers: In-memory, distributed, CDN-based
- Connection Pooling: Database, HTTP client, resource management
- Concurrency Patterns: Worker pools, pipelines, fan-out/fan-in
- Data Structure Optimization: Efficient algorithms and data structures
Key Terminology
- WebSocket: Protocol for full-duplex communication over TCP
- SSE (Server-Sent Events): HTML5 standard for server-to-client streaming
- GraphQL: Query language and runtime for APIs
- Event Sourcing: Architectural pattern storing sequence of events
- CQRS: Pattern separating read and write operations
- Circuit Breaker: Pattern to prevent cascading failures
- Bulkhead: Isolation pattern to prevent resource exhaustion
- Saga: Pattern for managing distributed transactions
4. Detailed Explanation and Examples (25-30 minutes)
Understanding Real-time Architecture
WebSocket vs REST Trade-offs
WebSockets Benefits:
- Real-time bidirectional communication
- Lower latency for frequent updates
- Stateful connections enable context preservation
- Efficient for high-frequency data exchange
WebSockets Challenges:
- Connection management complexity
- Scaling stateful connections
- Load balancing difficulties
- Debugging and testing complexity
GraphQL vs REST Comparison
| Aspect | REST | GraphQL |
|---|---|---|
| Data Fetching | Multiple endpoints, over/under-fetching | Single endpoint, precise data |
| Caching | HTTP caching, CDN-friendly | Complex caching, requires specialized tools |
| Learning Curve | Simple, well-understood | Steeper learning curve |
| Tooling | Mature ecosystem | Growing ecosystem |
| Flexibility | Fixed data structures | Dynamic queries |
Microservices Considerations
When to Use Microservices:
- Large, complex applications
- Multiple teams working independently
- Different scaling requirements per service
- Technology diversity needs
When to Avoid Microservices:
- Small applications or teams
- Simple, well-defined domains
- Strong consistency requirements
- Limited operational maturity
Advanced Security Considerations
Common Vulnerabilities in Web Applications
- Injection Attacks: SQL, NoSQL, OS command injection
- Authentication Bypass: Weak session management, JWT vulnerabilities
- Authorization Flaws: Broken access control, privilege escalation
- Data Exposure: Sensitive data in logs, error messages, responses
- Security Misconfiguration: Default configurations, unnecessary features
Security Patterns
- Defense in Depth: Multiple security layers
- Zero Trust: Never trust, always verify
- Principle of Least Privilege: Minimal necessary permissions
- Fail Secure: Secure defaults when systems fail
Common Misconceptions
-
Misconception: “WebSockets are always better than REST for real-time features” Reality: REST with polling or SSE might be simpler and more reliable for many use cases.
-
Misconception: “GraphQL replaces REST completely” Reality: Both have their place; GraphQL excels for complex, flexible data fetching while REST is simpler for CRUD operations.
-
Misconception: “Microservices automatically improve scalability” Reality: Microservices add complexity and can reduce performance if not properly implemented.
5. Live Coding Demonstration (20-25 minutes)
Example 1: WebSocket Chat Application with Room Management
// websocket/hub.go
package websocket
import (
"log"
"sync"
"time"
"github.com/gorilla/websocket"
)
// Hub maintains active clients and broadcasts messages
type Hub struct {
// Registered clients by room
rooms map[string]map[*Client]bool
// Inbound messages from clients
broadcast chan *Message
// Register requests from clients
register chan *ClientRegistration
// Unregister requests from clients
unregister chan *ClientRegistration
// Mutex for concurrent access
mutex sync.RWMutex
}
// ClientRegistration represents a client joining/leaving a room
type ClientRegistration struct {
Client *Client
Room string
}
// Message represents a chat message
type Message struct {
Type string `json:"type"`
Room string `json:"room"`
Username string `json:"username"`
Content string `json:"content"`
Data interface{} `json:"data,omitempty"`
}
// Client represents a WebSocket client
type Client struct {
// WebSocket connection
conn *websocket.Conn
// Buffered channel for outbound messages
send chan *Message
// User information
username string
userID uint
// Current room
room string
// Hub reference
hub *Hub
}
// NewHub creates a new WebSocket hub
func NewHub() *Hub {
return &Hub{
rooms: make(map[string]map[*Client]bool),
broadcast: make(chan *Message, 256),
register: make(chan *ClientRegistration, 256),
unregister: make(chan *ClientRegistration, 256),
}
}
// Run starts the hub's main loop
func (h *Hub) Run() {
for {
select {
case registration := <-h.register:
h.registerClient(registration)
case registration := <-h.unregister:
h.unregisterClient(registration)
case message := <-h.broadcast:
h.broadcastMessage(message)
}
}
}
// registerClient adds a client to a room
func (h *Hub) registerClient(registration *ClientRegistration) {
h.mutex.Lock()
client := registration.Client
room := registration.Room
// Initialize room if it doesn't exist
if h.rooms[room] == nil {
h.rooms[room] = make(map[*Client]bool)
}
// Add client to room
h.rooms[room][client] = true
users := h.getRoomUsers(room) // caller holds the lock
h.mutex.Unlock()
// Notify room about new user
userJoinMessage := &Message{
Type: "user_joined",
Room: room,
Username: client.username,
Content: client.username + " joined the room",
Data: map[string]interface{}{
"online_users": users,
},
}
h.broadcastToRoom(room, userJoinMessage)
log.Printf("Client %s joined room %s", client.username, room)
}
// unregisterClient removes a client from their room
func (h *Hub) unregisterClient(registration *ClientRegistration) {
h.mutex.Lock()
client := registration.Client
room := client.room
var users []string
removed := false
if room != "" && h.rooms[room] != nil {
if _, ok := h.rooms[room][client]; ok {
delete(h.rooms[room], client)
close(client.send)
removed = true
// Clean up empty rooms
if len(h.rooms[room]) == 0 {
delete(h.rooms, room)
} else {
users = h.getRoomUsers(room)
}
}
}
h.mutex.Unlock()
if removed {
h.broadcastToRoom(room, &Message{Type: "user_left", Room: room,
Username: client.username, Content: client.username + " left the room",
Data: map[string]interface{}{"online_users": users}})
}
log.Printf("Client %s left room %s", client.username, room)
}
// broadcastMessage sends a message to all clients in the message's room
func (h *Hub) broadcastMessage(message *Message) {
h.broadcastToRoom(message.Room, message)
}
// broadcastToRoom sends a message to all clients in a specific room
func (h *Hub) broadcastToRoom(room string, message *Message) {
h.mutex.RLock()
defer h.mutex.RUnlock() // map iteration and send must not race with close(client.send)
clients := h.rooms[room]
if clients == nil {
return
}
for client := range clients {
select {
case client.send <- message:
default:
// Client's send channel is full, remove the client
select {
case h.unregister <- &ClientRegistration{Client: client, Room: room}:
default: log.Print("unregister queue full")
}
}
}
}
// getRoomUsers returns a list of usernames in a room
func (h *Hub) getRoomUsers(room string) []string {
var users []string
if clients := h.rooms[room]; clients != nil {
for client := range clients {
users = append(users, client.username)
}
}
return users
}
// GetRoomStats returns statistics about a room
func (h *Hub) GetRoomStats(room string) map[string]interface{} {
h.mutex.RLock()
defer h.mutex.RUnlock()
clients := h.rooms[room]
if clients == nil {
return map[string]interface{}{
"online_users": 0,
"users": []string{},
}
}
return map[string]interface{}{
"online_users": len(clients),
"users": h.getRoomUsers(room),
}
}
// websocket/client.go
const (
// Time allowed to write a message to the peer
writeWait = 10 * time.Second
// Time allowed to read the next pong message from the peer
pongWait = 60 * time.Second
// Send pings to peer with this period (must be less than pongWait)
pingPeriod = (pongWait * 9) / 10
// Maximum message size allowed from peer
maxMessageSize = 512
)
// readPump pumps messages from the WebSocket connection to the hub
func (c *Client) readPump() {
defer func() {
c.hub.unregister <- &ClientRegistration{Client: c, Room: c.room}
c.conn.Close()
}()
c.conn.SetReadLimit(maxMessageSize)
c.conn.SetReadDeadline(time.Now().Add(pongWait))
c.conn.SetPongHandler(func(string) error {
c.conn.SetReadDeadline(time.Now().Add(pongWait))
return nil
})
for {
var message Message
err := c.conn.ReadJSON(&message)
if err != nil {
if websocket.IsUnexpectedCloseError(err, websocket.CloseGoingAway, websocket.CloseAbnormalClosure) {
log.Printf("WebSocket error: %v", err)
}
break
}
// Set message metadata
message.Username = c.username
message.Room = c.room
// Validate and process different message types
switch message.Type {
case "chat_message":
if message.Content != "" {
c.hub.broadcast <- &message
}
case "typing_start":
c.hub.broadcastToRoom(c.room, &Message{
Type: "typing_start",
Room: c.room,
Username: c.username,
})
case "typing_stop":
c.hub.broadcastToRoom(c.room, &Message{
Type: "typing_stop",
Room: c.room,
Username: c.username,
})
default:
log.Printf("Unknown message type: %s", message.Type)
}
}
}
// writePump pumps messages from the hub to the WebSocket connection
func (c *Client) writePump() {
ticker := time.NewTicker(pingPeriod)
defer func() {
ticker.Stop()
c.conn.Close()
}()
for {
select {
case message, ok := <-c.send:
c.conn.SetWriteDeadline(time.Now().Add(writeWait))
if !ok {
c.conn.WriteMessage(websocket.CloseMessage, []byte{})
return
}
if err := c.conn.WriteJSON(message); err != nil {
log.Printf("WebSocket write error: %v", err)
return
}
case <-ticker.C:
c.conn.SetWriteDeadline(time.Now().Add(writeWait))
if err := c.conn.WriteMessage(websocket.PingMessage, nil); err != nil {
return
}
}
}
}
// websocket/handler.go (same package as Hub and Client)
package websocket
import (
"log"
"net/http"
"os"
"github.com/gin-gonic/gin"
gorillaWS "github.com/gorilla/websocket"
"github.com/your-app/auth"
)
var upgrader = gorillaWS.Upgrader{
ReadBufferSize: 1024,
WriteBufferSize: 1024,
CheckOrigin: func(r *http.Request) bool {
// Configure the public browser origin (e.g. https://app.example.com).
allowed := os.Getenv("WS_ALLOWED_ORIGIN")
return allowed != "" && r.Header.Get("Origin") == allowed
},
}
// WebSocketHandler handles WebSocket connections
type WebSocketHandler struct {
hub *Hub
}
// NewWebSocketHandler creates a new WebSocket handler
func NewWebSocketHandler(hub *Hub) *WebSocketHandler {
return &WebSocketHandler{hub: hub}
}
// HandleWebSocket upgrades HTTP connection to WebSocket
func (h *WebSocketHandler) HandleWebSocket(c *gin.Context) {
// Get user from JWT token
userInterface, exists := c.Get("user")
if !exists {
c.JSON(http.StatusUnauthorized, gin.H{"error": "User not authenticated"})
return
}
claims, ok := userInterface.(*auth.Claims)
if !ok {
c.JSON(http.StatusInternalServerError, gin.H{"error": "Invalid user data"})
return
}
// Get room from query parameter
room := c.Query("room")
if room == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "Room parameter required"})
return
}
// Before upgrading, authorize claims.UserID for this room in your room store.
// Authentication alone does not grant access to every room.
// Upgrade HTTP connection to WebSocket
conn, err := upgrader.Upgrade(c.Writer, c.Request, nil)
if err != nil {
log.Printf("WebSocket upgrade error: %v", err)
return
}
// Create client
client := &Client{
conn: conn,
send: make(chan *Message, 256),
username: claims.Username,
userID: claims.UserID,
room: room,
hub: h.hub,
}
// Register client
h.hub.register <- &ClientRegistration{
Client: client,
Room: room,
}
// Start client goroutines
go client.writePump()
go client.readPump()
}
// GetRoomStats returns statistics about a room
func (h *WebSocketHandler) GetRoomStats(c *gin.Context) {
room := c.Param("room")
if room == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "Room parameter required"})
return
}
stats := h.hub.GetRoomStats(room)
c.JSON(http.StatusOK, stats)
}
// main.go - WebSocket integration
func main() {
// ... existing setup ...
// Create WebSocket hub
wsHub := websocket.NewHub()
go wsHub.Run()
// Setup WebSocket handler
wsHandler := websocket.NewWebSocketHandler(wsHub)
// WebSocket routes: this bearer-header middleware works for clients that
// can send Authorization on the handshake. Browser WebSocket cannot set
// arbitrary headers: use same-origin authenticated cookies plus origin/
// CSRF checks, or a short-lived one-use ticket, for browser clients.
router.GET("/ws", middleware.Auth(authService), wsHandler.HandleWebSocket)
router.GET("/api/v1/rooms/:room/stats", wsHandler.GetRoomStats)
// ... rest of main function ...
}
Example 2: GraphQL API Integration
// graphql/schema.go
package graphql
import (
"context"
"fmt"
"strconv"
"github.com/graphql-go/graphql"
"github.com/your-app/models"
"github.com/your-app/services"
)
// GraphQLHandler handles GraphQL requests
type GraphQLHandler struct {
userService services.UserService
postService services.PostService
schema graphql.Schema
}
// NewGraphQLHandler creates a new GraphQL handler
func NewGraphQLHandler(userService services.UserService, postService services.PostService) *GraphQLHandler {
handler := &GraphQLHandler{
userService: userService,
postService: postService,
}
schema, err := handler.createSchema()
if err != nil {
panic(fmt.Sprintf("Failed to create GraphQL schema: %v", err))
}
handler.schema = schema
return handler
}
// createSchema defines the GraphQL schema
func (h *GraphQLHandler) createSchema() (graphql.Schema, error) {
// Define User type
userType := graphql.NewObject(graphql.ObjectConfig{
Name: "User",
Description: "A user in the system",
Fields: graphql.Fields{
"id": &graphql.Field{
Type: graphql.NewNonNull(graphql.ID),
Resolve: func(p graphql.ResolveParams) (interface{}, error) {
if user, ok := p.Source.(*models.User); ok {
return user.ID, nil
}
return nil, nil
},
},
"username": &graphql.Field{
Type: graphql.NewNonNull(graphql.String),
Resolve: func(p graphql.ResolveParams) (interface{}, error) {
if user, ok := p.Source.(*models.User); ok {
return user.Username, nil
}
return nil, nil
},
},
"email": &graphql.Field{
Type: graphql.String,
Resolve: func(p graphql.ResolveParams) (interface{}, error) {
if user, ok := p.Source.(*models.User); ok {
return user.Email, nil
}
return nil, nil
},
},
"fullName": &graphql.Field{
Type: graphql.String,
Resolve: func(p graphql.ResolveParams) (interface{}, error) {
if user, ok := p.Source.(*models.User); ok {
return user.FullName, nil
}
return nil, nil
},
},
"posts": &graphql.Field{
Type: graphql.NewList(postType),
Resolve: func(p graphql.ResolveParams) (interface{}, error) {
if user, ok := p.Source.(*models.User); ok {
return h.postService.GetPostsByUserID(user.ID)
}
return nil, nil
},
},
},
})
// Define Post type
postType := graphql.NewObject(graphql.ObjectConfig{
Name: "Post",
Description: "A blog post",
Fields: graphql.Fields{
"id": &graphql.Field{
Type: graphql.NewNonNull(graphql.ID),
Resolve: func(p graphql.ResolveParams) (interface{}, error) {
if post, ok := p.Source.(*models.Post); ok {
return post.ID, nil
}
return nil, nil
},
},
"title": &graphql.Field{
Type: graphql.NewNonNull(graphql.String),
Resolve: func(p graphql.ResolveParams) (interface{}, error) {
if post, ok := p.Source.(*models.Post); ok {
return post.Title, nil
}
return nil, nil
},
},
"content": &graphql.Field{
Type: graphql.String,
Resolve: func(p graphql.ResolveParams) (interface{}, error) {
if post, ok := p.Source.(*models.Post); ok {
return post.Content, nil
}
return nil, nil
},
},
"author": &graphql.Field{
Type: userType,
Resolve: func(p graphql.ResolveParams) (interface{}, error) {
if post, ok := p.Source.(*models.Post); ok {
return h.userService.GetUserByID(post.UserID)
}
return nil, nil
},
},
"createdAt": &graphql.Field{
Type: graphql.DateTime,
Resolve: func(p graphql.ResolveParams) (interface{}, error) {
if post, ok := p.Source.(*models.Post); ok {
return post.CreatedAt, nil
}
return nil, nil
},
},
},
})
// Define root query
rootQuery := graphql.NewObject(graphql.ObjectConfig{
Name: "Query",
Fields: graphql.Fields{
"user": &graphql.Field{
Type: userType,
Description: "Get a user by ID",
Args: graphql.FieldConfigArgument{
"id": &graphql.ArgumentConfig{
Type: graphql.NewNonNull(graphql.ID),
Description: "User ID",
},
},
Resolve: func(p graphql.ResolveParams) (interface{}, error) {
idStr, ok := p.Args["id"].(string)
if !ok {
return nil, fmt.Errorf("invalid user ID")
}
id, err := strconv.ParseUint(idStr, 10, 32)
if err != nil {
return nil, fmt.Errorf("invalid user ID format")
}
return h.userService.GetUserByID(uint(id))
},
},
"users": &graphql.Field{
Type: graphql.NewList(userType),
Description: "Get all users",
Args: graphql.FieldConfigArgument{
"limit": &graphql.ArgumentConfig{
Type: graphql.Int,
DefaultValue: 10,
Description: "Number of users to return",
},
"offset": &graphql.ArgumentConfig{
Type: graphql.Int,
DefaultValue: 0,
Description: "Number of users to skip",
},
},
Resolve: func(p graphql.ResolveParams) (interface{}, error) {
limit := p.Args["limit"].(int)
offset := p.Args["offset"].(int)
return h.userService.GetUsers(limit, offset)
},
},
"post": &graphql.Field{
Type: postType,
Description: "Get a post by ID",
Args: graphql.FieldConfigArgument{
"id": &graphql.ArgumentConfig{
Type: graphql.NewNonNull(graphql.ID),
Description: "Post ID",
},
},
Resolve: func(p graphql.ResolveParams) (interface{}, error) {
idStr, ok := p.Args["id"].(string)
if !ok {
return nil, fmt.Errorf("invalid post ID")
}
id, err := strconv.ParseUint(idStr, 10, 32)
if err != nil {
return nil, fmt.Errorf("invalid post ID format")
}
return h.postService.GetPostByID(uint(id))
},
},
"posts": &graphql.Field{
Type: graphql.NewList(postType),
Description: "Get posts with filtering",
Args: graphql.FieldConfigArgument{
"authorId": &graphql.ArgumentConfig{
Type: graphql.ID,
Description: "Filter by author ID",
},
"limit": &graphql.ArgumentConfig{
Type: graphql.Int,
DefaultValue: 10,
Description: "Number of posts to return",
},
"offset": &graphql.ArgumentConfig{
Type: graphql.Int,
DefaultValue: 0,
Description: "Number of posts to skip",
},
},
Resolve: func(p graphql.ResolveParams) (interface{}, error) {
limit := p.Args["limit"].(int)
offset := p.Args["offset"].(int)
if authorIdStr, ok := p.Args["authorId"].(string); ok {
authorId, err := strconv.ParseUint(authorIdStr, 10, 32)
if err != nil {
return nil, fmt.Errorf("invalid author ID format")
}
return h.postService.GetPostsByUserID(uint(authorId))
}
return h.postService.GetPosts(limit, offset)
},
},
},
})
// Define root mutation
rootMutation := graphql.NewObject(graphql.ObjectConfig{
Name: "Mutation",
Fields: graphql.Fields{
"createPost": &graphql.Field{
Type: postType,
Description: "Create a new post",
Args: graphql.FieldConfigArgument{
"title": &graphql.ArgumentConfig{
Type: graphql.NewNonNull(graphql.String),
Description: "Post title",
},
"content": &graphql.ArgumentConfig{
Type: graphql.NewNonNull(graphql.String),
Description: "Post content",
},
},
Resolve: func(p graphql.ResolveParams) (interface{}, error) {
// Get user from context (set by authentication middleware)
userID, ok := p.Context.Value("userID").(uint)
if !ok {
return nil, fmt.Errorf("authentication required")
}
title := p.Args["title"].(string)
content := p.Args["content"].(string)
return h.postService.CreatePost(userID, title, content)
},
},
"updatePost": &graphql.Field{
Type: postType,
Description: "Update an existing post",
Args: graphql.FieldConfigArgument{
"id": &graphql.ArgumentConfig{
Type: graphql.NewNonNull(graphql.ID),
Description: "Post ID",
},
"title": &graphql.ArgumentConfig{
Type: graphql.String,
Description: "Post title",
},
"content": &graphql.ArgumentConfig{
Type: graphql.String,
Description: "Post content",
},
},
Resolve: func(p graphql.ResolveParams) (interface{}, error) {
userID, ok := p.Context.Value("userID").(uint)
if !ok {
return nil, fmt.Errorf("authentication required")
}
idStr := p.Args["id"].(string)
id, err := strconv.ParseUint(idStr, 10, 32)
if err != nil {
return nil, fmt.Errorf("invalid post ID format")
}
updates := make(map[string]interface{})
if title, ok := p.Args["title"]; ok {
updates["title"] = title
}
if content, ok := p.Args["content"]; ok {
updates["content"] = content
}
return h.postService.UpdatePost(uint(id), userID, updates)
},
},
},
})
// Create schema
return graphql.NewSchema(graphql.SchemaConfig{
Query: rootQuery,
Mutation: rootMutation,
})
}
// Execute executes a GraphQL query
func (h *GraphQLHandler) Execute(query string, variables map[string]interface{}, ctx context.Context) *graphql.Result {
result := graphql.Do(graphql.Params{
Schema: h.schema,
RequestString: query,
VariableValues: variables,
Context: ctx,
})
return result
}
// handlers/graphql_handler.go
package handlers
import (
"context"
"encoding/json"
"net/http"
"github.com/gin-gonic/gin"
"github.com/your-app/auth"
"github.com/your-app/graphql"
)
// GraphQLHTTPHandler handles HTTP GraphQL requests
type GraphQLHTTPHandler struct {
graphqlHandler *graphql.GraphQLHandler
}
// NewGraphQLHTTPHandler creates a new GraphQL HTTP handler
func NewGraphQLHTTPHandler(graphqlHandler *graphql.GraphQLHandler) *GraphQLHTTPHandler {
return &GraphQLHTTPHandler{
graphqlHandler: graphqlHandler,
}
}
// GraphQLRequest represents a GraphQL HTTP request
type GraphQLRequest struct {
Query string `json:"query"`
Variables map[string]interface{} `json:"variables"`
}
// HandleGraphQL processes GraphQL HTTP requests
func (h *GraphQLHTTPHandler) HandleGraphQL(c *gin.Context) {
var req GraphQLRequest
if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{
"errors": []gin.H{
{"message": "Invalid request format"},
},
})
return
}
// Create context with user information if authenticated
ctx := context.Background()
if userInterface, exists := c.Get("user"); exists {
if claims, ok := userInterface.(*auth.Claims); ok {
ctx = context.WithValue(ctx, "userID", claims.UserID)
ctx = context.WithValue(ctx, "username", claims.Username)
}
}
// Execute GraphQL query
result := h.graphqlHandler.Execute(req.Query, req.Variables, ctx)
// Return result
c.JSON(http.StatusOK, result)
}
// HandleGraphQLPlayground serves GraphQL Playground
func (h *GraphQLHTTPHandler) HandleGraphQLPlayground(c *gin.Context) {
playgroundHTML := `
<!DOCTYPE html>
<html>
<head>
<title>GraphQL Playground</title>
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/graphql-playground-react/build/static/css/index.css" />
<link rel="shortcut icon" href="https://cdn.jsdelivr.net/npm/graphql-playground-react/build/favicon.png" />
<script src="https://cdn.jsdelivr.net/npm/graphql-playground-react/build/static/js/middleware.js"></script>
</head>
<body>
<div id="root">
<style>
body { background-color: rgb(23, 42, 58); font-family: Open Sans, sans-serif; height: 90vh; }
#root { height: 100%; width: 100%; display: flex; align-items: center; justify-content: center; }
.loading { font-size: 32px; font-weight: 200; color: rgba(255, 255, 255, .6); margin-left: 20px; }
</style>
<div class="loading"> Loading... </div>
</div>
<script>
window.addEventListener('load', function (event) {
GraphQLPlayground.init(document.getElementById('root'), {
endpoint: '/graphql'
})
})
</script>
</body>
</html>`
c.Data(http.StatusOK, "text/html; charset=utf-8", []byte(playgroundHTML))
}
Example 3: Advanced Caching Strategy
// cache/manager.go
package cache
import (
"context"
"encoding/json"
"fmt"
"sync"
"time"
"github.com/redis/go-redis/v9"
)
// CacheManager handles multiple cache layers
type CacheManager struct {
// Local in-memory cache
localCache sync.Map
localTTL time.Duration
// Redis distributed cache
redisClient *redis.Client
// Cache statistics
stats CacheStats
mutex sync.RWMutex
}
// CacheStats tracks cache performance
type CacheStats struct {
LocalHits int64 `json:"local_hits"`
LocalMisses int64 `json:"local_misses"`
RedisHits int64 `json:"redis_hits"`
RedisMisses int64 `json:"redis_misses"`
TotalRequests int64 `json:"total_requests"`
}
// CacheItem represents a cached item with expiration
type CacheItem struct {
Data interface{} `json:"data"`
ExpiresAt time.Time `json:"expires_at"`
}
// CacheConfig holds cache configuration
type CacheConfig struct {
RedisURL string
LocalTTL time.Duration
RedisDBTTL time.Duration
MaxLocalItems int
}
// NewCacheManager creates a new cache manager
func NewCacheManager(config CacheConfig) *CacheManager {
// Setup Redis client
opt, err := redis.ParseURL(config.RedisURL)
if err != nil {
panic(fmt.Sprintf("Invalid Redis URL: %v", err))
}
redisClient := redis.NewClient(opt)
// Test Redis connection
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := redisClient.Ping(ctx).Err(); err != nil {
panic(fmt.Sprintf("Failed to connect to Redis: %v", err))
}
manager := &CacheManager{
localTTL: config.LocalTTL,
redisClient: redisClient,
}
// Start cleanup goroutine for local cache
go manager.cleanupExpiredItems()
return manager
}
// Get retrieves a value from cache (local first, then Redis)
func (c *CacheManager) Get(ctx context.Context, key string, dest interface{}) error {
c.incrementTotalRequests()
// Try local cache first
if item, found := c.localCache.Load(key); found {
cacheItem := item.(*CacheItem)
if time.Now().Before(cacheItem.ExpiresAt) {
c.incrementLocalHits()
return c.deserialize(cacheItem.Data, dest)
}
// Item expired, remove it
c.localCache.Delete(key)
}
c.incrementLocalMisses()
// Try Redis cache
val, err := c.redisClient.Get(ctx, key).Result()
if err == redis.Nil {
c.incrementRedisMisses()
return ErrCacheMiss
} else if err != nil {
return fmt.Errorf("redis get error: %w", err)
}
c.incrementRedisHits()
// Deserialize from Redis
if err := json.Unmarshal([]byte(val), dest); err != nil {
return fmt.Errorf("failed to unmarshal cached data: %w", err)
}
// Store in local cache for faster future access
c.setLocal(key, dest, c.localTTL)
return nil
}
// Set stores a value in both local and Redis cache
func (c *CacheManager) Set(ctx context.Context, key string, value interface{}, ttl time.Duration) error {
// Store in local cache
c.setLocal(key, value, ttl)
// Store in Redis
data, err := json.Marshal(value)
if err != nil {
return fmt.Errorf("failed to marshal data: %w", err)
}
if err := c.redisClient.Set(ctx, key, data, ttl).Err(); err != nil {
return fmt.Errorf("redis set error: %w", err)
}
return nil
}
// Delete removes a value from both caches
func (c *CacheManager) Delete(ctx context.Context, key string) error {
// Remove from local cache
c.localCache.Delete(key)
// Remove from Redis
if err := c.redisClient.Del(ctx, key).Err(); err != nil {
return fmt.Errorf("redis delete error: %w", err)
}
return nil
}
// InvalidatePattern removes all keys matching a pattern
func (c *CacheManager) InvalidatePattern(ctx context.Context, pattern string) error {
// Get all matching keys from Redis
keys, err := c.redisClient.Keys(ctx, pattern).Result()
if err != nil {
return fmt.Errorf("redis keys error: %w", err)
}
if len(keys) == 0 {
return nil
}
// Delete from Redis
if err := c.redisClient.Del(ctx, keys...).Err(); err != nil {
return fmt.Errorf("redis delete error: %w", err)
}
// Delete from local cache (check each key)
for _, key := range keys {
c.localCache.Delete(key)
}
return nil
}
// setLocal stores a value in local cache with TTL
func (c *CacheManager) setLocal(key string, value interface{}, ttl time.Duration) {
item := &CacheItem{
Data: value,
ExpiresAt: time.Now().Add(ttl),
}
c.localCache.Store(key, item)
}
// cleanupExpiredItems periodically removes expired items from local cache
func (c *CacheManager) cleanupExpiredItems() {
ticker := time.NewTicker(5 * time.Minute)
defer ticker.Stop()
for range ticker.C {
now := time.Now()
var expiredKeys []interface{}
c.localCache.Range(func(key, value interface{}) bool {
item := value.(*CacheItem)
if now.After(item.ExpiresAt) {
expiredKeys = append(expiredKeys, key)
}
return true
})
for _, key := range expiredKeys {
c.localCache.Delete(key)
}
}
}
// GetStats returns cache statistics
func (c *CacheManager) GetStats() CacheStats {
c.mutex.RLock()
defer c.mutex.RUnlock()
return c.stats
}
// Helper methods for statistics
func (c *CacheManager) incrementLocalHits() {
c.mutex.Lock()
c.stats.LocalHits++
c.mutex.Unlock()
}
func (c *CacheManager) incrementLocalMisses() {
c.mutex.Lock()
c.stats.LocalMisses++
c.mutex.Unlock()
}
func (c *CacheManager) incrementRedisHits() {
c.mutex.Lock()
c.stats.RedisHits++
c.mutex.Unlock()
}
func (c *CacheManager) incrementRedisMisses() {
c.mutex.Lock()
c.stats.RedisMisses++
c.mutex.Unlock()
}
func (c *CacheManager) incrementTotalRequests() {
c.mutex.Lock()
c.stats.TotalRequests++
c.mutex.Unlock()
}
// deserialize converts cached data to destination type
func (c *CacheManager) deserialize(data interface{}, dest interface{}) error {
// Handle different data types appropriately
switch v := data.(type) {
case string:
return json.Unmarshal([]byte(v), dest)
default:
// Direct assignment for simple types
jsonData, err := json.Marshal(data)
if err != nil {
return err
}
return json.Unmarshal(jsonData, dest)
}
}
var ErrCacheMiss = fmt.Errorf("cache miss")
// middleware/cache_middleware.go
package middleware
import (
"context"
"crypto/md5"
"fmt"
"net/http"
"strconv"
"strings"
"time"
"github.com/gin-gonic/gin"
"github.com/your-app/cache"
)
// CacheMiddleware provides HTTP response caching
func CacheMiddleware(cacheManager *cache.CacheManager, ttl time.Duration) gin.HandlerFunc {
return func(c *gin.Context) {
// Skip caching for non-GET requests
if c.Request.Method != "GET" {
c.Next()
return
}
// Skip caching if user is authenticated (for personalized content)
if _, exists := c.Get("user"); exists {
c.Next()
return
}
// Generate cache key
cacheKey := generateCacheKey(c.Request.URL.Path, c.Request.URL.RawQuery)
// Try to get cached response
var cachedResponse CachedResponse
err := cacheManager.Get(context.Background(), cacheKey, &cachedResponse)
if err == nil {
// Return cached response
for key, value := range cachedResponse.Headers {
c.Header(key, value)
}
c.Data(cachedResponse.StatusCode, cachedResponse.ContentType, cachedResponse.Body)
return
}
// Create response writer wrapper to capture response
writer := &responseWriter{
ResponseWriter: c.Writer,
body: make([]byte, 0),
statusCode: http.StatusOK,
}
c.Writer = writer
// Process request
c.Next()
// Cache successful responses
if writer.statusCode >= 200 && writer.statusCode < 300 {
cachedResponse := CachedResponse{
StatusCode: writer.statusCode,
ContentType: c.GetHeader("Content-Type"),
Headers: make(map[string]string),
Body: writer.body,
}
// Copy important headers
for _, header := range []string{"Content-Type", "Cache-Control", "ETag"} {
if value := c.GetHeader(header); value != "" {
cachedResponse.Headers[header] = value
}
}
// Store in cache
cacheManager.Set(context.Background(), cacheKey, cachedResponse, ttl)
}
}
}
// CachedResponse represents a cached HTTP response
type CachedResponse struct {
StatusCode int `json:"status_code"`
ContentType string `json:"content_type"`
Headers map[string]string `json:"headers"`
Body []byte `json:"body"`
}
// responseWriter captures HTTP response for caching
type responseWriter struct {
gin.ResponseWriter
body []byte
statusCode int
}
func (w *responseWriter) Write(data []byte) (int, error) {
w.body = append(w.body, data...)
return w.ResponseWriter.Write(data)
}
func (w *responseWriter) WriteHeader(statusCode int) {
w.statusCode = statusCode
w.ResponseWriter.WriteHeader(statusCode)
}
// generateCacheKey creates a unique cache key for the request
func generateCacheKey(path, query string) string {
key := path
if query != "" {
key += "?" + query
}
// Create MD5 hash for long keys
if len(key) > 200 {
hash := md5.Sum([]byte(key))
return fmt.Sprintf("http_cache:%x", hash)
}
return "http_cache:" + strings.ReplaceAll(key, "/", "_")
}
6. Hands-On Practice (30-40 minutes)
Exercise 1: Event Sourcing Implementation (15 minutes)
Objective: Implement an event sourcing system for user account changes
Instructions:
- Design event types for user registration, profile updates, and account deactivation
- Implement event store using database or Redis streams
- Create event handlers for updating read models
- Build an event replay system for rebuilding state
- Add snapshot functionality for performance optimization
Starter Code:
// events/types.go
type Event struct {
ID string `json:"id"`
AggregateID string `json:"aggregate_id"`
Type string `json:"type"`
Data []byte `json:"data"`
Version int `json:"version"`
Timestamp time.Time `json:"timestamp"`
}
// TODO: Implement event store interface
type EventStore interface {
Save(ctx context.Context, events []Event) error
GetEvents(ctx context.Context, aggregateID string, fromVersion int) ([]Event, error)
GetAllEvents(ctx context.Context, eventTypes []string) ([]Event, error)
}
// TODO: Implement specific event types
type UserRegisteredEvent struct {
UserID string `json:"user_id"`
Username string `json:"username"`
Email string `json:"email"`
}
Solution:
// events/store.go
package events
import (
"context"
"encoding/json"
"fmt"
"time"
"github.com/google/uuid"
"gorm.io/gorm"
)
// PostgreSQLEventStore implements EventStore using PostgreSQL
type PostgreSQLEventStore struct {
db *gorm.DB
}
// EventRecord represents the database model for events
type EventRecord struct {
ID string `gorm:"primaryKey;type:uuid"`
AggregateID string `gorm:"index;not null"`
Type string `gorm:"not null"`
Data []byte `gorm:"type:jsonb"`
Version int `gorm:"not null"`
Timestamp time.Time `gorm:"not null"`
}
func NewPostgreSQLEventStore(db *gorm.DB) *PostgreSQLEventStore {
return &PostgreSQLEventStore{db: db}
}
func (s *PostgreSQLEventStore) Save(ctx context.Context, events []Event) error {
var records []EventRecord
for _, event := range events {
if event.ID == "" {
event.ID = uuid.New().String()
}
if event.Timestamp.IsZero() {
event.Timestamp = time.Now()
}
records = append(records, EventRecord{
ID: event.ID,
AggregateID: event.AggregateID,
Type: event.Type,
Data: event.Data,
Version: event.Version,
Timestamp: event.Timestamp,
})
}
return s.db.WithContext(ctx).Create(&records).Error
}
func (s *PostgreSQLEventStore) GetEvents(ctx context.Context, aggregateID string, fromVersion int) ([]Event, error) {
var records []EventRecord
err := s.db.WithContext(ctx).
Where("aggregate_id = ? AND version >= ?", aggregateID, fromVersion).
Order("version ASC").
Find(&records).Error
if err != nil {
return nil, err
}
events := make([]Event, len(records))
for i, record := range records {
events[i] = Event{
ID: record.ID,
AggregateID: record.AggregateID,
Type: record.Type,
Data: record.Data,
Version: record.Version,
Timestamp: record.Timestamp,
}
}
return events, nil
}
// Event types
type UserRegisteredEvent struct {
UserID string `json:"user_id"`
Username string `json:"username"`
Email string `json:"email"`
FullName string `json:"full_name"`
}
type UserProfileUpdatedEvent struct {
UserID string `json:"user_id"`
Changes map[string]interface{} `json:"changes"`
}
type UserDeactivatedEvent struct {
UserID string `json:"user_id"`
Reason string `json:"reason"`
}
// Event handlers
type UserEventHandler struct {
userRepo repositories.UserRepository
}
func NewUserEventHandler(userRepo repositories.UserRepository) *UserEventHandler {
return &UserEventHandler{userRepo: userRepo}
}
func (h *UserEventHandler) Handle(ctx context.Context, event Event) error {
switch event.Type {
case "user_registered":
return h.handleUserRegistered(ctx, event)
case "user_profile_updated":
return h.handleUserProfileUpdated(ctx, event)
case "user_deactivated":
return h.handleUserDeactivated(ctx, event)
default:
return fmt.Errorf("unknown event type: %s", event.Type)
}
}
func (h *UserEventHandler) handleUserRegistered(ctx context.Context, event Event) error {
var eventData UserRegisteredEvent
if err := json.Unmarshal(event.Data, &eventData); err != nil {
return err
}
user := &models.User{
ID: parseUserID(eventData.UserID),
Username: eventData.Username,
Email: eventData.Email,
FullName: eventData.FullName,
Active: true,
}
return h.userRepo.Create(user)
}
// Event sourcing aggregate
type UserAggregate struct {
ID string
Version int
Username string
Email string
FullName string
Active bool
events []Event
}
func (u *UserAggregate) RegisterUser(username, email, fullName string) error {
if u.ID != "" {
return fmt.Errorf("user already exists")
}
eventData := UserRegisteredEvent{
UserID: uuid.New().String(),
Username: username,
Email: email,
FullName: fullName,
}
data, err := json.Marshal(eventData)
if err != nil {
return err
}
event := Event{
AggregateID: eventData.UserID,
Type: "user_registered",
Data: data,
Version: u.Version + 1,
}
u.apply(event)
u.events = append(u.events, event)
return nil
}
func (u *UserAggregate) apply(event Event) {
switch event.Type {
case "user_registered":
var eventData UserRegisteredEvent
json.Unmarshal(event.Data, &eventData)
u.ID = eventData.UserID
u.Username = eventData.Username
u.Email = eventData.Email
u.FullName = eventData.FullName
u.Active = true
case "user_profile_updated":
var eventData UserProfileUpdatedEvent
json.Unmarshal(event.Data, &eventData)
// Apply changes
for key, value := range eventData.Changes {
switch key {
case "full_name":
u.FullName = value.(string)
case "email":
u.Email = value.(string)
}
}
case "user_deactivated":
u.Active = false
}
u.Version = event.Version
}
func (u *UserAggregate) GetUncommittedEvents() []Event {
return u.events
}
func (u *UserAggregate) ClearEvents() {
u.events = nil
}
Exercise 2: Circuit Breaker Pattern (15 minutes)
Objective: Implement a circuit breaker for external service calls
Requirements:
- Monitor failure rates and response times
- Implement open, closed, and half-open states
- Add exponential backoff for recovery attempts
- Provide fallback mechanisms for when circuit is open
- Include metrics and monitoring
Hints:
- Use goroutines for state management and monitoring
- Implement timeout handling for external calls
- Add configuration for failure thresholds and recovery times
Exercise 3: Distributed Tracing Implementation (10 minutes)
Objective: Add comprehensive distributed tracing to your application
Challenge: Implement:
- Request tracing across all services and databases
- Custom span attributes for business context
- Error tracking and performance monitoring
- Correlation of logs with traces
- Integration with Jaeger or Zipkin
Extension Ideas:
- Add custom metrics based on trace data
- Implement trace sampling for high-volume applications
- Create trace-based alerts for performance issues
- Build custom dashboards for business metrics
7. Common Pitfalls and Troubleshooting (10 minutes)
Frequent Mistakes
-
Problem: WebSocket connections consuming too much memory Cause: Not properly managing connection lifecycle or storing too much per-connection state Solution: Implement connection pooling, proper cleanup, and minimize per-connection storage
-
Problem: GraphQL queries causing N+1 database problems Cause: Resolvers making individual database queries for each item in a list Solution: Use DataLoader pattern or batch queries in resolvers
-
Problem: Event sourcing causing performance issues Cause: Replaying too many events to rebuild aggregate state Solution: Implement snapshots and optimize event storage/retrieval
Debugging Tips
- Use proper structured logging with correlation IDs across all components
- Implement comprehensive monitoring for all external dependencies
- Add circuit breakers and timeouts for all network calls
- Use distributed tracing to track requests across service boundaries
- Monitor resource usage (memory, CPU, connections) continuously
8. Best Practices and Professional Tips (5 minutes)
Industry Standards
- Follow domain-driven design principles for complex business logic
- Implement proper error handling and recovery strategies
- Use feature flags for gradual rollouts and A/B testing
- Apply security best practices at every layer
- Design for observability from the beginning
Next Steps in Your Learning Journey
- Study distributed systems patterns and consensus algorithms
- Learn about event streaming platforms (Apache Kafka, AWS Kinesis)
- Explore advanced security topics (zero-trust architecture, supply chain security)
- Study system design for large-scale applications
- Practice building resilient, fault-tolerant systems
9. Summary and Review (5 minutes)
Key Takeaways
- Real-time features require careful architecture planning and resource management
- GraphQL provides flexibility but requires thoughtful design to avoid performance issues
- Advanced patterns like event sourcing and CQRS solve complex problems but add complexity
- Performance optimization requires understanding of bottlenecks and proper measurement
Quick Review Questions
- When should you choose WebSockets over Server-Sent Events?
- What are the trade-offs between GraphQL and REST APIs?
- How does event sourcing help with audit trails and temporal queries?
- What patterns help build resilient distributed systems?
For Instructors
Teaching Tips
- Use real-world examples from companies like Netflix, Uber, or Amazon
- Show performance comparisons between different architectural approaches
- Demonstrate how to debug complex distributed systems issues
- Emphasize the importance of incremental adoption of advanced patterns
- Connect technical patterns to business value and user experience
Assessment Ideas
- Build a real-time collaborative application (like Google Docs)
- Design and implement a microservices architecture for a specific domain
- Create a high-performance API with comprehensive caching strategies
- Implement event sourcing for a complex business workflow
- Build a resilient system with circuit breakers and fallback mechanisms
Adaptation Notes
- For beginners: Focus on one advanced topic per session to avoid overwhelm
- For advanced students: Introduce cutting-edge patterns and emerging technologies
- Online delivery: Provide sandbox environments with all necessary services pre-configured
- Time management: Each major topic (WebSockets, GraphQL, Event Sourcing) can be extended to full sessions