Shader Optimization

calendar_today June 6, 2025 schedule 13 min read

Learning Objectives: By the end of this lesson, you will be able to:

Identify and eliminate performance bottlenecks in 2D shaders

Balance visual quality with computational efficiency

Use profiling tools to debug and optimize shader performance

Implement platform-specific optimizations for mobile and web deployment

1. Welcome and Introduction (5 minutes)

Icebreaker Activity

Have you ever played a mobile game that looked amazing in screenshots but ran like molasses on your device? Or perhaps you’ve created a beautiful shader effect that works perfectly on your desktop but brings a smartphone to its knees? Performance optimization is the difference between a shader that impresses and one that actually ships. Today we’ll learn how to make your shaders not just beautiful, but blazingly fast across all platforms!

Lesson Overview

Duration: 90 minutes
Prerequisites: Solid understanding of shader syntax, texture techniques, and lighting systems
Materials Needed: Shader development environment with profiling capabilities, various target devices for testing

2. Learning Objectives and Session Outline (5 minutes)

What We’ll Cover Today:

Performance Analysis Fundamentals - Understanding GPU architecture and bottlenecks
Computational Optimization - Reducing ALU operations and mathematical complexity
Memory Optimization - Efficient texture usage and bandwidth management
Platform-Specific Strategies - Mobile, web, and desktop optimization techniques
Profiling and Debugging - Tools and techniques for performance measurement

Success Criteria:

By the end of this session, you should be able to analyze shader performance, identify bottlenecks, implement optimization strategies, and measure the impact of your improvements across different platforms.

3. Foundation Knowledge (15-20 minutes)

Core Concepts

GPU Architecture Understanding

Modern GPUs process shaders in groups of threads called “warps” (NVIDIA) or “wavefronts” (AMD). Understanding this parallel execution model is crucial for optimization.

Real-World Analogy: Think of a GPU like a factory assembly line with thousands of workers (cores) working in synchronized teams. When one worker in a team needs to wait (for memory access or conditional branching), the entire team must wait. Efficient shaders keep all workers busy simultaneously.

Key Principles:

SIMD Execution: Single Instruction, Multiple Data - all threads in a warp execute the same instruction
Memory Coalescing: Accessing memory in patterns that align with hardware capabilities
Occupancy: Maximizing the number of active warps to hide latency

Performance Bottlenecks in 2D Shaders

ALU (Arithmetic Logic Unit) Bound:

Too many mathematical operations per pixel
Complex trigonometric functions (sin, cos, atan)
Excessive conditional branching

Memory Bound:

Too many texture samples per pixel
Large texture sizes causing cache misses
Random memory access patterns

Bandwidth Bound:

High-resolution textures on mobile devices
Multiple render targets simultaneously
Excessive overdraw (drawing same pixels multiple times)

Key Terminology

Instruction Count: Number of GPU instructions per shader execution
Texture Cache: High-speed memory that stores recently accessed texture data
Overdraw: Rendering pixels multiple times in the same frame
Occupancy: Percentage of GPU cores actively executing shader threads
Memory Coalescing: Efficient memory access pattern that maximizes bandwidth
Static Branching: Compile-time conditional code paths
Dynamic Branching: Runtime conditional code paths (expensive on GPUs)

4. Detailed Explanation and Examples (25-30 minutes)

Mathematical Optimization Strategies

Expensive vs. Cheap Operations

Expensive Operations (avoid when possible):

// Expensive trigonometric functions
float angle = atan(y, x);
float wave = sin(x * 10.0);
float distance = sqrt(x*x + y*y);

// Expensive division
float result = a / b;

// Dynamic branching
if(condition) {
    // Different code path
}

Cheaper Alternatives:

// Use built-in length() instead of manual sqrt
float distance = length(vec2(x, y));

// Use multiplication instead of division
float invB = 1.0 / b; // Calculate once
float result = a * invB;

// Use step() instead of if/else
float selector = step(0.5, condition);
result = mix(valueA, valueB, selector);

// Approximate trigonometry with polynomials for simple cases
float fastSin = x - (x*x*x)/6.0; // Taylor series approximation

Vector Operation Optimization

// Inefficient: Component-wise operations
float r = color.r * brightness;
float g = color.g * brightness;
float b = color.b * brightness;
vec3 result = vec3(r, g, b);

// Efficient: Vector operations
vec3 result = color.rgb * brightness;

// Swizzling is free
vec3 bgr = color.bgr;
vec2 xy = position.xy;

Memory Access Optimization

Texture Sampling Patterns

// Poor: Multiple dependent texture lookups
vec4 sample1 = texture2D(u_texture, uv);
vec4 sample2 = texture2D(u_texture, uv + sample1.xy * 0.1);
vec4 sample3 = texture2D(u_texture, uv + sample2.xy * 0.1);

// Better: Pre-calculate offsets when possible
vec2 offset1 = uv + vec2(0.01, 0.0);
vec2 offset2 = uv + vec2(0.0, 0.01);
vec4 sample1 = texture2D(u_texture, uv);
vec4 sample2 = texture2D(u_texture, offset1);
vec4 sample3 = texture2D(u_texture, offset2);

Precision Optimization

// Use appropriate precision for mobile
precision mediump float; // Default for most calculations

// Use lowp for simple values
lowp float alpha = 0.5;
lowp vec3 simpleColor = vec3(1.0, 0.0, 0.0);

// Use highp only when necessary
highp float preciseCalculation = complex_math();

Conditional Optimization

Branch Reduction Techniques

// Expensive: Dynamic branching
if(distance < radius) {
    color = insideColor;
} else {
    color = outsideColor;
}

// Efficient: Mathematical blending
float mask = step(distance, radius);
color = mix(outsideColor, insideColor, mask);

// Even better: Smooth transitions
float mask = 1.0 - smoothstep(radius * 0.8, radius, distance);
color = mix(outsideColor, insideColor, mask);

Practical Applications

Mobile Game Optimization:

Limit texture samples to 4-6 per fragment
Use compressed texture formats (ETC2, ASTC)
Minimize overdraw through careful UI layering

Web Application Performance:

Consider WebGL 1.0 limitations (no loops, limited precision)
Optimize for integrated graphics commonly found in laptops
Bundle similar shaders to reduce compilation overhead

Desktop Applications:

Leverage higher precision and instruction counts
Use more complex algorithms for higher quality
Implement multiple quality tiers for different hardware

Common Misconceptions

Misconception: “More complex shaders always perform worse” Reality: Sometimes a complex shader can be more efficient than multiple simple passes
Misconception: “Mobile GPUs are just slower desktop GPUs” Reality: Mobile GPUs have different architectures with unique optimization requirements
Misconception: “Optimization should be done at the end of development” Reality: Performance considerations should be built into shader design from the beginning

5. Live Coding Demonstration (20-25 minutes)

Example 1: Basic Performance Profiling

// Unoptimized shader - identify bottlenecks
precision mediump float;

varying vec2 v_uv;
uniform float u_time;
uniform sampler2D u_texture;

float expensiveNoise(vec2 p) {
    float result = 0.0;
    for(int i = 0; i < 10; i++) { // Too many iterations
        float fi = float(i);
        result += sin(p.x * fi) * cos(p.y * fi) / (fi + 1.0); // Expensive trig
    }
    return result;
}

void main() {
    vec2 uv = v_uv;
    
    // Multiple expensive operations
    float noise = expensiveNoise(uv * 10.0 + u_time);
    vec4 tex1 = texture2D(u_texture, uv + noise * 0.1);
    vec4 tex2 = texture2D(u_texture, uv * 2.0);
    vec4 tex3 = texture2D(u_texture, uv * 0.5);
    
    // Complex blending
    vec4 result = tex1;
    if(noise > 0.5) {
        result = mix(tex1, tex2, noise);
    } else {
        result = mix(tex1, tex3, abs(noise));
    }
    
    gl_FragColor = result;
}

Example 2: Optimized Version

// Optimized shader - same visual effect, better performance
precision mediump float;

varying vec2 v_uv;
uniform float u_time;
uniform sampler2D u_texture;

// Simplified noise function
float fastNoise(vec2 p) {
    // Use fewer octaves and simpler math
    return fract(sin(dot(p, vec2(12.9898, 78.233))) * 43758.5453) * 2.0 - 1.0;
}

void main() {
    vec2 uv = v_uv;
    
    // Single noise calculation
    float noise = fastNoise(uv * 5.0 + u_time * 0.5);
    
    // Fewer texture samples with cached UV calculations
    vec2 distortedUV = uv + noise * 0.05;
    vec4 tex1 = texture2D(u_texture, distortedUV);
    vec4 tex2 = texture2D(u_texture, uv * 1.5);
    
    // Use mix instead of conditional
    float blendFactor = noise * 0.5 + 0.5; // Remap to [0,1]
    vec4 result = mix(tex1, tex2, blendFactor);
    
    gl_FragColor = result;
}

Performance Improvements:

Reduced loop iterations from 10 to 0 (eliminated loop)
Replaced expensive trigonometry with simple hash function
Reduced texture samples from 3 to 2
Eliminated conditional branching

Example 3: LOD (Level of Detail) Implementation

// Adaptive quality based on screen resolution or distance
precision mediump float;

varying vec2 v_uv;
uniform float u_time;
uniform float u_qualityLevel; // 0.0 = low, 1.0 = high
uniform vec2 u_resolution;

float adaptiveNoise(vec2 p, float quality) {
    // Adjust complexity based on quality setting
    int octaves = int(mix(2.0, 6.0, quality));
    float result = 0.0;
    float amplitude = 0.5;
    
    for(int i = 0; i < 6; i++) {
        if(i >= octaves) break;
        result += amplitude * fract(sin(dot(p, vec2(12.9898, 78.233))) * 43758.5453);
        p *= 2.0;
        amplitude *= 0.5;
    }
    
    return result;
}

void main() {
    // Adapt quality based on pixel density
    float pixelSize = length(dFdx(v_uv)) * u_resolution.x;
    float dynamicQuality = clamp(1.0 / pixelSize, 0.2, 1.0);
    
    float effectiveQuality = min(u_qualityLevel, dynamicQuality);
    
    vec2 uv = v_uv;
    float noise = adaptiveNoise(uv * 8.0, effectiveQuality);
    
    vec3 color = vec3(noise);
    gl_FragColor = vec4(color, 1.0);
}

Example 4: Memory-Efficient Multi-Effect Shader

// Combining multiple effects efficiently
precision mediump float;

varying vec2 v_uv;
uniform sampler2D u_texture;
uniform float u_brightness;
uniform float u_contrast;
uniform float u_saturation;
uniform vec3 u_colorTint;

void main() {
    // Single texture sample
    vec4 originalColor = texture2D(u_texture, v_uv);
    
    // Efficient color manipulation pipeline
    vec3 color = originalColor.rgb;
    
    // Brightness (simple multiplication)
    color *= u_brightness;
    
    // Contrast (single operation)
    color = (color - 0.5) * u_contrast + 0.5;
    
    // Saturation (efficient desaturate and mix)
    float gray = dot(color, vec3(0.299, 0.587, 0.114));
    color = mix(vec3(gray), color, u_saturation);
    
    // Color tint (simple multiplication)
    color *= u_colorTint;
    
    gl_FragColor = vec4(color, originalColor.a);
}

Debugging and Profiling Example

// Debug shader with performance visualization
precision mediump float;

varying vec2 v_uv;
uniform int u_debugMode; // 0=normal, 1=complexity, 2=texture usage
uniform sampler2D u_texture;

void main() {
    if(u_debugMode == 1) {
        // Visualize computational complexity
        float complexity = 0.0;
        for(int i = 0; i < 10; i++) {
            complexity += sin(v_uv.x * float(i)) * cos(v_uv.y * float(i));
        }
        gl_FragColor = vec4(vec3(complexity * 0.1 + 0.5), 1.0);
    } else if(u_debugMode == 2) {
        // Visualize texture sampling count
        vec4 tex = texture2D(u_texture, v_uv);
        gl_FragColor = vec4(1.0, 0.0, 0.0, 1.0); // Red = 1 sample
    } else {
        // Normal rendering
        gl_FragColor = texture2D(u_texture, v_uv);
    }
}

6. Hands-On Practice (30-40 minutes)

Exercise 1: Performance Bottleneck Identification (15 minutes)

Objective: Analyze and optimize a deliberately inefficient shader

Instructions:

Take the provided “slow shader” with multiple performance issues
Identify at least 5 optimization opportunities
Implement optimizations one by one
Measure performance improvement (if tools available)

Starter Code (Deliberately Inefficient):

precision highp float; // Unnecessarily high precision

varying vec2 v_uv;
uniform float u_time;
uniform sampler2D u_texture;

float slowFunction(vec2 p) {
    float result = 0.0;
    for(int i = 0; i < 20; i++) {
        float angle = atan(p.y, p.x) + float(i) * 0.1;
        result += sin(angle * 10.0) * cos(angle * 15.0);
        result += sqrt(pow(p.x, 2.0) + pow(p.y, 2.0));
    }
    return result / 20.0;
}

void main() {
    vec2 uv = v_uv;
    
    float effect = slowFunction(uv);
    
    vec4 tex1 = texture2D(u_texture, uv);
    vec4 tex2 = texture2D(u_texture, uv + 0.01);
    vec4 tex3 = texture2D(u_texture, uv - 0.01);
    vec4 tex4 = texture2D(u_texture, uv + vec2(0.01, -0.01));
    
    vec4 result;
    if(effect > 0.5) {
        result = tex1 + tex2;
    } else if(effect > 0.0) {
        result = tex2 + tex3;
    } else {
        result = tex3 + tex4;
    }
    
    gl_FragColor = result;
}

Optimization Targets:

Reduce precision where appropriate
Eliminate expensive math functions
Reduce texture samples
Remove conditional branching
Simplify calculations

Exercise 2: Mobile Optimization Challenge (15 minutes)

Objective: Create a mobile-friendly version of a complex effect

Requirements:

Start with a complex shader effect (particles, lighting, etc.)
Create a simplified version suitable for mobile devices
Maintain visual similarity while improving performance
Use appropriate precision qualifiers

Considerations:

Limit texture samples to 4 or fewer
Avoid loops and branches when possible
Use efficient mathematical approximations
Consider using lookup textures for complex functions

Exercise 3: Multi-Platform Shader Variants (10 minutes)

Objective: Implement adaptive quality based on platform capabilities

Challenge:

Create a single shader that adapts its complexity based on uniform parameters
Implement at least 3 quality levels: low, medium, high
Allow runtime switching between quality levels
Document the performance characteristics of each level

Extension Ideas:

Add automatic quality detection based on frame rate
Implement smooth transitions between quality levels
Create platform-specific preprocessor variants

7. Common Pitfalls and Troubleshooting (10 minutes)

Frequent Mistakes

Problem: Shader performs well on desktop but terribly on mobile Cause: Mobile GPUs have different architecture and capabilities Solution: Always test on target devices and use mobile-appropriate techniques
Problem: Optimization makes shader look significantly different Cause: Over-aggressive optimization without visual validation Solution: Use side-by-side comparison and maintain reference images
Problem: Performance varies dramatically across different scenes Cause: Content-dependent complexity (dynamic branching, varying texture access) Solution: Design shaders for worst-case scenarios or implement adaptive quality

Debugging Performance Issues

// Performance measurement shader template
precision mediump float;

uniform float u_enableProfiling;
uniform float u_frameTime; // Passed from application

varying vec2 v_uv;

// Your shader code here
void yourShaderFunction() {
    // Complex operations
}

void main() {
    if(u_enableProfiling > 0.5) {
        // Simple performance indicator
        vec3 perfColor = vec3(1.0, 0.0, 0.0); // Red = expensive
        if(u_frameTime < 16.67) { // 60 FPS
            perfColor = vec3(0.0, 1.0, 0.0); // Green = good
        } else if(u_frameTime < 33.33) { // 30 FPS
            perfColor = vec3(1.0, 1.0, 0.0); // Yellow = acceptable
        }
        gl_FragColor = vec4(perfColor, 1.0);
    } else {
        yourShaderFunction();
    }
}

Platform-Specific Considerations

Mobile (iOS/Android):

Use mediump precision as default
Limit to 4-8 texture samples per fragment
Avoid dependent texture reads
Be careful with fragment discard operations

WebGL:

Consider WebGL 1.0 limitations (no integer loops)
Shader compilation can be slow; cache when possible
Some desktop GPUs may have mobile-like limitations

Console/Desktop:

Can handle more complex calculations
Higher precision available
More texture samples feasible
Consider implementing quality options

8. Best Practices and Professional Tips (5 minutes)

Optimization Workflow

// Professional shader structure with optimization in mind
precision mediump float;

// === PERFORMANCE CONFIGURATION ===
#define MAX_LIGHTS 4
#define ENABLE_SHADOWS 1
#define MOBILE_OPTIMIZED 1

// === PRECISION DECLARATIONS ===
#if MOBILE_OPTIMIZED
    #define PRECISION mediump
#else
    #define PRECISION highp
#endif

// === OPTIMIZED UTILITIES ===
PRECISION float fastLength(vec2 v) {
    #if MOBILE_OPTIMIZED
        return abs(v.x) + abs(v.y); // Manhattan distance approximation
    #else
        return length(v); // Actual Euclidean distance
    #endif
}

void main() {
    // Optimized shader logic
}

Memory Management Best Practices

Texture Atlasing: Combine multiple textures to reduce state changes
Compression: Use platform-appropriate compressed formats
Mipmap Usage: Ensure proper mipmap chains for distant objects
Precision Management: Use lowest acceptable precision for each variable

Performance Testing Strategy

Baseline Measurement: Establish performance metrics before optimization
Incremental Changes: Optimize one aspect at a time
Cross-Platform Testing: Verify improvements across target devices
Regression Testing: Ensure optimizations don’t break functionality
User Experience Validation: Confirm visual quality remains acceptable

Industry Standards

60 FPS Target: Design shaders to maintain 60 FPS on target hardware
Scalable Quality: Implement multiple quality tiers for different devices
Memory Budgets: Stay within platform-specific memory limitations
Power Efficiency: Consider battery life impact on mobile devices

9. Summary and Review (5 minutes)

Key Takeaways

GPU architecture understanding is fundamental to effective shader optimization
Mathematical simplification often provides the biggest performance gains
Memory access patterns significantly impact performance on mobile devices
Platform-specific optimization is essential for cross-platform deployment

Quick Review Questions

What is the difference between ALU-bound and memory-bound shader performance?
Why are conditional branches particularly expensive on GPUs?
What precision qualifier should you typically use for mobile shader development?
How can you reduce texture bandwidth usage without sacrificing visual quality?

Answers:

ALU-bound = too many calculations; memory-bound = too many texture accesses or cache misses
GPUs execute threads in groups (warps); branches cause thread divergence and reduced efficiency
mediump float provides good balance of precision and performance on mobile
Use texture atlasing, compression, mipmaps, and reduce unnecessary texture samples

For Instructors

Teaching Tips

Use real performance data: Show actual frame time measurements when possible
Provide target hardware: Test optimizations on representative devices
Emphasize measurement: “Profile first, optimize second” methodology
Show visual comparisons: Demonstrate that optimization doesn’t mean visual compromise

Common Student Questions

“How do I know if my shader is fast enough?” Establish target frame rates and measure consistently
“Should I always use the lowest precision?” Balance precision needs with performance requirements
“Is optimization worth the complexity?” Depends on target platform and performance requirements

Assessment Ideas

Optimization challenges: Provide slow shaders for students to improve
Platform comparison: Test same shader across different device classes
Performance budgets: Give students specific performance targets to meet

Adaptation Notes

For mobile developers: Emphasize mobile-specific optimization techniques
For web developers: Focus on WebGL limitations and browser compatibility
Online delivery: Provide pre-configured performance testing environments
Time management: Theory (20 min), demonstrations (25 min), hands-on practice (35 min), review (10 min)