Machine Learning chevron_right Introduction to a Deep Learning Framework

Introduction to a Deep Learning Framework

calendar_today schedule 5 min read

Learning Objectives: By the end of this lesson, you will be able to:

  • Set up and configure a deep learning framework (PyTorch or TensorFlow/Keras)
  • Define and inspect a simple neural network model using the framework’s API
  • Understand the workflow for compiling/training models in the framework

1. Welcome and Introduction (5 minutes)

Icebreaker Activity

  • Have you ever wondered how frameworks like PyTorch or TensorFlow make it so easy to train deep neural networks with just a few lines of code?

Lesson Overview

  • Duration: 45 minutes
  • Prerequisites: Python programming, NumPy basics, understanding of feedforward networks
  • Materials Needed: Python 3.x, pip, VS Code or Jupyter Notebook

2. Learning Objectives and Session Outline (5 minutes)

What We’ll Cover Today:

  1. Installing and importing PyTorch or TensorFlow/Keras
  2. Defining a simple neural network architecture
  3. Compiling the model (TensorFlow) or preparing optimizer/loss (PyTorch)
  4. Inspecting model summary and parameters
  5. Hands-on practice: build and compile a basic network

Success Criteria:

By the end of this session, you should be able to set up the framework, define a simple model, and prepare it for training.

3. Foundation Knowledge (15-20 minutes)

Core Concepts

Framework Installation

  • pip install torch torchvision (PyTorch) or pip install tensorflow
  • Real-World Analogy: Frameworks are like a car engine—providing power and components so you don’t build every part from scratch.
  • Why This Matters: Deep learning frameworks handle complex tensor operations, GPU acceleration, and backpropagation automatically.

Model Definition API

  • Sequential vs. Functional API in Keras
  • nn.Module subclassing in PyTorch

Key Terminology

  • Tensor: Multi-dimensional array used for all computations
  • Optimizer: Algorithm to update weights (e.g., SGD, Adam)
  • Loss Function: Metric to quantify error (e.g., Cross-Entropy)

4. Detailed Explanation and Examples (25-30 minutes)

Understanding the API for Model Creation

TensorFlow/Keras Example:

from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense, Input

model = Sequential([
    Input(shape=(784,)),
    Dense(64, activation='relu'),
    Dense(10, activation='softmax')
])
model.summary()

PyTorch Example:

import torch.nn as nn

class SimpleNet(nn.Module):
    def __init__(self):
        super(SimpleNet, self).__init__()
        self.fc1 = nn.Linear(784, 64)
        self.relu = nn.ReLU()
        self.fc2 = nn.Linear(64, 10)
        self.softmax = nn.LogSoftmax(dim=1)

    def forward(self, x):
        x = self.relu(self.fc1(x))
        return self.softmax(self.fc2(x))

model = SimpleNet()
print(model)

Common Misconceptions

  1. Misconception: The framework trains the model automatically without configuration. Reality: You must specify optimizer, loss function, and invoke the training loop (or .fit()).
  2. Misconception: TensorFlow and PyTorch place work on devices in the same way. Reality: TensorFlow generally places operations on available devices automatically, with explicit placement available when needed. PyTorch examples commonly select a device and move both the model and tensors to it with .to(device).

5. Live Coding Demonstration (20-25 minutes)

Example 1: Basic Model Setup (TensorFlow)

from tensorflow.keras.optimizers import Adam

model.compile(optimizer=Adam(learning_rate=0.001),
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

Explanation:

  • Line 1: Chooses Adam optimizer with learning rate
  • Line 2: Specifies loss and evaluation metric

Example 2: Basic Model Setup (PyTorch)

import torch.optim as optim
criterion = nn.NLLLoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)

Debugging Demonstration

from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense, Input

# An explicit Input makes the expected feature shape clear and builds the model.
model = Sequential([Input(shape=(n_features,)), Dense(10, activation='relu')])
model.summary()

A Sequential model without an input shape can be built lazily when it first receives data. Adding Input is useful for declaring and inspecting the shape early; omitting it does not by itself cause a “Graph disconnected” error.

6. Hands-On Practice (30-40 minutes)

Exercise 1: Basic Practice (10 minutes)

Objective: Define and compile a 3-layer network in Keras. Instructions:

  1. Use input size 100 and output size 5.
  2. Use ReLU activations and softmax at the end.
  3. Compile with categorical_crossentropy.
def build_model():
    model = Sequential([
        Input(shape=(100,)),
        Dense(128, activation='relu'),
        Dense(64, activation='relu'),
        Dense(5, activation='softmax'),
    ])
    model.compile(
        optimizer='adam',
        loss='categorical_crossentropy',
        metrics=['accuracy'],
    )
    return model

Exercise 2: Intermediate Challenge (15 minutes)

Objective: Create and train a PyTorch nn.Module network on random data.

Requirements:

  • Use two hidden layers
  • Use MSELoss with synthetic regression targets
  • Train for 5 epochs and report loss

Hints:

  • Remember to call optimizer.zero_grad() before loss.backward()
  • Use torch.randn to generate input

Exercise 3: Creative Application (15 minutes)

Objective: Build a small classifier for the Iris dataset using your chosen framework.

Challenge: Load Iris from sklearn, preprocess labels, and train.

Extension Ideas:

  • Add dropout layers
  • Experiment with different activations

7. Common Pitfalls and Troubleshooting (10 minutes)

Frequent Mistakes

  1. Problem: Mismatched tensor shapes Cause: Incorrect input_shape or dimension ordering Solution: Print model.summary() or print(x.shape) at each stage

  2. Problem (PyTorch): GPU not utilized Cause: The model or a tensor remains on the CPU. Solution: Select a device and move both the model and each batch to it, for example device = torch.device('cuda' if torch.cuda.is_available() else 'cpu'), then use model.to(device) and x.to(device). TensorFlow generally places operations on available devices automatically.

Debugging Tips

  • Inspect model.summary() or print(model)
  • Use breakpoints or print() in the forward method

8. Best Practices and Professional Tips (5 minutes)

Industry Standards

  • Use configuration scripts (YAML/JSON) to track hyperparameters
  • Modularize model code for reusability
  • Version control your training scripts and artifacts

Next Steps in Your Learning Journey

  • Explore CNNs and RNNs in the chosen framework
  • Learn how to deploy models with TensorFlow Serving or TorchServe

9. Summary and Review (5 minutes)

Key Takeaways

  1. Frameworks abstract tensor ops and backprop for you
  2. You must still configure optimizer, loss, and training loop
  3. Debugging shape errors and device placement are critical skills

Quick Review Questions

  1. How do you specify an input shape in Keras?
  2. What does optimizer.zero_grad() do in PyTorch?
  3. How would you move a model to GPU?

For Instructors

Teaching Tips

  • Demonstrate both TensorFlow and PyTorch side by side
  • Encourage students to type code instead of copy/paste

Assessment Ideas

  • Short quiz on model definition syntax
  • Peer review: each student reviews another’s model

Adaptation Notes

  • For beginners: focus on Keras .fit() API
  • For advanced: dive into custom training loops in PyTorch