Machine Learning chevron_right Training a Simple Neural Network with a Framework

Training a Simple Neural Network with a Framework

calendar_today schedule 3 min read

Learning Objectives: By the end of this lesson, you will be able to:

  • Load and preprocess a basic dataset (e.g., MNIST or Iris)
  • Train a neural network model using .fit() in Keras or a manual loop in PyTorch
  • Monitor training progress and evaluate performance

1. Welcome and Introduction (5 minutes)

Icebreaker Activity

  • Think back to your first experience training a model: was it exciting or intimidating?

Lesson Overview

  • Duration: 45 minutes
  • Prerequisites: Lesson on framework setup and basic model definition
  • Materials Needed: Python environment, Jupyter Notebook or script

2. Learning Objectives and Session Outline (5 minutes)

What We’ll Cover Today:

  1. Dataset loading and preprocessing
  2. Training with .fit() or manual loop
  3. Monitoring loss and accuracy
  4. Evaluating on test data
  5. Visualizing training curves

Success Criteria:

  • Ability to load data and train a model end-to-end with a deep learning framework

3. Foundation Knowledge (15-20 minutes)

Core Concepts

Data Pipelines

  • Keras: tf.keras.datasets.mnist.load_data() returns training and test arrays.
  • PyTorch: torchvision.datasets.MNIST loads each image and label; a DataLoader batches the samples and can shuffle the training set.

Both examples scale pixel values to [0, 1]. Training loss and validation metrics monitor learning during training; evaluate on the held-out test set only after training and tuning are complete.

Key Terminology

  • Batch Size: Number of samples per gradient update
  • Epoch: One pass through the full training dataset
  • Validation Split: Fraction of data for validation during training

4. Detailed Explanation and Examples (25-30 minutes)

Understanding Training APIs

Keras Example:

from tensorflow.keras import Sequential
from tensorflow.keras.datasets import mnist
from tensorflow.keras.layers import Dense, Flatten, Input

# Load images and integer digit labels
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = x_train.astype("float32") / 255.0
x_test = x_test.astype("float32") / 255.0

model = Sequential([
    Input(shape=(28, 28)),
    Flatten(),
    Dense(128, activation="relu"),
    Dense(10, activation="softmax"),
])
model.compile(
    optimizer="adam",
    loss="sparse_categorical_crossentropy",
    metrics=["accuracy"],
)

history = model.fit(
    x_train, y_train,
    epochs=5,
    batch_size=32,
    validation_split=0.1,
)
test_loss, test_accuracy = model.evaluate(x_test, y_test, verbose=0)
print(f"Test loss: {test_loss:.4f}; test accuracy: {test_accuracy:.4f}")

PyTorch Example:

import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import datasets, transforms
from torch.utils.data import DataLoader

transform = transforms.ToTensor()  # MNIST pixels are converted to float values in [0, 1]
train_dataset = datasets.MNIST("data", train=True, download=True, transform=transform)
test_dataset = datasets.MNIST("data", train=False, download=True, transform=transform)
train_loader = DataLoader(train_dataset, batch_size=32, shuffle=True)
test_loader = DataLoader(test_dataset, batch_size=256, shuffle=False)

class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.flatten = nn.Flatten()
        self.hidden = nn.Linear(28 * 28, 128)
        self.relu = nn.ReLU()
        self.output = nn.Linear(128, 10)

    def forward(self, x):
        x = self.flatten(x)
        x = self.relu(self.hidden(x))
        return self.output(x)  # raw logits for CrossEntropyLoss

model = Net()
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)

for epoch in range(5):
    model.train()
    running_loss = 0.0
    examples_seen = 0
    for images, labels in train_loader:
        optimizer.zero_grad()
        logits = model(images)
        loss = criterion(logits, labels)
        loss.backward()
        optimizer.step()
        running_loss += loss.item() * labels.size(0)
        examples_seen += labels.size(0)
    print(f"Epoch {epoch + 1}: train loss={running_loss / examples_seen:.4f}")

# Evaluate once on the held-out test set after training
model.eval()
correct = 0
examples_seen = 0
with torch.no_grad():
    for images, labels in test_loader:
        logits = model(images)
        predictions = logits.argmax(dim=1)
        correct += (predictions == labels).sum().item()
        examples_seen += labels.size(0)

test_accuracy = correct / examples_seen
print(f"Test accuracy: {test_accuracy:.4f}")

Common Misconceptions:

  1. Misconception: More epochs always mean better model Reality: After a point, the model may overfit
  2. Misconception: Validation split is test accuracy Reality: Validation is for tuning; test set remains unseen

5. Live Coding Demonstration (20-25 minutes)

Example 1: Training Curve Visualization

import matplotlib.pyplot as plt
plt.plot(history.history['loss'], label='train loss')
plt.plot(history.history['val_loss'], label='val loss')
plt.legend(); plt.show()

Debugging Demonstration

# Common mistake: not shuffling data
data_loader = DataLoader(dataset, batch_size=32, shuffle=False)
# You may see poor convergence
# Correct: shuffle=True

6. Hands-On Practice (30-40 minutes)

Exercise 1: Basic Practice (10 minutes)

Objective: Train a model on Iris using Keras.

Exercise 2: Intermediate Challenge (15 minutes)

Objective: Implement manual training loop in PyTorch.

Exercise 3: Creative Application (15 minutes)

Objective: Compare SGD vs. Adam on the same dataset.