LLM chevron_right Parameter-Efficient Fine-Tuning: LoRA and Adapter Modules

Parameter-Efficient Fine-Tuning: LoRA and Adapter Modules

calendar_today schedule 8 min read

Learning Objectives: By the end of this lesson, you will be able to:

  • Explain the motivation for parameter-efficient fine-tuning (PEFT) in large language models (LLMs)
  • Describe and implement Low-Rank Adaptation (LoRA) and Adapter modules
  • Compare PEFT methods with full fine-tuning in terms of efficiency, performance, and practical trade-offs

1. Welcome and Introduction (5 minutes)

Icebreaker Activity

Have you ever tried to fine-tune a large model and run out of memory or patience? What do you think are the biggest challenges in adapting huge models to new tasks?

Lesson Overview

  • Duration: Approximately 2.5 hours including all exercises (or 60–90 minutes when selected activities are assigned)
  • Prerequisites: Understanding of basic neural network fine-tuning, Transformer architecture, and Python (PyTorch or TensorFlow)
  • Materials Needed: Python environment (Jupyter Notebook or script), Hugging Face Transformers and PEFT libraries, GPU access recommended

2. Learning Objectives and Session Outline (5 minutes)

What We’ll Cover Today:

  1. Motivation for Parameter-Efficient Fine-Tuning (PEFT) – Why do we need it?
  2. LoRA: Low-Rank Adaptation – How does it work?
  3. Adapter Modules – What are they and how are they used?
  4. Hands-on practice: Applying LoRA to a real model
  5. Common pitfalls and troubleshooting

Success Criteria:

By the end of this session, you should be able to implement LoRA or Adapters for a downstream task and explain their advantages over full fine-tuning.

3. Foundation Knowledge (15-20 minutes)

Core Concepts

Why Parameter-Efficient Fine-Tuning?

Fine-tuning large models (like BERT, GPT, or Llama) for every new task is expensive in terms of compute, memory, and storage. Full fine-tuning means updating all model parameters, which can be hundreds of millions or even billions. This is often impractical for organizations with limited resources or when deploying many task-specific models.

Real-World Analogy: Imagine you have a massive, complex machine (like a car factory) and you want to make it produce a slightly different car. Full fine-tuning is like re-engineering the entire factory for every new model. PEFT is like adding a few specialized tools or attachments to the existing machinery—much more efficient!

Why This Matters:

  • Reduces hardware requirements
  • Enables faster adaptation to new tasks
  • Makes it feasible to deploy many specialized models
  • Reduces storage costs (only small adapters or LoRA weights need to be saved)

Full Fine-Tuning vs. PEFT

  • Full Fine-Tuning: All model weights are updated. High compute/storage cost.
  • PEFT: Only a small subset of parameters are updated or added. The base model remains mostly frozen.

Key Terminology

  • Adapter Module: A small neural network inserted into each layer of a Transformer, trained for the new task while the main model is frozen.
  • LoRA (Low-Rank Adaptation): A technique that injects trainable low-rank matrices into existing weights, allowing efficient adaptation.
  • Frozen Weights: Model parameters that are not updated during training.

4. Detailed Explanation and Examples (25-30 minutes)

Understanding LoRA and Adapter Modules

LoRA: Low-Rank Adaptation

LoRA proposes that instead of updating the full weight matrices in a model, we can add two small trainable matrices (A and B) with much lower rank. During training, only these matrices are updated, while the original weights remain frozen. At inference, the effect of LoRA can be merged into the original weights for efficiency.

Mathematical Formulation: If W is a weight matrix, LoRA modifies it as W' = W + (alpha / r) BA, where r is the low rank and alpha / r is the usual scaling factor:

W' = W + ΔW, where ΔW = (alpha / r) BA

Here, B and A are low-rank matrices (e.g., rank 4 or 8), so the number of trainable parameters is much smaller than W.

Adapter Modules

Adapters are small neural networks (often a down-projection, non-linearity, and up-projection) inserted into each Transformer layer. Only the adapters are trained for the new task; the rest of the model is frozen. At inference, the adapters can be kept or removed (with some loss in performance).

Adapter Structure:

  • Down-project input to a smaller dimension
  • Apply non-linearity (e.g., ReLU)
  • Up-project back to original dimension

Key Terminology

  • Rank (in LoRA): The dimension of the low-rank matrices. Lower rank = fewer parameters.
  • Down-Projection/Up-Projection: Reducing/increasing the dimensionality in adapters.
  • PEFT Library: Hugging Face’s library for LoRA and several other parameter-efficient methods. Bottleneck-adapter APIs may come from a separate adapter library.

5. Live Coding Demonstration (20-25 minutes)

Example 1: Applying LoRA to a Transformer Model

from transformers import AutoModelForSequenceClassification, AutoTokenizer
from peft import LoraConfig, get_peft_model, TaskType

# Load a pre-trained model and tokenizer
model_name = 'bert-base-uncased'
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=2)

# Configure LoRA
lora_config = LoraConfig(
    task_type=TaskType.SEQ_CLS,  # Sequence classification
    r=8,                         # Rank
    lora_alpha=16,               # Scaling
    lora_dropout=0.1,            # Dropout
    target_modules=["query", "value"]  # Apply to attention layers
)

# Wrap the model with LoRA
lora_model = get_peft_model(model, lora_config)
lora_model.print_trainable_parameters()

Explanation:

  • The base BERT model is loaded and frozen.
  • LoRA is configured to inject low-rank matrices into the attention layers.
  • Only the LoRA parameters are trainable, drastically reducing the number of updated parameters.

Example 2: Training with LoRA

from transformers import TrainingArguments, Trainer

# Prepare your dataset (dummy example)
train_texts = ["I love this!", "I hate that!"]
train_labels = [1, 0]

# Tokenize
train_encodings = tokenizer(train_texts, truncation=True, padding=True)
from datasets import Dataset
dataset = Dataset.from_dict({**train_encodings, "labels": train_labels})

# Training arguments
training_args = TrainingArguments(
    output_dir='./results',
    num_train_epochs=1,
    per_device_train_batch_size=2,
    logging_dir='./logs',
    logging_steps=10,
)

# Trainer
trainer = Trainer(
    model=lora_model,
    args=training_args,
    train_dataset=dataset,
)

trainer.train()

Key Points:

  • Only LoRA parameters are updated.
  • Training is much faster and uses less memory.
  • The base model can be reused for other tasks.

Example 3: Verifying the PEFT Boundary

trainable = [(name, p.numel()) for name, p in lora_model.named_parameters()
             if p.requires_grad]
assert trainable and all("lora_" in name for name, _ in trainable)
print(trainable)

The current Hugging Face PEFT workflow uses get_peft_model with a supported PEFT configuration such as LoraConfig. Bottleneck adapters are a related method, but their API depends on the adapter library being used; do not assume an AdapterConfig exists in peft.

Debugging Demonstration

Show a common mistake: forgetting to freeze the base model.

# Common mistake: accidentally updating all parameters
for param in model.parameters():
    param.requires_grad = True  # This should be False for PEFT!

# Correct way:
for param in model.parameters():
    param.requires_grad = False
# Only LoRA/Adapter params should be trainable

6. Hands-On Practice (30-40 minutes)

Exercise 1: Basic Practice (10 minutes)

Objective: Apply LoRA to a pre-trained model for a simple classification task.

Instructions:

  1. Load a pre-trained BERT or DistilBERT model for sequence classification.
  2. Configure LoRA with rank 4 and apply it to the attention layers.
  3. Print the number of trainable parameters.

Starter Code:

from transformers import AutoModelForSequenceClassification, AutoTokenizer
from peft import LoraConfig, get_peft_model, TaskType

model_name = 'distilbert-base-uncased'
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=2)

# Your LoRA config and wrapping code here

Solution:

lora_config = LoraConfig(
    task_type=TaskType.SEQ_CLS,
    r=4,
    lora_alpha=8,
    lora_dropout=0.1,
    target_modules=["q_lin", "v_lin"]
)
lora_model = get_peft_model(model, lora_config)
lora_model.print_trainable_parameters()

Exercise 2: Intermediate Challenge (15 minutes)

Objective: Fine-tune a LoRA-wrapped model on a small custom dataset.

Requirements:

  • Prepare a small dataset (e.g., 10 sentences, binary labels)
  • Tokenize and create a DataLoader
  • Train for 1 epoch and evaluate accuracy

Hints:

  • Use PyTorch DataLoader
  • Use Trainer from Hugging Face

Exercise 3: Creative Application (15 minutes)

Objective: Compare LoRA and Adapter modules on the same task.

Challenge:

  • Implement both LoRA and Adapter on a classification model
  • Compare number of trainable parameters, training time, and accuracy

Extension Ideas:

  • Try different ranks or adapter sizes
  • Apply to a different model (e.g., RoBERTa)

7. Common Pitfalls and Troubleshooting (10 minutes)

Frequent Mistakes

  1. Problem: Accidentally updating all model parameters Cause: Not freezing the base model Solution: Ensure only LoRA/Adapter parameters are trainable

  2. Problem: Applying LoRA to the wrong modules Cause: Incorrect target_modules list Solution: Check model architecture and documentation

Debugging Tips

  • Use print_trainable_parameters() to verify which parameters are being updated
  • Monitor GPU memory usage
  • Start with a small rank/adapter size and scale up

8. Best Practices and Professional Tips (5 minutes)

Industry Standards

  • Use PEFT for rapid prototyping and deployment
  • Save only the adapter/LoRA weights for storage efficiency
  • Document which modules were adapted

Next Steps in Your Learning Journey

  • Explore prompt tuning and prefix tuning
  • Try PEFT on generative tasks (e.g., text generation)
  • Read the original LoRA and Adapter papers

9. Summary and Review (5 minutes)

Key Takeaways

  1. PEFT methods like LoRA and Adapters enable efficient adaptation of large models
  2. Only a small number of parameters are updated, saving compute and storage
  3. LoRA and Adapters can be easily implemented with Hugging Face’s PEFT library

Quick Review Questions

  1. What is the main advantage of PEFT over full fine-tuning?
  2. How does LoRA reduce the number of trainable parameters?
  3. What is the structure of an Adapter module?

For Instructors

Teaching Tips

  • Emphasize the practical benefits of PEFT
  • Encourage students to experiment with different ranks/sizes
  • Address confusion about which parameters are updated

Assessment Ideas

  • Quiz: Identify which modules are adapted in a given code snippet
  • Project: Fine-tune a model for a new task using LoRA or Adapters
  • Peer review: Compare results and discuss trade-offs

Adaptation Notes

  • For beginners: Focus on LoRA, skip Adapter details
  • For advanced: Dive into mathematical details and custom PEFT implementations
  • Online: Provide Colab notebooks; In-person: Live coding demos