LLM chevron_right Full Fine-tuning vs. Parameter-Efficient Fine-Tuning (PEFT) in LLMs

Full Fine-tuning vs. Parameter-Efficient Fine-Tuning (PEFT) in LLMs

calendar_today schedule 8 min read

Learning Objectives: By the end of this lesson, you will be able to:

  • Explain the difference between full fine-tuning and parameter-efficient fine-tuning (PEFT) for large language models (LLMs)
  • Identify the pros and cons of each approach
  • Recognize scenarios where PEFT is preferable and understand its motivation

1. Welcome and Introduction (5 minutes)

Icebreaker Activity

“Imagine you have a huge language model with billions of parameters. If you want to adapt it to a new task, do you think you need to update every single parameter? Why or why not?”

Lesson Overview

  • Duration: Approximately 2.5 hours including all exercises (or 60–90 minutes when selected activities are assigned)
  • Prerequisites: Understanding of LLM architectures (BERT, GPT), basics of model training and transfer learning
  • Materials Needed: Python environment (Jupyter or Colab), Hugging Face Transformers library, sample datasets

2. Learning Objectives and Session Outline (5 minutes)

What We’ll Cover Today:

  1. Full fine-tuning: what it is, how it works, and its limitations
  2. The motivation for parameter-efficient fine-tuning (PEFT)
  3. Overview of PEFT strategies
  4. Hands-on practice and exercises
  5. Common pitfalls and how to avoid them

Success Criteria:

By the end of this session, you should be able to describe and compare full fine-tuning and PEFT, and know when to use each.

3. Foundation Knowledge (15-20 minutes)

Core Concepts

Full Fine-tuning

Full fine-tuning means updating all the parameters of a pre-trained model on a new dataset or task. This is the traditional approach for adapting models.

Real-World Analogy: Think of retraining a whole orchestra to play a new song, rather than just teaching a few new instruments.

Why This Matters: Full fine-tuning can yield the best performance, but is resource-intensive and requires storing a new copy of the entire model for each task.

Parameter-Efficient Fine-Tuning (PEFT)

PEFT refers to techniques that adapt a model to new tasks by updating only a small subset of parameters, or by adding small trainable modules, while keeping most of the original model frozen.

Real-World Analogy: Instead of retraining the whole orchestra, you just swap out or retrain a few musicians or add a new section.

Why This Matters: PEFT enables efficient adaptation of very large models, saving compute, memory, and storage.

Key Terminology

  • Fine-tuning: Adapting a pre-trained model to a new task by further training on task-specific data.
  • Frozen parameters: Model weights that are not updated during training.
  • Adapters/LoRA: Examples of PEFT modules that are added to or modify the model for efficient adaptation.

4. Detailed Explanation and Examples (25-30 minutes)

Understanding Full Fine-tuning

Full fine-tuning involves unfreezing all model parameters and training them on a new dataset. This can lead to high accuracy but is expensive for large models.

Practical Applications

  • Adapting a BERT or GPT model to a specific domain (e.g., legal, medical)
  • Customizing a model for a proprietary dataset

Common Misconceptions

  1. Misconception: Full fine-tuning is always better. Reality: For large models, PEFT can sometimes approach full fine-tuning quality with much less compute and storage; validate this per task.
  2. Misconception: You must always update all parameters to adapt a model. Reality: Many tasks only require small changes, which PEFT can provide.

The Need for PEFT

As LLMs grow, full fine-tuning becomes impractical:

  • Compute: Updating billions of parameters is slow and expensive.
  • Storage: Each fine-tuned model is as large as the original.
  • Deployment: Serving many task-specific models is costly.

PEFT addresses these by:

  • Only updating a small number of parameters
  • Adding small modules (e.g., adapters, LoRA)
  • Allowing multiple tasks to share the same base model

5. Live Coding Demonstration (20-25 minutes)

Example 1: Full Fine-tuning a BERT Model

from transformers import AutoTokenizer, BertForSequenceClassification, Trainer, TrainingArguments
from datasets import load_dataset

# Load pre-trained BERT and dataset
model = BertForSequenceClassification.from_pretrained('bert-base-uncased')
dataset = load_dataset('imdb')
tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')
tokenized = dataset.map(
    lambda batch: tokenizer(batch['text'], truncation=True, max_length=256),
    batched=True,
    remove_columns=['text'],
)

# Define training arguments
training_args = TrainingArguments(
    output_dir='./results',
    num_train_epochs=1,
    per_device_train_batch_size=8,
    evaluation_strategy="epoch",
    save_strategy="epoch",
)

# Trainer for full fine-tuning
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized['train'].shuffle(seed=42).select(range(1000)),
    eval_dataset=tokenized['test'].shuffle(seed=42).select(range(500)),
)

trainer.train()

Explanation:

  • Lines 1-2: Import necessary libraries and load a pre-trained BERT model and dataset.
  • Lines 5-13: Set up training arguments.
  • Lines 15-22: Create a Trainer object for full fine-tuning.
  • Line 24: Start training, updating all model parameters.

Example 2: Motivation for PEFT

Suppose you want to fine-tune a 7B parameter model for 10 different tasks. Full fine-tuning would require storing 10 copies (70B parameters). With PEFT, you only store small adapters or LoRA weights for each task (often <1% of the model size).

Example 3: PEFT with LoRA

from peft import LoraConfig, TaskType, get_peft_model
from transformers import BertForSequenceClassification

model = BertForSequenceClassification.from_pretrained('bert-base-uncased')
lora_config = LoraConfig(
    task_type=TaskType.SEQ_CLS,
    r=8,
    lora_alpha=16,
    target_modules=["query", "value"],
)
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()

Key Points:

  • Only the adapter parameters are updated; the rest of the model is frozen.
  • Much less memory and compute required.
  • Multiple adapters can be added for different tasks.

Debugging Demonstration

Show a common mistake: forgetting to freeze the base model when using adapters.

# `train()` changes module mode; it does not select trainable parameters.
model.train()
trainable = [name for name, p in model.named_parameters() if p.requires_grad]
assert trainable and all("lora_" in name or "modules_to_save" in name
                         for name in trainable)
print(trainable)

6. Hands-On Practice (30-40 minutes)

Exercise 1: Basic Practice (10 minutes)

Objective: Fine-tune a small model on a text classification task using full fine-tuning.

Instructions:

  1. Load a pre-trained DistilBERT model for sequence classification.
  2. Fine-tune it on a small dataset (e.g., IMDb, AG News).
  3. Evaluate the model on a test set.

Starter Code:

from transformers import DistilBertForSequenceClassification, Trainer, TrainingArguments
from datasets import load_dataset

model = DistilBertForSequenceClassification.from_pretrained('distilbert-base-uncased')
dataset = load_dataset('ag_news')
# ...set up Trainer and train as above...

Solution:

from transformers import DistilBertForSequenceClassification, Trainer, TrainingArguments
from datasets import load_dataset

model = DistilBertForSequenceClassification.from_pretrained('distilbert-base-uncased')
dataset = load_dataset('ag_news')

training_args = TrainingArguments(
    output_dir='./results',
    num_train_epochs=1,
    per_device_train_batch_size=8,
    evaluation_strategy="epoch",
    save_strategy="epoch",
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=dataset['train'].shuffle(seed=42).select(range(1000)),
    eval_dataset=dataset['test'].shuffle(seed=42).select(range(500)),
)

trainer.train()

Exercise 2: Intermediate Challenge (15 minutes)

Objective: Compare the storage requirements of full fine-tuning vs. PEFT.

Requirements:

  • Calculate the disk space needed to store 5 fully fine-tuned models (e.g., BERT-base, ~400MB each).
  • Calculate the space needed for 5 adapter modules (typically <10MB each).
  • Discuss the implications for deploying many models.

Hints:

  • Use os.path.getsize() in Python to check file sizes.
  • Consider the cost for 100 tasks.

Exercise 3: Creative Application (15 minutes)

Objective: Propose a scenario where PEFT would be essential.

Challenge: Describe a real-world application (e.g., a multilingual chatbot for 50 languages) and explain why PEFT is the only practical solution.

Extension Ideas:

  • Sketch how you would organize adapters or LoRA modules for each language.
  • Consider how you would update or add new tasks efficiently.

7. Common Pitfalls and Troubleshooting (10 minutes)

Frequent Mistakes

  1. Problem: Accidentally updating all model parameters during PEFT. Cause: Not freezing the base model or using the wrong training method. Solution: Use the correct training API (e.g., train_adapter) and verify which parameters are trainable.

  2. Problem: Storage bloat from full fine-tuning. Cause: Saving entire model checkpoints for each task. Solution: Use PEFT to only store small task-specific modules.

Debugging Tips

  • Always check which parameters are being updated (model.parameters() and their requires_grad attribute).
  • Use model summaries to verify adapter/LoRA modules are present and active.
  • Read error messages carefully when integrating PEFT libraries.

8. Best Practices and Professional Tips (5 minutes)

Industry Standards

  • Use PEFT for large models and multi-task scenarios.
  • Document which modules are trained and how to load/apply them.
  • Monitor performance: sometimes full fine-tuning is still needed for highly specialized tasks.

Next Steps in Your Learning Journey

  • Explore specific PEFT methods (LoRA, Adapters, Prompt Tuning)
  • Try Hugging Face’s peft library
  • Read recent papers on efficient model adaptation

Current PEFT Workflow and Expectations

Tokenize datasets into model fields (input_ids, attention_mask, and labels) before passing them to Trainer; raw text is not accepted automatically without a preprocessing transform or specialized trainer. With Hugging Face PEFT, create a task-appropriate configuration (for example LoraConfig), call get_peft_model, and verify the boundary with model.print_trainable_parameters() plus inspection of requires_grad. PEFT can approach full fine-tuning on some tasks, but comparable quality is an empirical result—not a guarantee.

9. Summary and Review (5 minutes)

Key Takeaways

  1. Full fine-tuning updates all model parameters; PEFT updates only a small subset or adds small modules.
  2. PEFT is essential for scaling LLM adaptation to many tasks or users.
  3. Choosing the right fine-tuning strategy depends on resources, deployment needs, and task requirements.

Quick Review Questions

  1. What is the main difference between full fine-tuning and PEFT?
  2. Why is PEFT important for large language models?
  3. Give an example scenario where PEFT is preferable to full fine-tuning.

For Instructors

Teaching Tips

  • Emphasize the practical limitations of full fine-tuning at scale
  • Use real-world analogies to clarify PEFT concepts
  • Encourage students to experiment with both approaches

Assessment Ideas

  • Quiz: Compare and contrast full fine-tuning and PEFT
  • Project: Fine-tune a model using both methods and report on resource usage
  • Peer review: Present a PEFT use case and critique each other’s solutions

Adaptation Notes

  • For beginners: Focus on conceptual understanding and simple code examples
  • For advanced students: Dive into the internals of PEFT libraries and custom module design
  • Online: Use Colab for hands-on exercises; In-person: Pair programming and group discussions