LLM chevron_right Prompt Tuning, Prefix Tuning, and P-Tuning: Soft Prompts for LLM Adaptation

Prompt Tuning, Prefix Tuning, and P-Tuning: Soft Prompts for LLM Adaptation

calendar_today schedule 6 min read

Learning Objectives: By the end of this lesson, you will be able to:

  • Explain the concepts of prompt tuning, prefix tuning, and P-tuning for large language models (LLMs)
  • Implement soft prompt-based parameter-efficient fine-tuning (PEFT) methods
  • Compare soft prompt methods with LoRA/Adapters and understand their trade-offs

1. Welcome and Introduction (5 minutes)

Icebreaker Activity

Have you ever tried to get a language model to do something new just by changing the prompt? What was the most creative prompt you used, and did it work as expected?

Lesson Overview

  • Duration: Approximately 2.5 hours including all exercises (or 60–90 minutes when selected activities are assigned)
  • Prerequisites: Familiarity with LLMs, Transformers, and basic fine-tuning concepts
  • Materials Needed: Python environment, Hugging Face Transformers and PEFT libraries, GPU access recommended

2. Learning Objectives and Session Outline (5 minutes)

What We’ll Cover Today:

  1. Motivation for soft prompt-based PEFT
  2. Prompt Tuning, Prefix Tuning, and P-Tuning: What are they?
  3. How soft prompts modify input representations
  4. Hands-on practice: Implementing prompt tuning
  5. Comparing with LoRA/Adapters and common pitfalls

Success Criteria:

By the end of this session, you should be able to implement and explain soft prompt-based PEFT methods and know when to use them.

3. Foundation Knowledge (15-20 minutes)

Core Concepts

What is Prompt Tuning?

Prompt tuning is a parameter-efficient method for adapting LLMs to new tasks by learning a small set of task-specific vectors (soft prompts) that are prepended to the input. The base model remains frozen; only the soft prompts are trained.

Real-World Analogy: Imagine you want to teach a robot a new trick, but you can’t change its programming. Instead, you give it a special set of instructions at the start of every task—these instructions are like soft prompts!

Why This Matters:

  • Requires very few trainable parameters
  • Fast to train and easy to store/deploy
  • Useful when you need to adapt a model to many tasks without retraining the whole model

Prefix Tuning and P-Tuning

  • Prefix Tuning: Instead of learning prompts at the input, prefix tuning learns continuous vectors that are prepended to the hidden states at every layer of the Transformer. This gives more control and flexibility.
  • P-Tuning: Similar to prompt tuning, but often uses a small LSTM or MLP to generate the soft prompts, allowing for more expressive prompts.

Key Terminology

  • Soft Prompt: A set of learnable continuous vectors, not actual text, prepended to the input or hidden states
  • Hard Prompt: A prompt made of actual words/tokens
  • Frozen Model: The base LLM whose weights are not updated during training

4. Detailed Explanation and Examples (25-30 minutes)

Understanding Soft Prompt Methods

Prompt Tuning

  • Only a small set of prompt embeddings are trained
  • The rest of the model is frozen
  • At inference, the learned prompt is prepended to the input

Prefix Tuning

  • Learns prefix vectors for each layer’s key/value in the attention mechanism
  • More expressive, but slightly more complex

P-Tuning

  • Uses a small neural network to generate the soft prompt
  • Can capture more complex task-specific information

Practical Applications

  • Multi-task learning: One model, many tasks, each with its own soft prompt
  • Fast adaptation: Deploy new capabilities quickly
  • Storage efficiency: Only need to store small prompt vectors per task

Common Misconceptions

  1. Misconception: Soft prompts are just special text prompts Reality: They are learned vectors, not actual words
  2. Misconception: You need to update the whole model for new tasks Reality: Only the soft prompts are trained

5. Live Coding Demonstration (20-25 minutes)

Example 1: Basic Prompt Tuning with Hugging Face PEFT

from transformers import AutoModelForSequenceClassification, AutoTokenizer
from peft import PromptTuningConfig, get_peft_model, TaskType

model_name = 'bert-base-uncased'
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=2)

prompt_config = PromptTuningConfig(
    task_type=TaskType.SEQ_CLS,
    num_virtual_tokens=8,  # Number of soft prompt tokens
    tokenizer_name_or_path=model_name
)

prompt_model = get_peft_model(model, prompt_config)
prompt_model.print_trainable_parameters()

Explanation:

  • Only the soft prompt embeddings are trainable
  • The base model is frozen

Example 2: Prefix Tuning

from peft import PrefixTuningConfig

prefix_config = PrefixTuningConfig(
    task_type=TaskType.SEQ_CLS,
    num_virtual_tokens=8,
    tokenizer_name_or_path=model_name
)
prefix_model = get_peft_model(model, prefix_config)
prefix_model.print_trainable_parameters()

Key Points:

  • Prefix tuning is more expressive than prompt tuning
  • Slightly more parameters, but still much less than full fine-tuning

Example 3: P-Tuning (v2)

from peft import PromptEncoderConfig

ptuning_config = PromptEncoderConfig(
    task_type=TaskType.SEQ_CLS,
    num_virtual_tokens=8,
    encoder_hidden_size=128,
    tokenizer_name_or_path=model_name
)
ptuning_model = get_peft_model(model, ptuning_config)
ptuning_model.print_trainable_parameters()

Debugging Demonstration

Show a common error: using too few virtual tokens

# Common mistake: too few virtual tokens for a complex task
prompt_config = PromptTuningConfig(
    task_type=TaskType.SEQ_CLS,
    num_virtual_tokens=1,  # Too small!
    tokenizer_name_or_path=model_name
)
# Solution: Increase num_virtual_tokens (e.g., 8 or 16)

6. Hands-On Practice (30-40 minutes)

Exercise 1: Basic Practice (10 minutes)

Objective: Apply prompt tuning to a pre-trained model for a simple classification task

Instructions:

  1. Load a pre-trained BERT or DistilBERT model
  2. Configure prompt tuning with 8 virtual tokens
  3. Print the number of trainable parameters

Starter Code:

from transformers import AutoModelForSequenceClassification, AutoTokenizer
from peft import PromptTuningConfig, get_peft_model, TaskType

model_name = 'distilbert-base-uncased'
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=2)

# Your prompt tuning config and wrapping code here

Solution:

prompt_config = PromptTuningConfig(
    task_type=TaskType.SEQ_CLS,
    num_virtual_tokens=8,
    tokenizer_name_or_path=model_name
)
prompt_model = get_peft_model(model, prompt_config)
prompt_model.print_trainable_parameters()

Exercise 2: Intermediate Challenge (15 minutes)

Objective: Fine-tune a prompt-tuned model on a small dataset

Requirements:

  • Prepare a small dataset (e.g., 10 sentences, binary labels)
  • Tokenize and create a DataLoader
  • Train for 1 epoch and evaluate accuracy

Hints:

  • Use PyTorch DataLoader
  • Use Trainer from Hugging Face

Exercise 3: Creative Application (15 minutes)

Objective: Compare prompt tuning, prefix tuning, and LoRA on the same task

Challenge:

  • Implement all three methods on a classification model
  • Compare number of trainable parameters, training time, and accuracy

Extension Ideas:

  • Try different numbers of virtual tokens
  • Apply to a generative model (e.g., GPT-2)

7. Common Pitfalls and Troubleshooting (10 minutes)

Frequent Mistakes

  1. Problem: Too few virtual tokens Cause: Not enough capacity for the task Solution: Increase num_virtual_tokens
  2. Problem: Forgetting to freeze the base model Cause: Accidentally updating all parameters Solution: Use PEFT wrappers to ensure only soft prompts are trainable

Debugging Tips

  • Use print_trainable_parameters() to verify
  • Start with 8-16 virtual tokens and adjust as needed
  • Monitor validation accuracy for underfitting/overfitting

8. Best Practices and Professional Tips (5 minutes)

Industry Standards

  • Use soft prompt methods for rapid, low-resource adaptation
  • Store and manage prompt vectors for each task
  • Document prompt configurations for reproducibility

Next Steps in Your Learning Journey

  • Explore combining soft prompts with LoRA/Adapters
  • Try prompt tuning for generative tasks
  • Read the original prompt tuning, prefix tuning, and P-tuning papers

P-Tuning Terminology

Classic P-tuning learns continuous prompt embeddings and uses a prompt encoder, originally emphasizing natural-language-understanding settings. P-tuning v2 is a deep-prompt method that injects learned prompts across Transformer layers and is closer to prefix-style tuning. Keep these variants distinct when comparing methods. A virtual-token count is only a starting hyperparameter; tune it against validation quality, memory, and latency rather than treating one count as generally optimal.

9. Summary and Review (5 minutes)

Key Takeaways

  1. Soft prompt methods enable efficient adaptation of LLMs with minimal parameters
  2. Prompt tuning, prefix tuning, and P-tuning differ in where and how prompts are injected
  3. Hugging Face PEFT makes these methods easy to implement

Quick Review Questions

  1. What is a soft prompt?
  2. How does prefix tuning differ from prompt tuning?
  3. When might you prefer prompt tuning over LoRA?

For Instructors

Teaching Tips

  • Emphasize the difference between soft and hard prompts
  • Encourage experimentation with number of virtual tokens
  • Address confusion about where prompts are injected

Assessment Ideas

  • Quiz: Identify which method is used in a code snippet
  • Project: Fine-tune a model for a new task using prompt tuning
  • Peer review: Compare results and discuss trade-offs

Adaptation Notes

  • For beginners: Focus on prompt tuning basics
  • For advanced: Dive into prefix tuning and P-tuning details
  • Online: Provide Colab notebooks; In-person: Live coding demos