LLM chevron_right Building a Basic End-to-End RAG System

Building a Basic End-to-End RAG System

calendar_today schedule 6 min read

Learning Objectives: By the end of this lesson, you will be able to:

  • Integrate retriever and generator components into a functional RAG pipeline
  • Automate document ingestion, chunking, embedding, indexing, retrieval, and generation
  • Build and test a Q&A system over your own documents

1. Welcome and Introduction (5 minutes)

Icebreaker Activity

Have you ever wished you could instantly search and ask questions about your own notes or research papers? What would you use such a system for?

Lesson Overview

  • Duration: Approximately 2.5 hours including all exercises (or 60–90 minutes when selected activities are assigned)
  • Prerequisites: Familiarity with RAG components, Python, and basic NLP
  • Materials Needed: Python environment, Hugging Face Transformers, FAISS or ChromaDB, sample documents (e.g., PDFs, text files)

2. Learning Objectives and Session Outline (5 minutes)

What We’ll Cover Today:

  1. Orchestration: Combining retriever and generator
  2. Data ingestion: Loading, chunking, embedding, and indexing
  3. Building the RAG pipeline: Retrieval and generation
  4. Hands-on: Building and testing a Q&A system
  5. Common pitfalls and best practices

Success Criteria:

By the end of this session, you should be able to build a simple RAG system and answer questions using your own documents.

3. Foundation Knowledge (15-20 minutes)

Core Concepts

What is an End-to-End RAG System?

A RAG system combines a retriever (for finding relevant information) and a generator (for producing answers) into a single pipeline. The system can answer questions using up-to-date or private data, not just what was in the LLM’s training set.

Real-World Analogy: Imagine a librarian (retriever) who finds the right books and a subject expert (generator) who reads them and answers your question.

Why This Matters:

  • Enables dynamic, updatable knowledge for LLMs
  • Supports Q&A, search, and summarization over custom data
  • Powers modern AI assistants and enterprise search tools

Orchestration Tools

  • Libraries like LangChain, LlamaIndex, Haystack simplify RAG pipeline construction
  • You can also build from scratch for deeper understanding

Key Terminology

  • Orchestration: Coordinating multiple components in a pipeline
  • Data Ingestion: Loading and preparing documents for retrieval
  • Pipeline: Sequence of steps from query to answer

4. Detailed Explanation and Examples (25-30 minutes)

Data Ingestion and Preparation

  • Load documents (text, PDF, etc.)
  • Chunk into passages (e.g., 256-512 tokens)
  • Generate embeddings for each chunk
  • Index embeddings in a vector database

Building the RAG Pipeline

  • User submits a query
  • Embed the query and retrieve top-k relevant chunks
  • Construct a prompt with retrieved context and the query
  • Pass prompt to LLM generator for answer

Practical Applications

  • Q&A over research papers, company wikis, or personal notes
  • Customer support bots with up-to-date knowledge
  • Legal, medical, or academic document search

Common Misconceptions

  1. Misconception: RAG is only for large companies Reality: Anyone can build a basic RAG system with open-source tools
  2. Misconception: You need a huge dataset Reality: RAG works well even with small, focused corpora

5. Live Coding Demonstration (20-25 minutes)

Example 1: Data Ingestion and Indexing

from sentence_transformers import SentenceTransformer
import faiss
import numpy as np

# Load and chunk documents (simple example)
docs = [open('doc1.txt').read(), open('doc2.txt').read()]
chunks = [
    {"source_id": f"doc-{doc_i}:chunk-{chunk_i}", "text": para}
    for doc_i, doc in enumerate(docs)
    for chunk_i, para in enumerate(doc.split('\n\n')) if para.strip()
]

# Embed chunks
model = SentenceTransformer('all-MiniLM-L6-v2')
embeddings = model.encode([chunk["text"] for chunk in chunks], normalize_embeddings=True)

# Index with FAISS
index = faiss.IndexFlatIP(embeddings.shape[1])
index.add(np.array(embeddings))

Example 2: Retrieval and Generation

from transformers import pipeline

query = "What is the main finding in doc1?"
q_emb = model.encode([query], normalize_embeddings=True)
D, I = index.search(np.array(q_emb), k=3)
retrieved = [chunks[i] for i in I[0]]

# Build prompt
context = "\n".join(f"[{item['source_id']}] {item['text']}" for item in retrieved)
prompt = f"Answer the question based on the following context:\n{context}\n\nQuestion: {query}\nAnswer:"

# Generate answer
llm = pipeline('text2text-generation', model='google/flan-t5-base')
answer = llm(prompt, max_new_tokens=100)[0]['generated_text']
print(answer)

Example 3: Framework Integration Boundary

# Pseudocode: keep application logic behind a small, testable interface.
# If using LangChain, pin the exact LangChain and integration-package versions
# in your lockfile and implement this interface with those versions' v1 docs.
retrieved_records = retriever.search(query, k=3)
prompt = prompt_builder.format(query=query, sources=retrieved_records)
result = generator.invoke(prompt)
assert citations_exist(result, retrieved_records)

Debugging Demonstration

Show a common error: missing or poorly chunked data

# Problem: Some documents are not chunked, so relevant info is missed
# Solution: Ensure all documents are properly chunked and indexed

6. Hands-On Practice (30-40 minutes)

Exercise 1: Basic Practice (10 minutes)

Objective: Ingest and index your own documents

Instructions:

  1. Load 2-3 text files from your computer
  2. Split into paragraphs or fixed-size chunks
  3. Generate embeddings and index with FAISS

Starter Code:

from sentence_transformers import SentenceTransformer
import faiss
import numpy as np

# Your document loading and chunking code here

Solution:

docs = [open('doc1.txt').read(), open('doc2.txt').read()]
chunks = [para for doc in docs for para in doc.split('\n\n') if para.strip()]
model = SentenceTransformer('all-MiniLM-L6-v2')
embeddings = model.encode(chunks, normalize_embeddings=True)
index = faiss.IndexFlatIP(embeddings.shape[1])
index.add(np.array(embeddings))

Exercise 2: Intermediate Challenge (15 minutes)

Objective: Build a retrieval and generation pipeline

Requirements:

  • Accept a user query
  • Retrieve top-3 relevant chunks
  • Construct a prompt and generate an answer

Hints:

  • Use Hugging Face pipeline for generation
  • Print both retrieved context and answer

Exercise 3: Creative Application (15 minutes)

Objective: Extend your RAG system

Challenge:

  • Add support for PDF or web page ingestion (e.g., use PyPDF2 or BeautifulSoup)
  • Try a different LLM (e.g., Llama, Falcon)
  • Add source citation in generated answers

Extension Ideas:

  • Build a simple web interface for your RAG system
  • Experiment with chunk sizes and retrieval strategies

7. Common Pitfalls and Troubleshooting (10 minutes)

Frequent Mistakes

  1. Problem: Poor retrieval due to bad chunking or missing data Cause: Inconsistent or overly large/small chunks Solution: Standardize chunk sizes and check all docs are indexed
  2. Problem: LLM ignores context Cause: Weak prompt engineering Solution: Use explicit, clear prompt templates

Debugging Tips

  • Print retrieved context for each query
  • Test with known questions and answers
  • Monitor LLM output for hallucinations

8. Best Practices and Professional Tips (5 minutes)

Industry Standards

  • Automate data ingestion and indexing for scalability
  • Log queries, retrieved docs, and answers for analysis
  • Regularly evaluate retrieval and generation quality

Next Steps in Your Learning Journey

  • Explore advanced orchestration with LangChain or LlamaIndex
  • Add re-ranking, summarization, or multi-hop retrieval
  • Read about production RAG deployments

Production Notes

Use an instruction-following model designed for question answering rather than GPT-2 for a realistic generator, and carry stable source metadata (source_id, title, location, and chunk ID) from ingestion through retrieval and citation validation. Framework APIs change quickly: the LangChain snippet in this lesson is conceptual unless dependencies are pinned. For LangChain v1 projects, follow the installed version’s current retrieval-chain APIs and integration-package imports rather than copying legacy langchain.vectorstores examples.

9. Summary and Review (5 minutes)

Key Takeaways

  1. End-to-end RAG systems combine retrieval and generation for dynamic Q&A
  2. Data ingestion, chunking, and indexing are critical for good performance
  3. Prompt engineering and pipeline orchestration enable practical applications

Quick Review Questions

  1. What are the main steps in building a RAG system?
  2. Why is chunking and indexing important?
  3. How can you improve retrieval and generation quality?

For Instructors

Teaching Tips

  • Walk through the pipeline step by step
  • Encourage students to use their own documents for practice
  • Address confusion about orchestration and data ingestion

Assessment Ideas

  • Quiz: Identify pipeline steps in code
  • Project: Build a RAG system for a specific domain
  • Peer review: Evaluate retrieval and answer quality

Adaptation Notes

  • For beginners: Focus on text files and basic pipeline
  • For advanced: Add PDF/web ingestion, re-ranking, or web interface
  • Online: Provide Colab notebooks; In-person: Live coding demos