Building Retrieval-Augmented Generation (RAG) From Scratch with LangChain

langchain-ai/rag-from-scratch

This repository is essentially a tutorial that guides you through building a Retrieval-Augmented Generation (RAG) application from the ground up, using LangChain. RAG is a powerful technique for making Large Language Models (LLMs) more accurate and knowledgeable by connecting them to external, up-to-date, or proprietary data sources.

For a software engineer, RAG solves one of the biggest challenges when building with LLMs
grounding the model in specific, relevant information (like company documentation, internal codebases, or recent project notes).

The rag-from-scratch repository walks you through the core components of a RAG pipeline. Here is the typical setup process you would follow to integrate RAG into your application

You need to load your data (e.g., PDF files, Markdown, internal APIs).

Task
Read your proprietary files.

LangChain Tool
Document Loaders (e.g., PyPDFLoader, DirectoryLoader).

LLMs have a context window limit, and documents are often too large. You must split them into smaller, manageable chunks.

Task
Break a large document into pieces of text (e.g., 500 characters each).

LangChain Tool
Text Splitters (e.g., RecursiveCharacterTextSplitter).

Each text chunk is converted into a numerical vector (an embedding) that captures its meaning. These vectors are stored in a Vector Store.

Task
Create vectors and save them for efficient lookup.

LangChain Tools
Embedding Models (e.g., OpenAIEmbeddings, HuggingFaceEmbeddings) and Vector Stores (e.g., FAISS, Chroma, Pinecone).

When a user asks a question, the query is first embedded. The Vector Store is searched for the most semantically similar text chunks. These chunks are then packaged with the user's question and sent to the LLM to generate the final answer.

Task
Connect the pieces
Vector Store → LLM → Final Answer.

LangChain Tools
Retrievers and Chains/LCEL (LangChain Expression Language).

This example illustrates the core components from the rag-from-scratch concept, using popular tools.

# 1. Setup Environment and Imports
from langchain_community.document_loaders import TextLoader
from langchain_community.vectorstores import FAISS
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain_core.prompts import ChatPromptTemplate
from langchain.chains import create_retrieval_chain

# --- 1. Data Loading ---
# Load your company documentation file
loader = TextLoader("./company_docs.txt")
docs = loader.load()

# --- 2. Chunking ---
# Split the document into small chunks
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
chunks = text_splitter.split_documents(docs)

# --- 3. Embedding and Storage (The "Vector Store") ---
# Use OpenAI to create embeddings and store them in FAISS
embeddings = OpenAIEmbeddings()
vector_store = FAISS.from_documents(chunks, embeddings)

# --- 4. Retrieval and Generation ---

# A. Define the LLM and the Prompt
llm = ChatOpenAI(model="gpt-4o-mini")

# System prompt is critical for grounding the model
prompt = ChatPromptTemplate.from_template("""
Answer the user's question based *only* on the provided context.
If you cannot find the answer, state that you don't know.

Context: {context}
Question: {input}
""")

# B. Create the Document Chain (sends context to LLM)
document_chain = create_stuff_documents_chain(llm, prompt)

# C. Create the Retriever and Final Chain (gets chunks from vector store)
retriever = vector_store.as_retriever()
retrieval_chain = create_retrieval_chain(retriever, document_chain)

# --- Run the RAG Query ---
response = retrieval_chain.invoke({"input": "What is the policy for remote work?"})

print(response["answer"])
# print("--- Source Documents Used ---")
# print(response["context"]) # Shows the retrieved chunks

By working through the langchain-ai/rag-from-scratch tutorial, you'll gain the foundational understanding to implement a robust, production-ready RAG system for any internal or external application.