LEANN: The Software Engineer's Secret Weapon for Private and Portable RAG

yichuan-w/LEANN

LEANN is an innovative, open-source vector database designed for the modern, privacy-focused RAG stack. Its key value propositions are

Massive Storage Savings
LEANN can reduce the required storage for your vector index by up to 97% compared to traditional vector databases.

100% Privacy
Your data stays local on your device (e.g., your laptop). There's no need for cloud services, external APIs (like OpenAI), or sending your private documents off-device.

Speed and Accuracy
It's designed to be fast and maintains high search accuracy, even with its highly compressed storage.

As a software engineer, LEANN provides a powerful tool to build applications that were previously resource-prohibitive or violated privacy requirements

Building Personal AI Assistants
You can create powerful RAG systems that index a user's entire local data (file system, emails, chat history, browser history, etc.) and run semantic search/Q&A, all on a standard laptop. This unlocks truly personal and private AI copilots.

Cost Efficiency
By eliminating the need for vast cloud storage for pre-computed embeddings and reducing cloud-based vector DB costs, LEANN significantly lowers the operational costs of your RAG application.

Portability
The entire knowledge base is incredibly small and portable. You can easily transfer a user's "AI memory" between devices.

Handling Massive Local Data
It makes it feasible to RAG over millions of local documents without running out of disk space or needing a massive server infrastructure.

Traditional vector databases pre-compute and store the high-dimensional vector embeddings for every single document in your dataset. These embeddings take up a lot of space.

LEANN takes a different approach by using Graph-based Selective Recomputation.

It stores a highly compressed, pruned graph structure (using formats like Compressed Sparse Row or CSR) that represents the relationships between your documents.

It computes embeddings on-demand only when they are needed for a search query, rather than storing them all.

This allows it to maintain the necessary information for fast, accurate retrieval while drastically cutting down on storage overhead.

LEANN is a Python library and is designed to be easy to install and integrate into existing RAG workflows.

It's recommended to use uv for a faster installation, but pip also works.

# Recommended: Install uv first (if you don't have it)
pip install uv

# Install LEANN from PyPI
uv pip install leann

The workflow typically involves three main classes
LeannBuilder (to create the index), LeannSearcher (to perform retrieval), and LeannChat (for a full RAG pipeline).

Here is a basic example for indexing and searching

from leann import LeannBuilder, LeannSearcher
import os
import shutil

# --- Setup: Define paths and initial documents ---
INDEX_PATH = "./my_leann_index"

# Clean up previous index for a fresh start
if os.path.exists(INDEX_PATH):
    shutil.rmtree(INDEX_PATH)

# A simple list of documents (your real data could be files, emails, etc.)
documents = [
    "The 2024 operating budget focuses heavily on new cloud infrastructure.",
    "Python is the primary language for AI development and is known for its readability.",
    "A key feature of LEANN is its on-demand embedding recomputation for storage savings.",
    "The project deadline is set for the end of the third fiscal quarter."
]

print("Starting LEANN Index Building...")

# --- Step 1: Build the Index ---
# LeannBuilder takes your documents and creates the compressed, graph-based index.
builder = LeannBuilder(
    index_path=INDEX_PATH,
    model_name="all-MiniLM-L6-v2" # Specify the embedding model
)

# Add all documents to the index
for i, doc in enumerate(documents):
    builder.add(id=str(i), text=doc)

# Finalize the building process (creates the on-disk index)
builder.save()
print(f"Index built successfully at {INDEX_PATH}")

# --- Step 2: Search the Index ---
# LeannSearcher loads the compact index from disk.
searcher = LeannSearcher(index_path=INDEX_PATH)

query = "What is the main benefit of LEANN's technology?"

print(f"\nSearching for: '{query}'")

# Perform the semantic search
# k=2 means retrieve the top 2 most relevant documents
results = searcher.search(query, k=2)

# Display the results
for rank, result in enumerate(results):
    # result.id and result.text give you the retrieved document's ID and original content
    print(f"--- Rank {rank + 1} ---")
    print(f"Document ID: {result.id}")
    print(f"Content: {result.text}")

# Expected Output (content will vary based on model):
# --- Rank 1 ---
# Document ID: 2
# Content: A key feature of LEANN is its on-demand embedding recomputation for storage savings.