Introducing LEANN: a lightweight vector index for efficient rag systems

Researchers from UC Berkeley’s Sky Computing Lab have developed LEANN, an open-source vector index that significantly reduces storage requirements for retrieval-augmented generation (RAG) systems. The index, built on FAISS, eliminates the need to store embeddings by recomputing them at query time. Benchmarks show a 60 million chunk Wikipedia index shrinking from 201 GB to 6 GB. The paper, which won the Best Research Paper award at MLSys 2026, enables semantic search and RAG over personal data on laptops. LEANN uses a pruned graph and compact product quantization table, rebuilding embeddings on demand. It achieves a 1.8x speedup through batching and two-level search, though embedding recomputation still accounts for 76% of query latency. On NQ, LEANN matches HNSW in accuracy while using 4 GB versus 188 GB of storage. It also outperforms BM25 in exact match by up to 11.8%. The tool supports multiple applications, including document search, Apple Mail, and ChatGPT exports. It runs on various platforms and integrates with Ollama, Hugging Face, and OpenAI-compatible APIs. The GitHub repo and PyPI package are available for further exploration.