Master RAG for million-line repos with vector indexing and graph-based retrieval. Learn the 2026 architecture that scales AI context. Full breakdown.
Why flat retrieval breaks on million-line repos
Retrieval-augmented generation works when the right chunks reach the model before generation starts. On a small library that is easy. On a million-line monorepo it is not: a pure similarity search over file fragments often returns near-duplicates of the same utility layer while missing the call path that actually explains the behavior you care about. Code is not prose. Names collide across packages, APIs evolve in place, and the answer to a question often lives in the relationship between modules rather than in any single file.
A 2026-scale RAG stack therefore treats the repository as two complementary indexes: a vector store for semantic neighborhood, and a graph of structural edges for dependency, ownership, and control flow. Vector hits propose candidates. Graph expansion decides which neighbors must travel with them so the model sees a coherent slice of the system instead of a bag of similar strings.
Vector indexing that respects code structure
Chunking strategy dominates quality. Prefer units that match how engineers reason—functions, classes, modules, or interface blocks—over fixed token windows that split mid-signature. Attach metadata that retrieval can filter on: language, package path, test vs production, public API vs internal helper. Embeddings should be built from content that includes signatures and nearby comments, not only method bodies, so queries about “how clients authenticate” can match the contract, not only the implementation details buried three layers down.
Index maintenance is part of the architecture. Incremental updates on changed files keep the store honest without full rebuilds on every commit. When a file is deleted or renamed, remove or rekey its vectors so stale paths do not keep ranking. Deduplicate near-identical generated or vendored code at index time; otherwise similarity search wastes slots on noise that never belongs in the prompt.
Graph-based retrieval for multi-hop context
Build a lightweight graph from what static analysis already knows: imports, call edges, inheritance, interface implementations, and package boundaries. On a query, run vector search first, then walk a bounded number of hops from each hit—callers of a hot function, implementations of an interface, tests that import the module. Cap expansion by hop depth and by token budget so the graph cannot flood the context window.
- Use the graph to enforce locality: prefer edges inside the same package or service boundary before crossing the monorepo.
- Promote tests and type definitions when the query is about behavior or contracts; demote pure config and lockfiles unless the query names them.
- Rank final snippets jointly on embedding score, graph distance, and recency of change so dead code loses to live paths.
Assembly, evaluation, and operational guardrails
After ranking, assemble context in a stable order: entry points and public types first, then implementations, then tests. Compress with care—summaries help for distant modules, but keep exact signatures and error paths for the symbols the query names. Guardrails matter in large orgs: scope retrieval by team ownership or path allowlists so a prompt about one service cannot pull secrets or irrelevant systems from another. Log which chunks were selected and why; that audit trail is how you debug bad answers without guessing.
Measure success the way you measure search quality, not only generation flair: recall of known-relevant files on held-out developer questions, token efficiency of assembled context, and whether answers cite the right modules. Iterate on chunk boundaries and graph hop rules when those metrics stall. Vector indexing finds candidates; graph-based retrieval makes them useful. Together they form the architecture that keeps AI context scalable as the codebase grows past what any single prompt can hold.