Article · 2026-08-19 · 6 moments

GraphRAG: How AI Answers Questions Hidden Across Many Documents

GraphRAG was designed to handle the second kind of questions, and we are going to learn more about it in this article. ✦ AI generated

Read original ↗

GUEST article author

Definition

A knowledge graph stores entities (nouns) and typed relationships between them, each with a description, which preserves the connections between concepts that plain text chunking discards.

Knowledge graphs capture the entities and relationships within a corpus, creating a connected structure that can reveal paths and patterns not explicit in any single document.

transcript

article author: A knowledge graph stores two kinds of things: Entities are the nouns a corpus talks about, such as people, services, teams, incidents, and decisions. Relationships are the typed connections between those entities. Both carry a plain-text description.

provides context · 1

Data

GraphRAG has higher indexing cost (due to two LLM passes for extraction and merging) than standard RAG, but Microsoft's LazyGraphRAG variant reduces this cost to 0.1% by using NLP for indexing and deferring LLM work to query time.

GraphRAG's main cost is upfront indexing, which involves multiple LLM passes. A variant called LazyGraphRAG dramatically cuts this cost by simplifying indexing and doing more work at query time.

transcript

article author: GraphRAG indexing runs two language model passes over the corpus... Graph extraction accounts for roughly 75 percent of indexing cost... LazyGraphRAG builds its index using NLP rather than a language model, skips summarization entirely, and defers all language model work to query time. In this case, indexing cost matches vector RAG and lands at 0.1 percent of full GraphRAG.

provides context · 1

Mechanism◆

GraphRAG indexing builds a community hierarchy over the entity graph using Leiden clustering and pre-generates a summary report for every community at every level, making the entire collection's content answerable at query time.

GraphRAG's indexing pipeline clusters entities into hierarchical communities and generates a summary report for each, which pre-computes the answers to questions that span the entire corpus.

transcript

article author: For every community at every level, a language model generates a community report. Each report contains an overview of that community along with its key entities, relationships, and claims... This is the step that makes whole-collection questions answerable. A summary of what a cluster of documents collectively says gets written during indexing, well before anyone asks about it.

provides context · 3

Mechanism

GraphRAG provides two distinct search paths: local search expands from matched entities through related text units, reports, and neighbors, while global search aggregates pre-written community reports using a map-reduce process.

GraphRAG offers different search strategies: local search for entity-focused questions and global search that synthesizes pre-written summaries to answer questions spanning the entire document collection.

transcript

article author: Local search begins by matching the query against entity description embeddings... Global search leaves the entity graph untouched. Community reports from a chosen hierarchy level are split into batches... a map stage runs each batch through a language model and produces an intermediate answer... a reduce stage then collects the highest-rated points across all batches and generates the final answer.

provides context · 3

Claim◆

Standard RAG's similarity-based assumption fails for 'global queries' that require reasoning across a large collection of documents, because the relevant information is distributed, not located in a few text regions that resemble the query.

Standard RAG cannot effectively answer 'global queries' (like 'what failure causes recur most often') because the answer exists as a pattern across many documents, not in chunks similar to the query.

transcript

article author: A global query requires reasoning across large portions of a dataset, or across all of it... The real answer to the question exists across two hundred documents as a distribution, which spans the corpus rather than occupying one retrievable location.

provides context · 1

Definition

Standard RAG works by chunking documents, embedding them into vectors, and retrieving chunks whose vectors are closest to the query vector, relying on the assumption that the text answering a question will be similar to the question itself.

Standard RAG retrieves information by treating document chunks as independent units and finding those most similar to a query, which works well when the answer resembles the question.

transcript

article author: We take a collection of documents, slice each one into chunks of a few hundred to a few thousand tokens, and pass every chunk through an embedding model. The embedding model returns a vector... The whole design rests on one simple assumption, which is that text answering a question would resemble that question.

Highlight slides

GraphRAG Community Reports✦ from: GraphRAG indexing builds a community hierarchy over the entity graph using Leiden clustering and pre-generates a summary report for every community at every level, making the entire collection's content answerable at query time.Standard RAG Fails on Global Queries✦ from: Standard RAG's similarity-based assumption fails for 'global queries' that require reasoning across a large collection of documents, because the relevant information is distributed, not located in a few text regions that resemble the query.Pre-Computed at Index Time✦ from: GraphRAG indexing builds a community hierarchy over the entity graph using Leiden clustering and pre-generates a summary report for every community at every level, making the entire collection's content answerable at query time.Example: Recurring Failure Causes✦ from: Standard RAG's similarity-based assumption fails for 'global queries' that require reasoning across a large collection of documents, because the relevant information is distributed, not located in a few text regions that resemble the query.Why It Works✦ from: GraphRAG indexing builds a community hierarchy over the entity graph using Leiden clustering and pre-generates a summary report for every community at every level, making the entire collection's content answerable at query time.

Related episodes

Video · Lenny's Podcast

The rise of taste, human authenticity and judgment in an AI world | Adam Mosseri (Head of IG)

→

Video · The TWIML AI Podcast

Teaching Computers to Smell | Alex Wiltschko

→

Video · Syntax

I Built an LLM From Scratch

→