DefinitionArticle
Standard RAG works by chunking documents, embedding them into vectors, and retrieving chunks whose vectors are closest to the query vector, relying on the assumption that the text answering a question will be similar to the question itself.
Standard RAG retrieves information by treating document chunks as independent units and finding those most similar to a query, which works well when the answer resembles the question. ✦ AI generated
article author · ByteByteGo Newsletter · 2026-08-19 · original ↗
We take a collection of documents, slice each one into chunks of a few hundred to a few thousand tokens, and pass every chunk through an embedding model. The embedding model returns a vector... The whole design rests on one simple assumption, which is that text answering a question would resemble that question.
Read full article ↗excerpt · fair-use quotation
Around this claim
This moment responds to
provides context → GraphRAG provides two distinct search paths: local search expands from matched entities through related text units, reports, and neighbors, while global search aggregates pre-written community reports using a map-reduce process.article author · ByteByteGo Newsletterprovides context → GraphRAG indexing builds a community hierarchy over the entity graph using Leiden clustering and pre-generates a summary report for every community at every level, making the entire collection's content answerable at query time.article author · ByteByteGo Newsletterprovides context → Standard RAG's similarity-based assumption fails for 'global queries' that require reasoning across a large collection of documents, because the relevant information is distributed, not located in a few text regions that resemble the query.article author · ByteByteGo Newsletterprovides context → Naively chunking documents by a fixed number of characters loses important context, whereas semantically chunking legal documents along their natural structural boundaries (sections, subsections, bullets) while retaining hierarchy preserves accuracy that a naive approach would leave on the table.Alex Boucott · The TWIML AI Podcastprovides context → A knowledge graph stores entities (nouns) and typed relationships between them, each with a description, which preserves the connections between concepts that plain text chunking discards.article author · ByteByteGo Newsletterprovides context → GraphRAG has higher indexing cost (due to two LLM passes for extraction and merging) than standard RAG, but Microsoft's LazyGraphRAG variant reduces this cost to 0.1% by using NLP for indexing and deferring LLM work to query time.article author · ByteByteGo Newsletterprovides context → For accuracy-critical, citation-sensitive domains like tax law, agents grepping over a file system aren't yet accurate enough to replace a purpose-built retrieval system — RAG isn't dead for this use case, even though it may be for others.Alex Boucott · The TWIML AI Podcast