For accuracy-critical, citation-sensitive domains like tax law, agents grepping over a file system aren't yet accurate enough to replace a purpose-built retrieval system — RAG isn't dead for this use case, even though it may be for others.
Alex Boucott argues that while 'RAG is dead' may hold for some use cases, Sphere's need for exact, verifiable citations means agent-based grepping over documents still can't match a dedicated retrieval pipeline's accuracy. ✦ AI generated
Alex Boucott · The TWIML AI Podcast · 2026-06-09 · original ↗
starts at this moment · 28:39
“What's your take on the whole rag is dead argument that some folks make?”
I think for us and or at least for this particular problem, because we are so sensitive to accuracy and we're so sensitive to the exact right citation... as of today, I don't think, you know, agents are just searching over the file system, grepping over it is at a point where we could switch over and and not lose accuracy.
verbatim transcript · starts at 28:39
28:39Um and I think we could set up some sort of system where, you know, we just have all this legislation in a file system and then an agent can grep over it and find the relevant pieces that way. And I think yeah, for for some sorts of problems that works well. I think for us and or at least for this particular problem, because we are so sensitive to
29:01accuracy and we're so sensitive to the exact right citation. Essentially, we need like a more finely finely tuned scalpel to like find us the the relevant portion and we need it to be highly accurate. And so like anecdotally at least when I use Claude code or something and I see it grepping through the code base, there's lots of times it misses. Like I'll go off and I'll find a file that I
29:29really wish it would have found. Like this file had the answer I was looking for. And so, you know, maybe I think maybe we're on a path you know, 5 years from now our rag system still working the way they are today. I'm sure they won't be. But as of today, I don't think you know, agents are just searching over the file system grepping over it is at a
29:51point where we could switch over and and not lose accuracy. >> Talk a little bit about the citations that you mentioned, how you use those and how the retrieval system helps you deliver them. >> As part of the ingestion process, we carry through a hierarchy of these different passages of text that we end up indexing and each of them carries a citation. And and different passages might share the same
30:20citation, but that's very important for us eventually upstream when the tax expert goes to review because those citations also have links which will link the tax expert out to the source document where we collected this because a lot of times, you know, they want to review that the tax expert that is, they want to review, you know, a bit more context than maybe the the model gave them in its in its, you know,
- ·Sphere's domain is highly accuracy-sensitive
- ·Exact, verifiable citations are non-negotiable
- ·Purpose-built retrieval still outperforms agents here
- ·Agents grepping the file system aren't accurate enough yet
- ·Switching now risks losing citation accuracy
- ·'RAG is dead' may hold elsewhere, not here