Claim◆Article
Adding an LLM to an existing stack comes down to one question: how deeply should the LLM reach into the runtime?
The central design question for integrating LLMs into production systems is the depth of runtime integration, and the infrastructure already in place determines each company's answer. ✦ AI generated
The article author · ByteByteGo Newsletter · 2026-07-28 · original ↗
Adding an LLM to an existing stack comes down to one question: how deeply should the LLM reach into the runtime? Ultimately, DoorDash, Instacart, and Uber Eats each answered that question differently, and the specific LLM each chose was secondary. The specific model each company picked was secondary. The infrastructure they already had in place is what determined the answer.
Read full article ↗excerpt · fair-use quotation
- ·How deeply should the LLM reach into the runtime?
- ·The specific LLM model chosen is secondary
- ·DoorDash, Instacart, and Uber Eats all answered differently
- ·Existing infrastructure dictated each company's approach
- ·The LLM itself was a secondary concern
Around this claim
Context · 3
DoorDash's LLM lives mostly offline in batch, working on the periphery of the runtime — it enriches the knowledge graph and parses queries into chunks with constrained outputs linked back to graph fields.The article author · ByteByteGo Newsletter · conf 95%Instacart uses LLMs at the query understanding layer, with offline RAG for head queries and a real-time fine-tuned Llama-3-8B for tail queries, while retrieval downstream is still traditional.The article author · ByteByteGo Newsletter · conf 95%Uber Eats fine-tuned a Qwen LLM into the embedding backbone of two-tower retrieval, so every query and every document gets an LLM-derived vector.The article author · ByteByteGo Newsletter · conf 95%