DoorDash's LLM lives mostly offline in batch, working on the periphery of the runtime — it enriches the knowledge graph and parses queries into chunks with constrained outputs linked back to graph fields.

DoorDash already had a knowledge graph for items and restaurants when LLMs became viable for production. The graph held structured attributes for every item, including dish type, dietary preference, cuisine, brand, and flavor. Their approach was to use LLMs to enrich this graph offline by extracting attributes from SKU data, and to use LLMs at runtime only for parsing queries into chunks that could link back to the graph. Retrieval itself stayed keyword and graph-driven... What distinguishes DoorDash's approach is how they constrain the LLM's outputs. They use retrieval-augmented generation as a guardrail rather than as a generator. For each query segment, an approximate nearest neighbor lookup retrieves the top 100 closest taxonomy concepts from the existing graph. The LLM is then prompted to pick from that list rather than invent labels. This is a clever inversion of the usual RAG pattern, where RAG typically injects context into a generator. Here, RAG defines the entire output space, so the system only ever produces concepts that the rest of the design already knows how to handle. The measured impact is a roughly 30% lift in the trigger rate for popular dish carousels, all delivered through an architecture whose runtime stays mostly classical. The takeaway is that DoorDash's LLM lives mostly offline, mostly in batch, working on the periphery of the runtime.

Read full article ↗excerpt · fair-use quotation