Why DoorDash, Instacart, and Uber Eats Integrated LLMs Into Search Three Different Ways

The article author: This pattern repeats across several common failure modes in food search. For example: Synonyms: 'Soda' and 'soft drink' describe the same product, but a keyword engine treats them as different tokens. Typos: 'Mozzarela' should retrieve mozzarella results, but the spelling mismatch breaks the lookup. Shorthand: 'Gf pizza' meaning gluten-free pizza requires the system to recognize an abbreviation as a synonym for the full phrase. Language mix: The Spanish word 'pan' means bread, while the English word 'pan' means a cooking vessel, so a bilingual search bar has to disambiguate. Word-sense ambiguity: 'Apple' the fruit and 'Apple' the company share the same spelling but mean different things, and the right answer depends on context. Each one is a potential moment where the user's intent and the catalog's words fail to line up. Two harder problems lie beneath this: The long tail... The constraint problem... Food search is the right domain to watch this play out, because all of these failure modes show up at the same time.

The article author: Instacart's strategy is layered across three approaches: Context engineering: Retrieval-augmented generation pulls Instacart-specific context (top-converted categories, historical conversion data, catalog details) into the prompt before the LLM sees a query. Post-processing guardrails: Semantic similarity filters drop LLM outputs that drift away from the original query. Fine-tuning: For the most advanced tasks, the team fine-tunes Llama-3-8B on Instacart's proprietary data, so domain knowledge gets baked directly into the model weights... The serving architecture splits along the head-versus-tail distribution. Head queries hit an offline RAG-and-cache pipeline that is latency-tolerant and deeply context-engineered, while tail queries hit a real-time fine-tuned Llama-3-8B model that keeps latency under 300ms through adapter merging, H100 GPUs, and autoscaling... After this solution, query rewrite coverage jumped from 50% to over 95%, with 90%+ precision across substitutes, broader rewrites, and synonyms. The real-time fine-tuned model improved search quality for the bottom 2% of queries (the cold-start tail), cutting scroll depth by 6% and complaints about poor tail-query results by half. The takeaway is that Instacart's LLMs live at the query understanding layer, with some offline-cached and some online and fine-tuned, while retrieval and ranking downstream are still done by traditional machine learning and information retrieval systems.

The article author: The architecture they decided on is a classic two-tower setup, where a query encoder and a document encoder each produce vectors in a shared space, and matching is done by similarity in that space. The twist is in what sits inside each tower, because both towers use a fine-tuned Qwen LLM as their backbone embedding layer. The query tower runs online, embedding each incoming query in real time, while the document tower runs offline and pre-embeds billions of documents into an HNSW vector index... What makes the system viable at scale is a stack of optimizations: Matryoshka Representation Learning trains a single model whose embedding can be truncated to different lengths, and Uber serves at 256 dimensions in production with under 0.3% recall loss compared to the full 1,536. Scalar quantization (int7 instead of float32) cuts latency in half again. Pre-filters on hexagon, city, and fulfillment type shrink the candidate set before the ANN search even runs... The takeaway is that Uber Eats' LLM is the embedding model itself, since every query and every document gets an LLM-derived vector, and retrieval at every level depends on LLM-produced representations.