Uber Eats fine-tuned a Qwen LLM into the embedding backbone of two-tower retrieval, so every query and every document gets an LLM-derived vector.

Uber Eats uses a two-tower architecture with a fine-tuned Qwen LLM as the shared embedding backbone. The query tower runs online, the document tower pre-embeds billions of documents offline into an HNSW index. Matryoshka Representation Learning, scalar quantization, and pre-filters make it viable at scale. ✦ AI generated

The article author · ByteByteGo Newsletter · 2026-07-28 · original ↗

The architecture they decided on is a classic two-tower setup, where a query encoder and a document encoder each produce vectors in a shared space, and matching is done by similarity in that space. The twist is in what sits inside each tower, because both towers use a fine-tuned Qwen LLM as their backbone embedding layer. The query tower runs online, embedding each incoming query in real time, while the document tower runs offline and pre-embeds billions of documents into an HNSW vector index... What makes the system viable at scale is a stack of optimizations: Matryoshka Representation Learning trains a single model whose embedding can be truncated to different lengths, and Uber serves at 256 dimensions in production with under 0.3% recall loss compared to the full 1,536. Scalar quantization (int7 instead of float32) cuts latency in half again. Pre-filters on hexagon, city, and fulfillment type shrink the candidate set before the ANN search even runs... The takeaway is that Uber Eats' LLM is the embedding model itself, since every query and every document gets an LLM-derived vector, and retrieval at every level depends on LLM-produced representations.

Read full article ↗excerpt · fair-use quotation