YouTube removed the search index entirely with PLUM, which assigns videos semantic IDs and treats retrieval as a generation task: an adapted Gemini-family language model produces the codes of the next videos a user is likely to watch, gaining much wider long-tail coverage while risking generated identifiers that map to no video.

YouTube's PLUM removes the retrieval index: it gives every video a Semantic ID derived from its content, adapts a Gemini-family language model by adding these IDs to its vocabulary, and performs retrieval as a generation task producing the Semantic IDs of likely next videos via beam search. The payoff is far wider long-tail coverage and a 4.96 percent click-through lift on Shorts, at the cost of generating identifiers that map to no video (kept under five percent). ✦ AI generated

Author (article How to Fight Clickbait: Meta, LinkedIn & YouTube Case Studies) · ByteByteGo Newsletter · 2026-08-10 · original ↗

The system, called PLUM, assigns every video a Semantic ID, a short sequence of discrete codes derived from the video's own content. Videos with similar content receive similar codes, so the identifier carries information about the item rather than acting as a random label. PLUM then adapts a pretrained language model (from the Gemini family) by adding these Semantic IDs to its vocabulary and continuing to train it on video metadata and user activity. After this adaptation, the model performs retrieval as a generation task. Given a user's recent history, it produces the Semantic IDs of the videos that the user is likely to watch next... This approach, however, introduces a failure mode that the index-based designs avoid, which is generating an identifier that maps to no video, and PLUM reports keeping that rate below five percent after fine-tuning. Measured against the previous production system, PLUM surfaced a far wider range of long-tail videos, and on YouTube Shorts, it raised panel click-through by 4.96 percent.

Read full article ↗excerpt · fair-use quotation