MechanismArticle

Multi-Token Prediction provides a significant inference speed boost for Qwen 3.8 27B, with one test showing a 72% improvement over the default GGUF configuration.

The model runs slowly by default (15-30 tokens/second) but supports Multi-Token Prediction, a built-in optimization that can boost speed dramatically—tested at 72% faster on a DGX Spark. ✦ AI generated

Simon Willison · Simon Willison's Weblog · 2026-08-16 · original ↗

Share card ↓Export .md Export .txt

One of the most promising optimizations is baked into the model itself. Qwen supports Multi-Token Prediction, an architecture trick where a cheaper mechanism guesses several tokens ahead and the main model can then quickly verify if the guesses were correct. This can have quite a dramatic effect on inference performance. I had GPT-5.6 in Codex run a comparative benchmark on the Spark and the --spec-type draft-mtp server outperformed the LM Studio default GGUF by around 72%.

Read full article ↗excerpt · fair-use quotation

✳ llama.cpp ✳ Multi-Token Prediction ✳ Qwen 3.8 27B

Around this claim

This moment responds to

gives example → Inference engineering still yields huge gains of 20%, 100%, or even 200%, and the race to 10x faster inference is realistic by stacking optimizations such as quantization, speculative decoding, and disaggregation.Philip Kiely · Latent Space