MechanismArticle

Xiaomi reached 1000 tokens/second on a 1-trillion-parameter model by codesigning the model with its software stack—FP4 quantization, DFlash speculative decoding, and TileRT-optimized inference—running on an ordinary 8-GPU commodity node rather than specialized hardware.

Xiaomi's MiMo-V2.5-Pro-UltraSpeed model hits 1000 tokens per second through co-designed quantization and speculative decoding techniques running on commodity 8-GPU nodes, reflecting a broader push by Chinese firms to maximize efficiency amid export controls. ✦ AI generated

Xiaomi · Import AI · 2026-06-15 · original ↗

Export .md Export .txt

Xiaomi was able to do this by codesigning the model with the software stack around it, including obvious things like FP4 quantization, as well as using DFlash (a "speculative decoding method based on block-level masked parallel prediction"), and also working closely with TileRT, software from startup Tile AI which speeds up LLM inference on commodity hardware.

Read full article ↗excerpt · fair-use quotation

▦ Tile AI ▦ Xiaomi

Related moments

Claim◆Article

Cracking down on open source AI out of fear of China would backfire: American startups already rely on open source models, including Chinese ones, to compete, and restricting them would only weaken US education, innovation, and competition.

Kevin Xu (Interconnected, co-author) · Interconnects

Data◆Article

Sonnet 5 requires roughly 3x the agentic turns of Sonnet 4.6 on benchmarks like AA-Briefcase and GDPval-AA, and at max effort it takes around 6x more turns than at low effort.

Artificial Analysis · Latent Space

PredictionAudio · 65:37 · 2m

Whether the US or China ends up dominating AI compute hinges on takeoff speed: a fast AI takeoff lets the West's compounding revenue and compute investment widen its lead, but if AI progress is slow enough to stretch out toward 2035, China's more vertically indigenized semiconductor supply chain could let it eventually scale past the West.

Dylan Patel · Dwarkesh Podcast