Mechanism◆Audio · 12:44 — 17:16
Open-source video generation models trail closed models like Veo and Kling mainly because of the quadratic attention bottleneck from the sheer number of tokens, and long-form video will likely require autoregressive generation despite current quality shortcomings.
Ali explains the wide gap between open and closed video models, rooted in the token explosion and O(n²) attention cost — five seconds of video already means 35,000 tokens — and argues future long-form video will need autoregressive generation, even though every current autoregressive video model has poor quality. ✦ AI generated
Ali Taha · Latent Space · 2026-08-03 · original ↗
plays this moment only · 12:44 — 17:16
Elicited by
“What about audio, video? What's the other side of inference engineering? Ali, you're pretty big in video diffusion.”
The video models are... The difference between the best open source LLM and best open closed-source LLM is very small... Video models are definitely not. There's a huge gap... difference is night and day. ... For attention, for just five seconds, you're running attention on 35,000 tokens... And because it's O(n²)... to generate a good cut scene of like one minute, it's almost impossible to do within the same compute time... Autoregressive video seems to me like that is the bet that the future's gonna be making, but there are no good open source autoregressive video models out there today... they have to be autoregressive in order to exceed that five second frame.
verbatim transcript · starts at 12:44
- ·Gap between open and closed video models is huge
- ·Difference is 'night and day' unlike LLMs
- ·LLM open/closed gap is now very small
- ·Five seconds of video = 35,000 attention tokens
- ·O(n²) attention makes long video nearly impossible
- ·Good one-minute cut scene too costly in compute
- ·Future long-form video likely requires autoregressive generation
- ·Exceeding five seconds demands autoregressive approach
- ·No good open-source autoregressive video models today
Around this claim