DataArticle · 7:24 · 2m
Kimi K3 produces significantly simpler, more readable code than Codex, but misses edge cases that Codex catches, making it roughly equivalent to a GPT-54 or 55 level for coding tasks.
Florian Brand · Interconnects
Data◆Article
Current frontier models cannot beat DiG-bench, with only Opus 5 and Fable 5 reaching Tier 7, suggesting AI still struggles with creative discovery compared to humans.
Despite being difficult, games in DiG-bench are beatable by humans, but today's best AI models struggle significantly, indicating a gap in discovery capabilities. ✦ AI generated
author · Import AI · 2026-08-17 · original ↗
Overall, this seems really hard! ... some frontier models are already capable of some fairly impressive feats of discovery, but still struggle compared to humans (for instance, a 20% success rate on Tier 7 is pretty poor compared to the fact individual humans were able to get 100% on the tests).
Read full article ↗excerpt · fair-use quotation
- ·Frontier models cannot beat DiG-bench overall
- ·Only Opus 5 and Fable 5 reach Tier 7
- ·20% success rate on Tier 7 is poor compared to humans
- ·Individual humans achieved 100% on tests
- ·AI still trails behind human discovery capabilities
- ·Games are beatable but models lag significantly
Related moments
DataArticle · 68 words
Today's AI systems are extraordinarily capable engineers but lack tasteful, original creativity — they fail to produce research of top ML conference caliber because they commit to narrow research paths early and cannot reverse out of unpromising approaches.
Authors of the shadow-evaluation study (Princeton, Cornflower Labs, UK AI Security Institute, et al.) · Import AI