Data◆Article
Qwen3.6 actually performs better when run through the Codex harness than through its own purpose-built Qwen-Code harness.
Sebastian Raschka · Ahead of AI
DataArticle
On a new independent AutomationBench leaderboard spanning 657 tasks across 40 simulated SaaS apps, Claude Fable 5 leads with 48.6% accuracy, narrowly ahead of Opus 4.8 at 48.5%, while every model tested still breaks business-rule guardrails at some rate.
Artificial Analysis's independent AutomationBench-AA leaderboard shows Claude Fable 5 narrowly leading over Opus 4.8, with Gemini and GPT-5.5 further behind, and notes every model still violates business-rule guardrails. ✦ AI generated
Artificial Analysis · Latent Space · 2026-07-07 · original ↗
AutomationBench-AA adds a more realistic agent eval: @ArtificialAnlys launched an independent leaderboard for Zapier’s AutomationBench, evaluating agents across 657 tasks and 40 simulated SaaS apps with both objectives and guardrails. Claude Fable 5 led at 48.6%, narrowly ahead of Opus 4.8 at 48.5%, with Gemini 3.5 Flash at 42.6% and GPT-5.5 xhigh at 42.1%.
Read full article ↗excerpt · fair-use quotation
Related moments
Data◆Article
Frontier coding agents operating ENPIRE's autonomous feedback loop can develop real-world robot manipulation policies that hit up to 99% success rates on tasks like PushT, pin organizing, and zip-tie cutting.
NVIDIA researchers · Import AI
DataArticle
Current frontier AI agents remain far from reliable at long-horizon, multi-program computer-use tasks, with the best-performing setup reaching only 20.6% binary accuracy on OSWORLD 2.0, though rapid improvement is likely given how quickly scores rose on OSWORLD 1.0.
OSWorld 2.0 researchers · Import AI