30 moments across 7 channels for “Frontier AI agents' reliability on long-horizon, multi-step, mul”
Amid a flurry of new AI regulatory moves in Washington, the authors warn that open source AI could get swept up in bans or restrictions, which they call a serious error.
Kevin Xu (Interconnected, co-author) · Interconnects · listen to the original →
Kevin Xu (Interconnected, co-author) · Interconnectsconf 80%
Kevin Xu (Interconnected, co-author) · Interconnectsconf 80%
Kevin Xu (Interconnected, co-author) · Interconnectsconf 80%
Kevin Xu (Interconnected, co-author) · Interconnectsconf 80%
Kevin Xu (Interconnected, co-author) · Interconnectsconf 80%
OSWorld 2.0, a benchmark of 108 long-horizon computer-use tasks averaging 1.6 hours for a human to complete, shows even top models like Claude Opus 4.8 achieving only 20.6% binary accuracy, though rapid gains are expected as happened with OSWorld 1.0.
OSWorld 2.0, a benchmark of multi-hour, multi-program computer-use tasks, shows even Claude Opus 4.8 with maximum thinking only reaches 20.6% binary accuracy, though the field expects a rapid ramp similar to OSWORLD 1.0's rise from ~30% to ~75%.
Thomas Ahle notes that scaling AI agents to hardware workloads would cost roughly $10 billion in commercial EDA licensing fees, which he says is part of why AI models remain undertrained for chip design.
Current AI describes itself as a global partnership building a public option for AI; it was founded as a non-profit at the Paris AI Action Summit in February 2025 and has already secured $400m in funding.
Cursor's Sualeh Asif explains that cloud agents can't 'complain' about errors the way local agents can, so Cursor built a system where agents give periodic 'confessions' that get routed to the infra team.
The AARRI-Bench benchmark evaluates AI agents on research-intern-level tasks—like spotting fabricated data or refusing to falsify results under pressure—with the top model, Claude-Opus-4.7, scoring 68.3%, suggesting AI is starting to be useful as a research assistant.
Meta engineers told the author the outage stemmed from unreviewed AI-generated code and gutted Integrity teams whose staff had been reassigned to AI labeling duties.
Jack Clark argues that AI capability growth is outpacing humans' ability to develop new comparative advantages, predicting that person-light, AI-heavy organizations will increasingly dominate the economy despite human innovation and augmentation.
Peter Steinberger got fed up with local OpenClaw agents overheating his CPU and slowing his system, so he built Crabbox to run them remotely in the cloud.
Gergely Orosz hypothesizes that cloud agents are taking off now due to converging factors: sufficiently capable coding models, mature agent infrastructure (MCP/skills), bigger context windows, and abundant cloud GPU capacity.
Matei explains that Omnigent emerged from noticing internal coding-agent tooling and custom enterprise agents kept hitting identical problems — switching models and harnesses, sharing sessions, security — so Databricks built one common layer to serve both.
Import AI's author argues that stronger, independently-developed alignment techniques are essential as AI takes on more autonomous research and self-improvement work, and that watchdog-style organizations can help sound the alarm on frontier labs.
Startup Recursive reported new state-of-the-art results on small-model training and GPU kernel optimization benchmarks using its automated research-loop system, framing it as an early proof point for recursive self-improvement.
Dylan argues fast AI progress favors the US because Anthropic-style revenue and compute investment compound faster than China can build comparable lab-scale infrastructure, but a slow enough timeline gives China's fully indigenized supply chain room to catch up and overtake the West's more fragmented, multi-country one.
Thomas Ahle warns that AI risk isn't only about AI capability increasing — it's compounded by humans getting lazier and less knowledgeable, e.g. no longer reading papers themselves and just asking AI to explain them.
Z.ai's GLM-5.2 Max posted a leading Code Arena Frontend score of 1595, beating Opus 4.8 and closing in on Claude Fable 5, while also edging out Opus 4.8 Max on an agentic reliability benchmark.
Katelyn Lesse, head of engineering for Claude Platform, describes Claude Managed Agents as a major six-month engineering effort to host long-running agents across multiple cloud providers.
Citing the Remote Labor Index's jump from 2.5% to 16.1% success in nine months, Jack Clark argues AI capability growth is outpacing human adaptation, predicting AI-heavy, person-light organizations will take over parts of the economy.
Jon Durbin argues inference efficiency has become the central strategic bottleneck in AI, since all upstream work — data, RL, and agents — ultimately resolves into test-time compute costs.
AINews frames OpenAI's explosive internal Codex growth as proof that even AI insiders with free, unlimited access were dramatically underusing AI tools until very recently.
Simon Willison pinpoints November 2025 as the inflection point when AI agents became genuinely useful, explaining the current wave of corporate spending on the technology.
Sequent positions its research strategy as seeking principled confidence that alignment generalizes to uncontrolled, real-world situations, contrasting this with what it sees as the reactive methods used by major AI labs.
By assuming training and inference compute costs should roughly equalize, and plugging in estimates of global tokens-per-second and model active-parameter counts, Reiner Pope derives that frontier models are likely over-trained by about a factor of 100 relative to the Chinchilla-optimal point — a consequence of needing cheap inference and RL-generation compute, not just training-optimal quality.
CAIS and Scale researchers' Remote Labor Index shows frontier models' success on real-world freelance tasks (3D/CAD, design, video, data analysis) jumping from 2.5% to 16.1% in nine months, with Fable 5 leading at 16.1%.
Stephano describes a recurring failure mode where agents fixate on nonsensical goals (like minimizing an energy bar) and get stuck unable to abandon the wrong hypothesis, something a human would instantly see through.