31 moments across 7 channels for “AI agents' success rate on end-to-end, economically valuable fre”
Amid a flurry of new AI regulatory moves in Washington, the authors warn that open source AI could get swept up in bans or restrictions, which they call a serious error.
Kevin Xu (Interconnected, co-author) · Interconnects · listen to the original →
Kevin Xu (Interconnected, co-author) · Interconnectsconf 80%
Kevin Xu (Interconnected, co-author) · Interconnectsconf 80%
Kevin Xu (Interconnected, co-author) · Interconnectsconf 80%
Kevin Xu (Interconnected, co-author) · Interconnectsconf 80%
Kevin Xu (Interconnected, co-author) · Interconnectsconf 80%
The Remote Labor Index shows frontier AI success on real freelance projects jumping from 2.5% to 16.1% in under a year, with Fable 5 leading GPT-5.5 and Opus 4.8.
Jack Clark argues that AI capability growth is outpacing humans' ability to develop new comparative advantages, predicting that person-light, AI-heavy organizations will increasingly dominate the economy despite human innovation and augmentation.
Thomas Ahle notes that scaling AI agents to hardware workloads would cost roughly $10 billion in commercial EDA licensing fees, which he says is part of why AI models remain undertrained for chip design.
Citing the Remote Labor Index's jump from 2.5% to 16.1% success in nine months, Jack Clark argues AI capability growth is outpacing human adaptation, predicting AI-heavy, person-light organizations will take over parts of the economy.
Current AI describes itself as a global partnership building a public option for AI; it was founded as a non-profit at the Paris AI Action Summit in February 2025 and has already secured $400m in funding.
Cursor's Sualeh Asif explains that cloud agents can't 'complain' about errors the way local agents can, so Cursor built a system where agents give periodic 'confessions' that get routed to the infra team.
The AARRI-Bench benchmark evaluates AI agents on research-intern-level tasks—like spotting fabricated data or refusing to falsify results under pressure—with the top model, Claude-Opus-4.7, scoring 68.3%, suggesting AI is starting to be useful as a research assistant.
NVIDIA's ENPIRE harness lets coding agents autonomously iterate on real-world robot policies, achieving up to 99% success on dexterous manipulation tasks like PushT and zip-tie cutting.
Meta engineers told the author the outage stemmed from unreviewed AI-generated code and gutted Integrity teams whose staff had been reassigned to AI labeling duties.
Peter Steinberger got fed up with local OpenClaw agents overheating his CPU and slowing his system, so he built Crabbox to run them remotely in the cloud.
Dylan argues fast AI progress favors the US because Anthropic-style revenue and compute investment compound faster than China can build comparable lab-scale infrastructure, but a slow enough timeline gives China's fully indigenized supply chain room to catch up and overtake the West's more fragmented, multi-country one.
Gergely Orosz hypothesizes that cloud agents are taking off now due to converging factors: sufficiently capable coding models, mature agent infrastructure (MCP/skills), bigger context windows, and abundant cloud GPU capacity.
Thomas Ahle warns that AI risk isn't only about AI capability increasing — it's compounded by humans getting lazier and less knowledgeable, e.g. no longer reading papers themselves and just asking AI to explain them.
Matei explains that Omnigent emerged from noticing internal coding-agent tooling and custom enterprise agents kept hitting identical problems — switching models and harnesses, sharing sessions, security — so Databricks built one common layer to serve both.
Katelyn Lesse, head of engineering for Claude Platform, describes Claude Managed Agents as a major six-month engineering effort to host long-running agents across multiple cloud providers.
Simon Willison pinpoints November 2025 as the inflection point when AI agents became genuinely useful, explaining the current wave of corporate spending on the technology.
AINews frames OpenAI's explosive internal Codex growth as proof that even AI insiders with free, unlimited access were dramatically underusing AI tools until very recently.
Jon Durbin argues inference efficiency has become the central strategic bottleneck in AI, since all upstream work — data, RL, and agents — ultimately resolves into test-time compute costs.
OSWorld 2.0's creators find that even the strongest model setup only hits 20.6% binary accuracy on tasks averaging 1.6 hours of human effort, with agents struggling most on hidden-state recovery, tracking many items, and adapting to changing requirements.
Import AI's author argues that stronger, independently-developed alignment techniques are essential as AI takes on more autonomous research and self-improvement work, and that watchdog-style organizations can help sound the alarm on frontier labs.
Thomas Ahle argues that reporting benchmark pass-rates (like '70-80% of tests correct') is misleading, since a program that fails even some tests is likely not truly correct overall.
A large four-study experiment (18,978 conversations, 6,923 people) found frontier AI models reliably out-persuaded expert human debaters, even ones who prepared extensively and were paid cash bonuses to perform well.
D Smith explains that the headline 36% score on ARC-AGI-3 doesn't mean 36% of games were solved — it's an action-efficiency ratio versus a human baseline, so models solving far more games than 36% still score low if they're inefficient.
OSWorld 2.0, a benchmark of 108 long-horizon computer-use tasks averaging 1.6 hours for a human to complete, shows even top models like Claude Opus 4.8 achieving only 20.6% binary accuracy, though rapid gains are expected as happened with OSWorld 1.0.
Eric Jang notes he rebuilt a strong Go bot for about $10K in rented compute — versus AlphaGo Zero's roughly 3E23 FLOPS — because being first to solve a problem is inherently far more expensive than catching up once someone else has already solved it.
Simon Willison describes overcoming his initial skepticism about cross-model code review, saying he now routinely has Anthropic's best model review OpenAI's output and vice versa because it reliably turns up useful findings.