Article · 2026-08-05 · 3 moments

Third-party cyber evaluations involving OpenAI models

Third-party cyber evaluations involving OpenAI models And another one. I had to create a accidental-cyberattacks to group these all together. This post from OpenAI covers both the UK AI Safety Institute attack (see my previous post) and another attack enabled by Irregular: Irregular, one of our external cybersecurity testing partners, was running Capture-the-Flag-style evaluations intended to be isolated from the internet, but a testing-environment misconfiguration allowed models to access th ✦ AI generated

Read original ↗

GUEST OpenAI GUEST Author (original poster)HOST Simon Willison

Fact◆

A testing-environment misconfiguration during Irregular's CTF-style evaluations allowed OpenAI models to access the public internet instead of staying isolated.

An OpenAI post covers attacks enabled by Irregular's misconfigured CTF evaluation environment, which leaked internet access to the models.

transcript

OpenAI: Irregular, one of our external cybersecurity testing partners, was running Capture-the-Flag-style evaluations intended to be isolated from the internet, but a testing-environment misconfiguration allowed models to access the public internet.

supports · 1

Mechanism◆

In one test, the fictional target's name in the CTF challenge unintentionally coincided with a real domain, and the misconfigured environment led the model to exploit a real website it mistook for the simulation.

A coincidental name collision between a fictional CTF target and a real domain, combined with the leaked internet connection, caused the model to attack a genuine website.

transcript

OpenAI: In one test, the name of the fictional target for the CTF challenge unintentionally coincided with a real domain. Because the testing environment was mistakenly connected to the internet, the model exploited a real website, mistaking it to be part of the simulated environment.

explains mechanism · 2

Context

Irregular also hosted the misconfigured evaluation environment that gave Claude live internet access during some of Anthropic's tests.

Irregular appears in Anthropic's write-up as the provider of the misconfigured environment that allowed Claude live internet access during some tests.

transcript

Author (original poster): Irregular also feature in Anthropic's write-up - they were hosting the misconfigured evaluation environment which gave Claude live internet access during some of those tests.

extends · 1

Highlight slides

Irregular's CTF test leaked internet access✦ from: A testing-environment misconfiguration during Irregular's CTF-style evaluations allowed OpenAI models to access the public internet instead of staying isolated.Name collision lures model onto a real website✦ from: In one test, the fictional target's name in the CTF challenge unintentionally coincided with a real domain, and the misconfigured environment led the model to exploit a real website it mistook for the simulation.

Related episodes

Video · The Cognitive Revolution

Fable's Back, AI Engineer Recap, & SambaNova

→

Audio · All-In Podcast

Anthropic's $30B Ramp, Mythos Doomsday, OpenClaw Ankled, Iran War Ceasefire, Israel's Influence

→

Audio · All-In Podcast

Elon's Anthropic Deal, The Next AI Monopoly?, "FDA for AI" Panic, Trading the AI Boom

→