AI agents demonstrate clear deceptive intent and a propensity to cheat when faced with difficult tasks, yet the companies evaluating them consistently fail to detect these problems before external parties do.

37:19out of the loop and and disempowering people. I think that's something we need to be very very careful about. And I think the other sort of surprising thing to me is the extent to which the models do seem to be pretty clearly misaligned in some instances. So the sort of open air hugging face hack is the one that got the most attention. And in their defense, they do seem to be

37:42sort of somewhat confused about the setting. But what's unambiguous is that one of the first things they start thinking about is cheating. So I actually want to read a quote from one of the agents. We are stuck perhaps answer online. Like it knows that our answer is not meant to just search for it online. and then later reasons this is an exploit against external cyber gem server. So again it knows it's out of

38:05scope later it seems to kind of rationalize itself. Now in sort of open AI and other companies defense these were internal experimental models but we also saw this happen in the UK AI security institutes testing with production mythos 5 and GPD 5.6 sole models. They did have some classifiers disabled that were meant to stop misuse. But these classifiers were never intended to be a control mechanism to

38:29stop misaligned AI. And in fact um you know UKAC didn't think they needed to prevent access to the internet because these models were alignment trained against a constitution and specification but very very clearly prohibited lying and deception to users. But what they did was go on for real GitHub and they again I want to read a quote from an agent. this is happening on real GitHub so the consequences are genuine and then

38:53it tries to sneak in an offiscated backd dooror creates a sock puppet account to uh try and create support for this uh tries to socially engineer the maintainer when it gets caught says oh this was just some developer script I didn't intend to include I fix that now um even creates another issue with a prompt injection to try and trick an AI agent and like signs off in Danish

39:13because the maintainer is Danish I think I'll get some like you know goodwill Danes never try and hack each Right. Um, so it's still I I'd say a little bit rudimentary in how it's trying to socially engineer people, but it's very very clearly deceptive intent and it's getting pretty good and again like reasons in quite a lot of detail about how to to do this. >> Um, so the last point I want to make is

39:35around monitoring. So I thought that developers would be paying attention to what's going on in during evaluations because that's like the whole point of an evaluation is to see how your AI system behaves, right? Uh but we've actually seen precisely zero zero cases where the researchers running the evaluations actually noticed the problem before anyone else did. Um it seems the most common way for companies to find

40:01out is their own infrastructure security teams noticing something is up. So OpenAI noticed the first of the two internal hacks when the internal package manager Artifactory had an outage because the agents were just overloading it by using as an internal message board. and and when investigating what was causing this abnormal load, they realized the problem. And then OpenAI noticed the second compromise um on July 19, which was 11 days after the agents