AI auditors cannot provide honest public assessments while their continued access depends entirely on the goodwill of the companies they investigate.

9:33really interesting frontier, and we'll hopefully have a chance to get some good takes um from our guests today, especially with Adam Glee from FAR, uh our our first guest back from the break. But, you know, something I observed going all the way back to my GPT4 red team days and I've heard over and over again from leaders of these sort of, you know, they're not just auditors,

9:58right? They they're they're redteamers. Going back to GPT4, you know, Meter was famously a redteamer. They're the ones that had that um finding where the model hired a human on like Fiverr or whatever to solve a capture. You know, this goes this was fall of 2022. So, it's a full six a full full four years since that happened with the original GPT4 um early model. That's what they call it now,

10:22GPT4 early. But the the challenge was always like you're working at the pleasure of the frontier model company. Mhm. >> In my case, you know, I for better or worse was so unimpressed with what they were doing at the on the GVD4 red team project that I ended up taking it to the board and getting kicked out of the project. And you know, I've not done any such work

10:50for them since. And I've heard over and over again, you know, from I won't attribute this to anyone, but you know, there's a relatively small universe of companies that have been kind of in the game where they get these early access opportunities and sometimes they get, you know, special access, including chain of thought access, so they can dig into that. And, you know, sometimes they're doing gonzo type stuff like

11:14we've had past guests that have done, you know, can this AI run a business? And across the board, you know, regardless of kind of the angle, all these companies have expressed to me over and over again, the most important thing I've got to watch out for is I've got to be invited back next time. You know, I can and and they they sort of all have this kind of like um

11:35appreciation for the fact that OpenAI does this. You know, they all kind of recognize like they didn't have to do this at all, right? There's no law that says they have to. They're doing it entirely out of goodwill and belief that it's the right thing to do. Um, but you know, our position is pretty tenuous. Like we have no guarantees. You know, we have no contract. We have no

11:55rights. You know, nobody's going to there's no rule that says anybody has to replace us if they if you know, if they deem us to be doing a bad job for whatever reason. And so protecting their access has been such a huge priority that that that would be my biggest worry now at this stage of the game is like how do we make sure that these people

12:16who have done this for this long who have earned the credibility you know who OpenAI brings in in a crisis how do we make sure that we really as a public get to hear what they really think in a fully honest way and I would never say you know I think I think they've all navigated ated it pretty well to date, but obviously the stakes are rising um