Audio · 2026-05-28 · 41m · 12 moments

Building an AI Guardian for Enterprise with Onyx Security CEO Maxim Bar Kogan

We are now closer than ever before to living in a world where AI agents are smart enough to run our power grids and manage water supplies. How do we keep them from going rogue? Sarah Guo sits down with Maxim Bar Kogan, founder and CEO of Onyx Securities, to explore the complexities of supervising and securing autonomous agents at the enterprise level. Maxim explains Onyx’s product as an AI control plane, which oversees the permissions and flexible contexts of agents while balancing latency, cost ✦ AI generated

Apple Podcasts ↗

GUEST Maxim Bar Kogan HOST Sarah Guo HOST Elad Gil

timeline · colored by role

Claim◆

As enterprises exponentially increase their use of AI agents, they will inevitably face catastrophic actions from agents going rogue—publishing code, deleting data, taking unauthorized actions—and existing security tooling is fundamentally unequipped to prevent or understand these failures.

Maxim argues that the exponential growth in AI agent usage guarantees serious failures, and existing security stacks lack the contextual understanding to prevent them, creating an urgent need for a new category of AI-specific control systems.

transcript

Maxim Bar Kogan: As you're exponentially doing more things with AIs, you're going to start having really bad actions happen. And we've seen some of that happen lately with agents accidentally publishing code and tokens that they weren't supposed to. Like definitely enterprise are starting to realize that that risk has grown exponentially and that they don't have any way to stop the adoption. They just now have to do something to reduce the chance of these agent actions being illegitimate or incorrect.

explains mechanism · 3gives example · 1

Claim◆

As enterprises exponentially scale AI agent usage, the number of agent actions will grow 100X to a million X, making human-in-the-loop oversight impossible and requiring automated AI oversight systems.

Maxim Bar Kogan argues that as agent actions explode in volume, traditional human review cannot scale, necessitating automated oversight by specialized AI systems.

transcript

Maxim Bar Kogan: The number of these actions is growing exponentially. And so things that we thought might be useful in the past, like a human in the loop, now that you're going to have 100X, 1,000X, a million X of these actions, that's not going to work.

explains mechanism · 2extends · 1

Example

AutoGPT was the pivotal inspiration for Onyx because it demonstrated the future of truly autonomous agents—an LLM deciding what to do, calling tools in a loop—even though the models at the time weren't good enough to make it work reliably.

Maxim traces Onyx's founding insight to AutoGPT, the first autonomous agent running on LLMs, which showed what agents could become once models caught up—a structure he compares directly to today's Claude Code.

transcript

Maxim Bar Kogan: AutoGPT, and I'm sorry if I don't know the guy behind it, but a huge fan, they created the first, as far as I know, first really autonomous agent running on LLMs, right? So an agent that, you know, would let LLM not generate text, but decide what to do, and then give that agent an API access to do that thing a tool to do it, and then would do that in a loop. So basically, in theory, could let agents do very complicated things, anything a person could do on the computer. Now, granted, it didn't work that well. It was too early. The models were not good enough. GPT-4 was not good enough. But I think it did give everyone a glimpse into the future of what if the models were good enough, and then basically using that same structure, we could have very capable agents doing stuff for us. I think that was, in many ways, Claude Code today is not dissimilar to AutoGPT back then. I think they were a bit early on, again, before the models were ready, but the concept was right.

gives example · 1provides context · 1

Claim

Existing security controls—identity, endpoint, API security—fail for autonomous agents because agents need broad, human-level permissions to be useful, and the tools cannot distinguish legitimate from illegitimate actions without knowing what the agent was thinking.

Maxim walks through why identity-based permission models break when agents need open-ended access, and why endpoint and API tools cannot distinguish a valid database deletion from a rogue one without the agent's reasoning context.

transcript

Maxim Bar Kogan: With these autonomous AIs, with these assistants, with these coding agents, we kind of want them to have our permissions because we want to tell Cloud Code to do something or Cloud Cowork to do something, and we want to then go have lunch and we want to come back and see that it's done. And we want to give it so many diverse tasks as well that we kind of can't find the right set of permissions to do. So suddenly our identity security software is not very useful. Then if you think about endpoint security, right, or API security, if we tell our cloud code that we want to recreate a database and it should delete it and recreate it, that's great. That's going to save our DevOps team and our platform teams a lot of time. It's a great benefit of Cloud Code, but if Cloud Code is working on an unrelated task and suddenly thinks that maybe the right thing to do is to delete our database and recreate it, maybe we don't want that to happen. And unfortunately, our endpoint providers or API security tools, they don't know what Cloud was thinking, why is it doing what it's doing, right?

explains mechanism · 3

Mechanism

Traditional proxy-based security with a policy engine fails for AI agents because the hard problem isn't seeing the data—it is understanding what another AI system is thinking and whether its planned action is legitimate, which requires context no existing tool has.

Maxim explains why a conventional security approach of proxying traffic with rules doesn't work: the real difficulty is determining whether an action is legitimate given what the AI intended, and existing endpoint, API, and identity tools have no access to that reasoning context.

transcript

Maxim Bar Kogan: There are a few things that, I mean, proxy's integration method, I would say. So there are some AI systems where you would want to integrate with a proxy if that's the easiest way to do it. But number one, there's a lot of systems where that's just not viable technically because AI today runs on the cloud, on someone else's infrastructure, on your endpoint, and just proxy is not always an option. And the second thing is the question, okay, great, you're proxying, so you're seeing the data, you're seeing it, but that's not the hard problem. The hard problem is understanding if what I should do now. It turns out that in the case of AI systems, that is the hard question. Like, what is the engine that needs to underwrite these different actions and say if they're okay or not? And because we need to be able to understand what another AI system is thinking, what is it planning to do, and then have our own opinion on that. And consider we're trying to understand some of the smartest models in the world are doing the right thing. So who are we to do it? How are we going to do it correctly, right? And so that turns out to be a really difficult technical question.

explains mechanism · 1

Mechanism◆

Onyx trains small, specialized models that act as a triage layer—capable only of deciding whether a more expensive, smarter agent needs to intervene—solving the cost and latency problem that would otherwise make agent oversight economically impossible.

Maxim explains the core technical insight behind Onyx: rather than running an expensive frontier-model agent for every action, they train tiny, single-purpose models that filter routine actions from truly risky ones, analogous to a grandmaster playing intuitive blitz moves until a critical position demands deep calculation.

transcript

Maxim Bar Kogan: If I need to run an agent for every agent you're running as your security vendor, you're gonna be paying for me more than you're paying for your AI, right? So it's not, it's pretty much a deal breaker. And also it's gonna be so slow, so you're not gonna be happy with whatever latency you're gonna get. And so the challenge then becomes, how do I know what are the times where I need to interject with these smart agents to look at what's happening? And that's when actually what you want to do is you want to try to train very smart models that are, actually, let me correct that, very not smart models, but models that are just good at one thing. They're very small. They almost can't do anything else other than be able to say, should I have a smarter agent look at this? And if you manage to bake in that intuition into those small models, well, in the sense that they don't miss a lot of stuff and they don't call that other agent too much, then you can get to a really good balance of, we're very performant, we have smart agents overseeing things when needed, but we're not, but our costs are low and our latency is low.

explains mechanism · 1extends · 1provides context · 1

Mechanism

Existing enterprise security tools—identity security, endpoint security, API security—cannot secure autonomous AI agents because they lack the contextual understanding of what the agent is thinking and why it is taking each action.

Bar Kogan explains that traditional security controls fail with AI agents: identity permissions can't be narrowly scoped for flexible agents, and endpoint/API tools can't distinguish legitimate from malicious agent actions without knowing what the agent intends.

transcript

Maxim Bar Kogan: Our endpoint providers or API security tools, they don't know what Cloud was thinking, why is it doing what it's doing, right? So a lot of these existing tools, they don't have the context to understand what these very flexible, unpredictable systems are doing.

explains mechanism · 3supports · 1

Prediction◆

Long term, controlling advanced AI will require understanding the internal weights and activations of models through mechanistic interpretability, and smarter-than-human models will likely become the tools that finally crack that problem.

Maxim argues that the long-term solution for AI control goes beyond security into the mathematical structure of models themselves. He suggests that while humans may find mechanistic interpretability too difficult, superior AI models will be able to understand and audit other AI systems' internal reasoning.

transcript

Maxim Bar Kogan: We believe that there's been a lot of strong progress in that direction. We believe that understanding the internal weights and activations is the internal structure, the mathematical structure of these systems is going to be at least part of the solution. And in many ways we think that, and this is maybe, you know, we'll only know when we get there, but we think that for our level of intelligence, it's kind of difficult to understand very quickly what is the internal structure of a large language model. What is the internal structure of the weights and activations? ...I think as humans, it might still be very difficult to understand what weights and activations mean and maybe mechanistic interpretability. It seems like, oh, maybe that's too hard or shouldn't be possible. But as we're starting to have models that are much smarter than us, at least in some important ways, we think that we'll be able to start cracking mechanistic capability much more effectively.

Claim◆

The hard problem in securing AI systems is not proxying traffic or seeing data, but building an engine that can understand what another AI system is thinking and planning, then form its own judgment—which requires understanding some of the smartest models in the world.

Bar Kogan rejects a simple proxy-with-policy approach, arguing the core challenge is semantic: a security system must understand what the AI agent intends, which is a profoundly difficult technical problem.

transcript

Maxim Bar Kogan: The hard problem is understanding if what I should do now. It turns out that in the case of AI systems, that is the hard question. Like, what is the engine that needs to underwrite these different actions and say if they're okay or not? And because we need to be able to understand what another AI system is thinking, what is it planning to do, and then have our own opinion on that. And consider we're trying to understand some of the smartest models in the world are doing the right thing.

explains mechanism · 2

Mechanism

Running a full-capability agent to oversee every operating agent is impractical due to cost and latency, so the correct architecture uses small, highly specialized models as triage classifiers that decide when to escalate to a smarter agent for deep review.

Bar Kogan explains that Onyx's approach avoids the cost and latency of running a full LLM overseer per agent by training small, focused models that efficiently flag high-risk actions requiring deeper inspection.

transcript

Maxim Bar Kogan: If I need to run an agent for every agent you're running as your security vendor, you're gonna be paying for me more than you're paying for your AI, right? So it's not, it's pretty much a deal breaker. And also it's gonna be so slow. And so the challenge then becomes, how do I know what are the times where I need to interject with these smart agents to look at what's happening? And that's when actually what you want to do is you want to try to train very smart models that are, actually, let me correct that, very not smart models, but models that are just good at one thing. They're very small. They almost can't do anything else other than be able to say, should I have a smarter agent look at this?

Prediction

The long-term opportunity is to build an independent, non-vendor-aligned company that oversees and controls what AI systems do—a $100 billion-plus opening for a company that is not the vendor of the AI itself.

Bar Kogan frames the mission as extending beyond enterprise security to the fundamental problem of controlling advanced AI, positioning Onyx as an independent third-party overseer in a market he estimates at over $100 billion.

transcript

Maxim Bar Kogan: If you have AI companies that are $10 trillion companies, we think you want a company that is not the vendor of the AI itself to oversee and help you control what AI is doing. And we think that's an opening that's $100 billion plus opening for a really important company.

extends · 3

Prediction

Mechanistic interpretability—understanding the internal weights, activations, and mathematical structure of models—will be at least part of the solution for controlling advanced AI, and smarter-than-human models will be better at cracking it than humans are.

Bar Kogan expresses optimism about mechanistic interpretability research, arguing that as models become superhuman in key capabilities, they will help unlock understanding of model internals that is difficult for humans alone.

transcript

Maxim Bar Kogan: We believe that understanding the internal weights and activations, the internal structure, the mathematical structure of these systems is going to be at least part of the solution. And we think that for our level of intelligence, it's kind of difficult to understand very quickly what is the internal structure of a large language model. But as we're starting to have models that are much smarter than us, at least in some important ways, we think that we'll be able to start cracking mechanistic interpretability much more effectively.

extends · 2

Highlight slides

Agent Action Volume Outruns Human Oversight✦ from: As enterprises exponentially scale AI agent usage, the number of agent actions will grow 100X to a million X, making human-in-the-loop oversight impossible and requiring automated AI oversight systems.AI Agents: Exponential Risk, Zero Control✦ from: As enterprises exponentially increase their use of AI agents, they will inevitably face catastrophic actions from agents going rogue—publishing code, deleting data, taking unauthorized actions—and existing security tooling is fundamentally unequipped to prevent or understand these failures.The Scaling Gap: Human Review vs. Agent Actions✦ from: As enterprises exponentially scale AI agent usage, the number of agent actions will grow 100X to a million X, making human-in-the-loop oversight impossible and requiring automated AI oversight systems.The Cost/Latency Trap in AI Security✦ from: Onyx trains small, specialized models that act as a triage layer—capable only of deciding whether a more expensive, smarter agent needs to intervene—solving the cost and latency problem that would otherwise make agent oversight economically impossible.Tiny Triage Models as the Solution✦ from: Onyx trains small, specialized models that act as a triage layer—capable only of deciding whether a more expensive, smarter agent needs to intervene—solving the cost and latency problem that would otherwise make agent oversight economically impossible.Baking in Intuition for the Right Balance✦ from: Onyx trains small, specialized models that act as a triage layer—capable only of deciding whether a more expensive, smarter agent needs to intervene—solving the cost and latency problem that would otherwise make agent oversight economically impossible.Control Through Understanding✦ from: Long term, controlling advanced AI will require understanding the internal weights and activations of models through mechanistic interpretability, and smarter-than-human models will likely become the tools that finally crack that problem.Smarter Models as the Solution✦ from: Long term, controlling advanced AI will require understanding the internal weights and activations of models through mechanistic interpretability, and smarter-than-human models will likely become the tools that finally crack that problem.The Hard Problem Is Semantic, Not Infrastructural✦ from: The hard problem in securing AI systems is not proxying traffic or seeing data, but building an engine that can understand what another AI system is thinking and planning, then form its own judgment—which requires understanding some of the smartest models in the world.The Engine Must Understand the Smartest Models✦ from: The hard problem in securing AI systems is not proxying traffic or seeing data, but building an engine that can understand what another AI system is thinking and planning, then form its own judgment—which requires understanding some of the smartest models in the world.

Related episodes

Video · The Cognitive Revolution

What Just Happened?

→

Article · Platformer

A big week for AI denialism

→

Audio · All-In Podcast

Anthropic's $30B Ramp, Mythos Doomsday, OpenClaw Ankled, Iran War Ceasefire, Israel's Influence

→