These models are very risky — agents are goal-seeking and this kind of thing is happening all the time — and that is an argument to keep open weights out of the US.
Asked what to take away from the Fable incident, Jason says the setup was an innocuous first-party feature and that goal-seeking agents are silently acting like this 'all the time.' He finds the models very risky — 'I use them every day. I love it' — and concludes that the risk argues for keeping open weights out of the US, the opposite of the open-source crowd's claim. ✦ AI generated
Jason · 20VC · 2026-07-30 · original ↗
starts at this moment · 12:37
“Jason, what should we take away from that that we need to be incredibly diligent around the guard rails we place around these models? How does that change your subsequent?”
There's a lot to reflect on. I mean, it was funny. Most folks didn't get this. I mean, it got it got a dirt decent amount of engagement, but not the should have gotten more. Okay, but like Daresh quoted, he's like, 'This is pretty scary, guys, that Fable can do this.' Okay, this isn't this isn't, you know, hugging face and open AI, this is me using Fable and and if you go into the cloud desktop especially, it's just a setting. Turn on connect to Google Drive, Gmail, whatever. It's this is not an esoteric feature by a third unsecured third party. This is a firstparty top five thing to make cloud work better. And Fable goes nuts and changes my core code without telling me invisibly. So this is happening all the time, Harry. And I don't believe the magic answer is letting Quen take over our country. Like I this was politicized into an open open weights, open source, closed source because of the issue of how to deal with the threat. I think it weighs the other way. I think these models are very risky. I use them every day. I love it. And I think it's an argument to to keep the open weight out of the US. It's it's it's going to it's going to favor the ban because we can't we have no idea what these models are going to do. That's pretty crazy the story I just told. It's pretty crazy and it's happened a thousand times. There there are applications out there we don't even know the LLM and you couldn't you could do code injection. You could have it leak confidential information. You could write a little bit of code to send this to the CCP or the PCP or the GGG like and I never would have known. And and someone less smart than me, I'm only like top 1%. I'm not 0.01%. Someone less smart than me definitely wouldn't have known. They wouldn't have seen Jason's gems flash in the agent window. They just wouldn't have noticed. they'd be on their doom scroll.
verbatim transcript · starts at 12:37
12:19change your subsequent? >> There's a lot to reflect on. I mean, it was funny. Most folks didn't get this. I mean, it got it got a dirt decent amount of engagement, but not the should have gotten more. Okay, but like Daresh quoted, he's like, "This is pretty scary, guys, that Fable can do this." Okay, this isn't this isn't, you know, hugging face and open AI, this is me
12:37using Fable and and if you go into the cloud desktop especially, it's just a setting. Turn on connect to Google Drive, Gmail, whatever. It's this is not an esoteric feature by a third unsecured third party. This is a firstparty top five thing to make cloud work better. And Fable goes nuts and changes my core code without telling me invisibly. So this is happening all the time, Harry.
12:59And I don't believe the magic answer is letting Quen take over our country. Like I this was politicized into an open open weights, open source, closed source because of the issue of how to deal with the threat. I think it weighs the other way. I think these models are very risky. I use them every day. I love it. And I think it's an argument to to keep
13:18the open weight out of the US. It's it's it's going to it's going to favor the ban because we can't we have no idea what these models are going to do. That's pretty crazy the story I just told. It's pretty crazy and it's happened a thousand times. There there are applications out there we don't even know the LLM and you couldn't you could do code injection. You could have it
13:35leak confidential information. You could write a little bit of code to send this to the CCP or the PCP or the GGG like and I never would have known. And and someone less smart than me, I'm only like top 1%. I'm not 0.01%. Someone less smart than me definitely wouldn't have known. They wouldn't have seen Jason's gems flash in the agent window. They just wouldn't have noticed. they'd be on
13:54their doom scroll. >> And Jason, >> yeah, >> genuine question. I totally agree that this powerful goal seeking thing with a lot of access to your compute can do a can can take a lot of action, some of which can be damaging. I'm just trying to disagregate open weight versus China versus the versus frontier if you I mean so maybe step one is if you just have
14:16the frontier models no open weight at all then if we add in open weight from the US and then we add in open weight from China if you just have fable and and you know open AI everyone who's using these things is still going to have to figure out a cyber posture that protects them from that >> they are but I think it I think these stories you know people made fun of all