ATRIUMsearch → argument graph
MechanismVideo · 16:22 — 21:44

What China regulates is the AI service offered to the public, not the open-weight model itself, because giant models are too hard to run on one's own — and both this services-based view and the American worst-case-focused view are valid.

Nathan explains a key divergence: China regulates AI mostly at the service level rather than the raw model, reasoning that open-weight frontier models are too large to run in isolation and are ultimately used inside regulated services — a view he finds legitimate, even as he holds that American worst-case analysis is also essential. ✦ AI generated

Nathan Labenz · The Cognitive Revolution · 2026-08-02 · original ↗

starts at this moment · 16:22

My broad sense is that what is regulated in China is a service. You are offering an AI service to the public. That is the kind of thing that gets regulated. ... it's out there as open weights but it's not like any random deranged person can download it to a phone or a laptop do harm with it it's really going to be used in the context of other services and so we should look more at like the context in which that model is ultimately used than just the model itself in isolation.

verbatim transcript · starts at 16:22

Transcript · around this moment

16:22That's claude.ai/TCR. And check out Claude Pro, which includes all of the features mentioned in today's episode. That's claude.ai/TCR. There's one thing that is worth keeping in mind for multiple reasons here which is that sometimes I think there is a little bit of confusion between the openw weight model that a Chinese company might release and the actual service that it provides to the public. I talked last time in my review of the

16:59Chinese AIS about how sometimes within the Chinese apps, I experienced this on multiple different Chinese AI apps. If you ask a sensitive question, you will sometimes see that the answer is coming in and then you'll have that old school Bing experience where all of a sudden the answer that you were starting to read disappears and you get refusal. Sorry, I can't help you with that. So clearly this is a multi-art system,

17:26right? It is not just the model. It's a model, but then it's also some monitor that sits on top of the model that's classifying or otherwise reviewing the output and can come down and say, "Nope, we're going to cut you off right there, even though the model itself was happy to answer." So I bring this up because I think this can sometimes muddle the results and make

17:49the differences look a little more stark than they are. I did go into the methodology for the Concordia report and they did say that wherever possible they are testing the API direct from the company. So whatever systems they have in the API would be included. Do they have the same systems in the API as they do in their first party consumer app? not always clear. And then in other

18:13contexts when you see results like this like I think when Anthropic does some of their testing and reports on the Chinese models I think they are generally if I understand correctly just testing the model itself without whatever surrounding additional measures it's deployed with in practice in the Chinese economy. Why does that matter? I mean, I think as always with these things, like both data points are interesting. The

18:42most hawkish AI safety line of thought would be like, well, you put an openw weight model out into the world, anybody can use it. So, we need to understand the worst case scenario. I think that's like totally valid and definitely is important for us to understand. And I think the Chinese might underestimate the importance of that in some ways because they tend to think a lot more

19:02about services than just the model. My broad sense is that what is regulated in China is a service. You are offering an AI service to the public. That is the kind of thing that gets regulated. You're putting a model out into the public domain. Okay. Who's going to use it and under what context? they seem to kind of have the mental model that like when a company releases an open weights

19:29model that mostly what's going to happen with that is like other businesses are going to pick it up and they're going to build their own services around it and so if that is another Chinese company building a new service around an open weights model then again that will be regulated at the service level and I think they sort of expect that like people around the world or countries

Related moments