Video · 2026-08-02 · 2h 16m · 6 moments

Nathan Goes to China – Part 2: AI Safety with Chinese Characteristics

✦ AI generated

timeline · colored by role

Claim

China does care about AI safety and has at times slowed down its AI companies in the name of safety, so the idea that 'China doesn't care and China will never slow down' is a misconception born of ignorance.

Nathan opens by rebutting the familiar 'but China' objection in AI safety discourse — the claim that the US can never regulate or slow its AI race because China doesn't care — and promises the episode will show China does care and will slow its companies when it deems it necessary.

transcript

Nathan Labenz: I think it should become clear by the end of everything that I'm about to take you through that China does care about AI safety. Not it's not the only thing that they care about certainly, but they do care. China has at times slowed down their AI companies in the name of safety. Not necessarily existential safety, but safety as they understand it, domains that they care about.

Fact

Chinese AI companies do not have as strong safeguards against misuse as American ones, but the American lead is carried almost entirely by OpenAI and Anthropic; without those two, the safety differential between the US and Chinese ecosystems would largely disappear.

Leading plainly with the uncomfortable data point, Nathan states that Chinese models currently have weaker misuse safeguards than American ones — but argues the difference is driven overwhelmingly by OpenAI and Anthropic, and that subtracting those two companies would make the US-versus-China safety picture much murkier.

transcript

Nathan Labenz: I think it is fair to say that as of now, Chinese AI companies and models do not have as strong of safeguards protecting the public against the potential for misuse of the models as the American companies do have. ... The difference I think really is driven by two companies in the United States. ... Those two companies are really doing a lot to bring up the American average. If you were to subtract those two companies from the mix and you were to look at the rest of the American companies and how they compare to the Chinese companies, honestly, a lot of the safety differential that gets reported would disappear.

Mechanism◆

The guiding principle of the Chinese AI ecosystem is the '45-degree line': capabilities and safety measures should grow together in tandem, being neither too far ahead of nor pointlessly over-engineered relative to each other.

Nathan introduces Xiao Bowen's '45-degree line' concept — that safety measures must grow step-for-step with capabilities — as the broadly accepted guiding principle in China, then cites Concordia AI's public data showing Chinese open-weight models sit below that line while mostly-American proprietary API models sit at or above it.

transcript

Nathan Labenz: the idea of the 45 degree line is that capabilities and safety measures should grow together. Right? That's the the 45 is kind of the y equals x slope of one. As your capabilities rise, your safety standards and measures also need to get stronger and stronger. And as long as those two grow in tandem and the safety measures are up to the challenge presented by the capabilities at any given level, then you're good.

Mechanism◆

What China regulates is the AI service offered to the public, not the open-weight model itself, because giant models are too hard to run on one's own — and both this services-based view and the American worst-case-focused view are valid.

Nathan explains a key divergence: China regulates AI mostly at the service level rather than the raw model, reasoning that open-weight frontier models are too large to run in isolation and are ultimately used inside regulated services — a view he finds legitimate, even as he holds that American worst-case analysis is also essential.

transcript

Nathan Labenz: My broad sense is that what is regulated in China is a service. You are offering an AI service to the public. That is the kind of thing that gets regulated. ... it's out there as open weights but it's not like any random deranged person can download it to a phone or a laptop do harm with it it's really going to be used in the context of other services and so we should look more at like the context in which that model is ultimately used than just the model itself in isolation.

Context

President Xi Jinping's opening keynote at WIC shows Chinese leadership is genuinely earnest about AI safety — calling for safeguards against loss of control, risk-awareness, and keeping AI under human control — and it reads as more AI-safety-hawkish than any comparable statement from a prominent American politician.

Drawing on carefully-prepared translated quotes from the opening 20 minutes of WIC, Nathan argues President Xi's framing — human coexistence with thinking machines, calibrated governance, safeguards against loss of control, and keeping AI under human control — is strikingly aligned with American AI safety concerns and exceeds anything a US politician has said, undercutting claims that China will never take these issues seriously.

transcript

Nathan Labenz: The faster AI advances, the more firmly its direction must be anchored toward human benefit, the more precisely governance must be calibrated, and the more rapidly safeguards against loss of control must improve. And then he closed by saying that countries should strengthen risk awareness, confront AI's inherent and downstream risks, build legal technical monitoring, early warning and emergency response systems, prevent misuse and malicious use, and keep AI under human control.

Anecdote

There is no such thing yet as 'alignment with Chinese characteristics' — nobody in China appears to be working on mapping the durable Confucian tradition onto AI alignment the way the Claude Constitution imagines a model growing into a virtue ethicist, and that is a genuine green-field opportunity.

In the episode's closest hinge, Nathan hunts for a Confucian-equivalent of the Claude Constitution and finds none — prompting one professor to explain that his generation of Chinese engineers is the weakest on traditional philosophy — leaving what Nathan frames as an exciting, unexplored opportunity to build future AI characters on plural wisdom traditions.

transcript

Nathan Labenz: One professor told me when I asked him about this, he said, 'It's an interesting idea, but we are probably the generation in all of Chinese history that is the weakest on this traditional philosophy.' He said, 'You got to remember, we're all engineers.' ... the AI safety community in China is much more on the open AI side of the cordability versus character debate. They're about having clear rules, having the AI follow those rules, trying to make that as reliable and consistent as possible.

Highlight slides

The '45-Degree Line' Guiding Principle✦ from: The guiding principle of the Chinese AI ecosystem is the '45-degree line': capabilities and safety measures should grow together in tandem, being neither too far ahead of nor pointlessly over-engineered relative to each other.Where Mainstream Models Sit on the Line✦ from: The guiding principle of the Chinese AI ecosystem is the '45-degree line': capabilities and safety measures should grow together in tandem, being neither too far ahead of nor pointlessly over-engineered relative to each other.China regulates the AI service, not the model✦ from: What China regulates is the AI service offered to the public, not the open-weight model itself, because giant models are too hard to run on one's own — and both this services-based view and the American worst-case-focused view are valid.Two valid, divergent regulatory views✦ from: What China regulates is the AI service offered to the public, not the open-weight model itself, because giant models are too hard to run on one's own — and both this services-based view and the American worst-case-focused view are valid.

Related episodes

Article · Interconnects

Kimi K3: The open-weights escalation

→

Video · All-In Podcast

OpenAI vs Anthropic IPOs, Anthropic $3T, Zuck's Price War, China Ends Open Source?, Trump Accounts

→

Video · 20VC

Jensen's Open-Weights Letter | Google Cloud Grows 82% But The Market Tanks

→