Video · 2026-01-31 · 4h 25m · 6 moments

State of AI in 2026: LLMs, Coding, Scaling Laws, China, Agents, GPUs, AGI | Lex Fridman Podcast #490

✦ AI generated

GUEST Nathan Lambert GUEST Sebastian Raschka GUEST Lex Fridman

timeline · colored by role

Claim

No single company will hold exclusive access to AI technology because researchers constantly move between labs, so the real differentiator between AI companies will be budget and hardware, not proprietary ideas.

Sebastian argues that because ML researchers frequently switch jobs between labs, no company can keep a durable technological edge — competition will instead be won on compute budget and hardware access.

transcript

Sebastian Raschka: One thing I know for sure is that I don't think nowadays, in 2026, that there will be any company that has access to technology that no other company has access to. That is mainly because researchers are frequently changing jobs and labs. They rotate. I don't think there will be a clear winner in terms of technology access.

Context

Chinese AI companies release open-weight models as a deliberate strategy to gain global influence and market share, since Western institutions won't pay for API subscriptions from Chinese providers over security concerns.

Nathan explains that Chinese labs open-weight their models because US enterprises won't subscribe to Chinese-hosted APIs for security reasons, so distributing free weights is how they buy into the global AI market.

transcript

Nathan Lambert: They're smart and realize the same constraints: a lot of top US tech companies and other IT companies won't pay for an API subscription to Chinese companies for security concerns. This has been a long-standing habit in tech, and the people at these companies then see open weight models as an ability to influence and take part of a huge growing AI expenditure market in the US.

gives example · 1

Prediction◆

Pre-training scaling laws are fundamentally unlikely to stop working since they have held across 13 orders of magnitude of compute already; the real constraint will be the practical difficulty of testing ever-larger scales.

Nathan argues that pre-training scaling laws show no sign of breaking down after 13 orders of magnitude of compute, and expects them to keep holding — the bottleneck will be the logistics of testing bigger scales, not the underlying law.

transcript

Nathan Lambert: It's held for 13 orders of magnitude of compute, why would it ever end? So I think fundamentally it is pretty unlikely to stop, it's just eventually we're not even gonna be able to test the bigger scales because of all the problems that come with more compute.

Mechanism

RLVR trains language models by having them attempt verifiable tasks like math or code, using measured accuracy on those tasks as the reward signal for reinforcement learning updates.

Nathan lays out the core mechanism of Reinforcement Learning with Verifiable Rewards: the model generates completions to problems with checkable answers, and correctness becomes the reward that drives training.

transcript

Nathan Lambert: Reinforcement learning is classically an agent that acts in an environment, and the environment gives it a state and a reward back, and you try to maximize this reward. In the case of language models, the reward is normally accuracy on a set of verifiable tasks, whether it's math problems or coding tasks.

gives example · 2

Anecdote

A base model's accuracy on a math benchmark can jump from 15% to 50% after only 50 steps of RLVR training, showing that RL is unlocking latent pre-trained knowledge rather than teaching genuinely new mathematical understanding.

Sebastian recounts training Qwen 3's base model with RLVR on MATH-500 and watching accuracy leap from 15% to 50% in minutes, arguing this proves RL mainly unlocks pre-existing knowledge rather than instilling new understanding.

transcript

Sebastian Raschka: I was training the Qwen 3 base model with RLVR on MATH-500. The base model had an accuracy of about 15%. Just 50 steps, like in a few minutes with RLVR, the model went from 15% to 50% accuracy.

gives example · 1

Claim◆

The most significant but underappreciated impact of LLMs is making virtually all of human knowledge instantly and reliably accessible to anyone in the world, a bigger shift than search engines ever were.

Lex argues that beyond flashy capabilities, the quiet but enormous impact of LLMs is democratizing access to human knowledge globally — a bigger leap than Google Search — with effects that will compound across generations.

transcript

Lex Fridman: One thing that is hard to articulate is the huge difference between Google Search and an LLM. I feel like I can basically ask an LLM anything and get an answer, and it's doing less and less hallucination. And that means understanding my own life, figuring out a career trajectory, solving the problems all around me, learning about anything through human history.

Highlight slides

Scaling Laws Aren't Breaking Down✦ from: Pre-training scaling laws are fundamentally unlikely to stop working since they have held across 13 orders of magnitude of compute already; the real constraint will be the practical difficulty of testing ever-larger scales.The Real Constraint: Testing at Scale✦ from: Pre-training scaling laws are fundamentally unlikely to stop working since they have held across 13 orders of magnitude of compute already; the real constraint will be the practical difficulty of testing ever-larger scales.LLMs vs. Search: A Bigger Leap✦ from: The most significant but underappreciated impact of LLMs is making virtually all of human knowledge instantly and reliably accessible to anyone in the world, a bigger shift than search engines ever were.Everyday Impact✦ from: The most significant but underappreciated impact of LLMs is making virtually all of human knowledge instantly and reliably accessible to anyone in the world, a bigger shift than search engines ever were.

Related episodes

Article · Interconnects

Kimi K3: The open-weights escalation

→

Video · a16z Podcast

Ben Horowitz on the Global Race for Tech, Power, and Influence

→

Article · Big Technology

Everything You Need to Know About Kimi K3, the Latest Model From China to Shake Up AI

→