Video · 2026-03-13 · 1h 18m · 5 moments

When AI Discovers the Next Transformer — Robert Lange

✦ AI generated

GUEST Robert Lange GUEST Tim HOST Tim Scarfe

timeline · colored by role

Claim

LLMs with evolutionary harnesses can dramatically improve scientific discovery through sample-efficient, iterative stepping-stone accumulation.

Robert Lange explains how evolutionary LLM-driven methods like Shinka Evolve cut costs and evaluation time, making scientific discovery more democratically accessible.

transcript

Robert Lange: One thing that sort of is important about sort of using all of these evolutionary LLM driven methods is sample efficiency, right? So, and many of these systems sample like let's say a thousand programs for a given task and what we tried to do with Shinka Evolve was try to essentially cut down costs as well as sort of computation evaluation time by introducing a set of sort of technical innovations to this evolutionary search

Claim◆

Current evolutionary LLM systems are limited because the problem is given as fixed — true innovation requires co-evolving the problem and the solution together.

Lange identifies the 'problem problem' — current systems optimize for a fixed task, but real breakthroughs require inventing new sub-problems and co-evolving problem and solution.

transcript

Robert Lange: Um with all of these systems so far, maybe except for the AI scientist, which we can also talk about, the problem is given, right? So, you have an evaluator, you have a correctness checker, and you sample programs only on that single problem, right? But, oftentimes innovation for a specific problem might require first inventing a different problem, right?

explains mechanism · 1extends · 1

Prediction◆

The future of research is a shepherd-ship paradigm where humans steer distributed, multi-threaded AI systems that run experiments autonomously.

Lange envisions a future where researchers act as shepherds — steering AI systems that run experiments in parallel overnight, with humans analyzing results rather than executing experiments manually.

transcript

Robert Lange: My ideal future scenario is one in which um you as a researcher sort of during the day cool work with like a system like Shinka or the AI scientist you um sort of steer the ship like a shepherd in some sense and then during the night you you you you press play and you go to bed and in the in the background you've multiple experiments running and automatically new ones being proposed by LLMs, evidence being accumulated and then in the morning you come back and sort of you have an uh multi-threaded sort of system running in parallel and you're more like the shepherd of the ship than the person actually executing experiments and analyzing.

extends · 1

Claim

The reason I'm not that worried yet about labor market disruption is I still believe deeply that humans are the source of deep understanding and creativity in the world.

Tim argues that humans remain the indispensable source of deep understanding and creativity, suggesting AI will amplify rather than replace human capabilities, at least for now.

transcript

Tim: The reason why I'm not that worried yet about labor market disruption is I still believe deeply that humans are the source of deep understanding and creativity in the world. If I didn't believe that, I would be very worried.

Prediction

One of the Rubicon moments will be when a new Transformer-scale architecture is discovered by AI and we're all using it.

Tim identifies the AI-driven discovery of a foundational architecture (like the next Transformer) as a watershed moment that would signal a fundamental shift in the relationship between human and machine innovation.

transcript

Tim: I think one of the Rubicon moments is when the the new Transformers architecture or something massive is discovered by AI and we're all using it.

extends · 1

Highlight slides

The 'Problem Problem' in Evolutionary LLM Systems✦ from: Current evolutionary LLM systems are limited because the problem is given as fixed — true innovation requires co-evolving the problem and the solution together.Key Insight: Co-Evolution of Problem and Solution✦ from: Current evolutionary LLM systems are limited because the problem is given as fixed — true innovation requires co-evolving the problem and the solution together.The Shepherd-Ship Paradigm for Research✦ from: The future of research is a shepherd-ship paradigm where humans steer distributed, multi-threaded AI systems that run experiments autonomously.Day-Night Research Workflow✦ from: The future of research is a shepherd-ship paradigm where humans steer distributed, multi-threaded AI systems that run experiments autonomously.

Related episodes

Video · The Cognitive Revolution

Intelligence on the Edge: Liquid AI's Ramin Hasani on the Search for Device-Native Foundation Models

→

Video · Syntax

I Built an LLM From Scratch

→

Video · a16z Podcast

Garry Tan: New Rules for Founders

→