Video · 2026-05-20 · 1h 17m · 30 moments

Intelligence is collective, not artificial — Prof. Michael I. Jordan (UC Berkeley / Inria)

✦ AI generated

GUEST Michael Jordan GUEST Michael I. Jordan HOST Tim Scarfe

timeline · colored by role

Claim◆

Anthropomorphizing AI systems with words like 'understand' and 'intelligence' is unnecessary, inappropriate, and a distraction for most real problems — it belongs to science fiction, not engineering.

Jordan argues that applying words like 'understand' and 'intelligence' to AI systems is an unnecessary, distracting anthropomorphism that belongs to science fiction rather than engineering.

transcript

Michael Jordan: I don't think we need to see I think this anthropomorphizing of intelligence and understanding all that is not necessary, not appropriate and is is a distraction for many many problems. why say it understands. I think it's science fiction and I think science fiction is important for society but it's also at the level it's being promoted... it's really hurting 25 and 20 year olds.

supports · 1

Claim◆

It's so demoralizing when thought leaders who built gradient descent algorithms without understanding intelligence now tell young technologists that AI will either wipe out humanity or bring superintelligence so soon there's nothing left for them to do.

Jordan argues that AI 'thought leaders' declaring the field either dangerously close to wiping out humanity or about to produce superintelligence is deeply demoralizing for 20- and 25-year-olds who want to build useful technology.

transcript

Michael I. Jordan: they're kind of being told by the leaders, well, we had our fun and we developed a bunch of algorithms. We we did it and we were just interested in the pure, you know, understand intelligence even though they didn't understand intelligence. They built gradient descent algorithms and now you guys, you can't do this because it's dangerous.

Claim

AGI is a PR term that distorts research priorities, confuses young people, and manufactures false grandiosity rather than describing a rigorous scientific concept.

Jordan dismisses 'AGI' as a hyped PR term rather than a scientific concept, arguing it distorts young researchers' understanding of the field.

transcript

Michael I. Jordan: AGI to me is just a bit of it's it's a PR term. Uh, and it it's uh some people think it's it's fun because you have to have these great aspirations. I think it's just distortion. I think it confuses young people.

extends · 1

Claim

AGI is essentially a PR term rather than a rigorous concept, and its promotion distorts research priorities and confuses young people about what the technology actually is.

Jordan dismisses 'AGI' as marketing language rather than science, arguing it misleads young people about the field's real aspirations and progress.

transcript

Michael I. Jordan: AGI to me is just a bit of it's a PR term. And some people think it's fun because you have to have these great aspirations. I think it's just distortion. I think it confuses young people.

rebuts · 1

Mechanism

Assuming that scaling LLMs into multi-agent systems will automatically yield sound economic behavior is naive engineering, comparable to chemical engineers in the 1940s throwing substances together without underlying theory and causing explosions and harm.

Jordan rejects the Silicon Valley notion that multi-agent LLMs will automatically produce good economic outcomes, likening it to unprincipled early chemical engineering that caused real harm.

transcript

Michael I. Jordan: I mean, it's just not a good way to think about engineering. I mean, if you were a chemical engineer back in the 40s and 50s saying we're just going to throw a lot of stuff together and make it work. Well, you could do it, but you'd get a lot of explosions and a lot of economically nonviable things. You'd hurt a lot of people.

Claim

Assuming that stitching LLMs into multi-agent systems will automatically yield sound economic behavior 'for free' is bad engineering, comparable to 1940s chemical engineers throwing substances together without theory and causing explosions and harm.

Jordan criticizes Silicon Valley figures like Ilya Sutskever for assuming multi-agent LLM systems inherit good economic properties automatically, arguing this ignores real harms (citing Facebook and teen mental health) and lacks the rigor of a genuine engineering discipline.

transcript

Michael I. Jordan: It's just not a good way to think about engineering. I mean, if you were a chemical engineer back in the 40s and 50s saying we're just going to throw a lot of stuff together and make it work — well, you could do it, but you'd get a lot of explosions and a lot of economically nonviable things. You'd hurt a lot of people.

Claim

Assuming that turning LLMs into multi-agent systems automatically gives you sound economic behavior 'for free' is bad engineering, comparable to 1940s chemical engineers throwing substances together without theory and causing explosions and harm.

Jordan rejects the Silicon Valley idea that stitching LLMs into multi-agent systems will automatically produce good economic outcomes, likening it to unprincipled early chemical engineering that caused real harm.

transcript

Claim

Simply combining LLMs into multi-agent systems will not automatically produce sound economic behavior 'for free' — without deliberate economic engineering, it's like 1940s chemical engineers throwing substances together and getting explosions and harm rather than viable systems.

Jordan rejects the Silicon Valley notion that turning LLMs into multi-agent systems yields good economic properties automatically, comparing it to reckless early chemical engineering that caused explosions and harm before proper theory existed.

transcript

Michael Jordan: I mean, it's just not a good way to think about engineering. I mean, if you were a chemical engineer back in the 40s and 50s saying we're just going to throw a lot of stuff together and make it work. Well, you could do it, but you'd get a lot of explosions and a lot of economically nonviable things. You'd hurt a lot of people.

provides context · 1

Claim

Assuming that turning LLMs into multi-agent systems will automatically produce sound economic/behavioral outcomes 'for free' is bad engineering, comparable to 1940s chemical engineers throwing substances together without theory and causing explosions and harm.

Jordan rejects the Silicon Valley notion that multi-agent LLM systems automatically inherit sound economics, comparing it to reckless early chemical engineering that caused real harm, and points to Facebook's damage to teenagers as evidence no one is thinking this through.

transcript

Michael I. Jordan: It's just not a good way to think about engineering. I mean, if you were a chemical engineer back in the 40s and 50s saying we're just going to throw a lot of stuff together and make it work. Well, you could do it, but you'd get a lot of explosions and a lot of economically nonviable things. You'd hurt a lot of people.

Mechanism

Foundation models like AlphaFold are highly accurate overall but give badly overconfident, biased predictions on edge-of-knowledge questions with little training data; merging a small amount of ground-truth data via 'prediction-powered inference' can correct the error bars while preserving statistical power.

Using AlphaFold as an example, Jordan describes finding that its predictions on rare, edge-of-knowledge questions (quantum fluctuations and phosphorylation) were narrowly overconfident and far from the true value, and describes 'prediction-powered inference,' a method his group developed to fix this by blending in a bit of ground-truth data.

transcript

Michael Jordan: What if I add a little bit of ground truth data to the 200 million? Can I shift the error bar so it stays somewhat narrow? So I have high power, but it covers the truth. And the answer is yeah, there's a methodology. We've developed something called prediction powered inference that does exactly that. And so it'll cover the truth just like in a classical statistical setting, but it's using this rather highly biased architecture.

Example

AlphaFold's predictions are narrow and highly biased precisely on the novel, edge-of-knowledge scientific questions researchers most want to ask, but adding a small amount of ground-truth data via 'prediction powered inference' can correct the error bars while preserving statistical power.

Jordan describes empirical work showing AlphaFold's confidence intervals were narrow but far from the true value on understudied questions (e.g., quantum fluctuations and phosphorylation), and presents 'prediction powered inference' as a fix that blends model output with a little real data.

transcript

Michael I. Jordan: So now I have a good statistical question. What if I add a little bit of ground truth data to the 200 million? Can I shift the error bar so it stays somewhat narrow? So I have high power, but it covers the truth. And the answer is yeah, there's a methodology. We've developed something called prediction powered inference that does exactly that.

Mechanism

AlphaFold-style foundation models can be made statistically trustworthy on questions at the edge of knowledge by merging a small amount of ground-truth data with the model's biased predictions, via a method called prediction-powered inference, rather than assuming the bias will just disappear with more data.

Analyzing AlphaFold's 200 million predicted structures, Jordan found the model gives overconfident, biased answers on understudied questions (e.g. quantum fluctuations and phosphorylation), but developed 'prediction-powered inference' to correct this by blending in small amounts of ground-truth data.

transcript

Michael I. Jordan: What if I add a little bit of ground truth data to the 200 million? Can I shift the error bar so it stays somewhat narrow? So I have high power, but it covers the truth. And the answer is yeah, there's a methodology. We've developed something called prediction powered inference that does exactly that.

Mechanism

Foundation models like AlphaFold are systematically biased and overconfident precisely on the novel, edge-of-knowledge questions scientists actually care about, but merging a small amount of ground-truth data via 'prediction powered inference' can correct the error bars while retaining statistical power.

Jordan describes empirical work showing AlphaFold's confidence intervals are narrow but wrong for understudied phenomena like quantum fluctuations tied to phosphorylation, and explains 'prediction powered inference' as a fix that blends foundation-model output with small amounts of ground truth.

transcript

Mechanism

Foundation models like AlphaFold produce confidently biased predictions on novel questions at the edge of knowledge because training data there is sparse, so they must be paired with statistical methods (like prediction-powered inference) that merge in ground-truth data to yield trustworthy uncertainty estimates.

Using AlphaFold as a case study, Jordan shows foundation models are systematically biased on edge-of-knowledge questions and argues ground-truth data must be statistically merged in to correct this.

transcript

Michael I. Jordan: There needs to be around any foundation model the ability to maybe collect a bit of ground truth data to merge it in with some procedure like this and then to give out a more trustable answer. That's all not science fiction. that's what can be done and what really needs to be done and I'm sure the AlphaFold people are on board with that.

rebuts · 1

Claim◆

Anthropomorphizing AI systems by asking whether they 'understand' is unnecessary, inappropriate, and a distraction — what matters is that a system's input-output behavior is predictable and useful, not whether it possesses understanding.

Pushing back on the idea that AlphaFold's iterative refinement process amounts to 'understanding,' Jordan argues that anthropomorphizing AI with words like understanding and intelligence distracts from the real engineering questions.

transcript

Michael I. Jordan: I don't think we need to see I think this anthropomorphizing of intelligence and understanding all that is not necessary, not appropriate and is is a distraction for many many problems. why say it understands.

rebuts · 1supports · 1

Claim

Anthropomorphizing AI systems with words like 'understands' or 'intelligence' is unnecessary, inappropriate, and a distraction from solving real problems.

Jordan argues that applying human cognitive language like 'understanding' to systems like AlphaFold is an unnecessary anthropomorphism that distracts from the real engineering and scientific work.

transcript

rebuts · 1

Claim

Anthropomorphizing intelligence and asking whether a system 'understands' is unnecessary, inappropriate, and a distraction — what matters is whether the predictive system is safely embedded in a larger engineering ecosystem, not whether anyone can peer inside it.

Jordan argues that demanding AI 'understand' things (as with AlphaFold or Amazon's supply-chain models) is a category error; what's needed instead is engineering context — inputs, outputs, and constraints — not interpretability of an internal 'understanding'.

transcript

Michael I. Jordan: You cannot there's no way that any human can understand what's happening in that big big box. Um but it's not necessary. And in fact you can ask does that overall system understand you know transport and logistics and the answer is who cares. It does a very important optimization and prediction process that allows an engineering system to be built around it.

gives example · 1rebuts · 2

Mechanism

A healthy data economy requires deliberate mechanism design — such as platforms offering users a tunable, priced level of differential privacy — rather than assuming markets or regulators will spontaneously produce a fair equilibrium.

Using a 'three-layer data market' model (users, platforms, third-party data buyers), Jordan shows that offering users a tunable, priced level of differential privacy creates feedback loops that can be analyzed and optimized as an equilibrium problem, rather than left to chance.

transcript

Michael I. Jordan: In an effective economic system what would happen is that the platforms would say well we'll offer you a tunable level of differential privacy for some cost... the user looks at that and says, Ah, 7, that's better. I really care about my privacy, so I'll go there. That company then will get start to get more data and their service will get even better.

Example

In a three-layer data market (users, platforms, third-party data buyers), selling user data shifts the equilibrium because users lose privacy; the fix is economic mechanisms like tunable, priced levels of differential privacy rather than waiting for regulators to intervene.

Jordan walks through a 'three-layer data market' model showing how selling user data to third parties destabilizes equilibrium, and proposes priced, tunable differential-privacy levels as a market-based fix.

transcript

Michael I. Jordan: In an effective economic system what would happen is that the uh you wouldn't just wait for the regulator to come in the government say you know no this can't be done what you would do is that the platforms would say well we'll offer you uh a tunable level of differential privacy for some cost or we'll just say that this our company I'm Google I'll offer you level you know.3 and some other company says well I'll offer you level 7.

Example

In a three-layer data market (user, platform, third-party data buyer), platforms can offer tunable levels of differential privacy at a cost, and this creates a shifting equilibrium where higher user privacy improves platform data collection but lowers the data's resale value to third-party buyers.

Jordan walks through his 'three-layer data market' model, showing how offering tunable differential privacy creates competing incentive shifts between users, platforms, and third-party data buyers that can be formalized as an equilibrium (Stackelberg game) problem, not just an optimization.

transcript

Michael I. Jordan: The platforms would say well we'll offer you a tunable level of differential privacy for some cost... the user looks at that and says, Ah, that's better, I really care about my privacy, so I'll go there. That company then will get start to get more data and their service will get even better... but now the data buyers will look at the data from that person, more noise has been added, it's less valuable to the data buyer.

Claim◆

Telling ambitious young technologists that AI will either produce a superintelligent utopia or wipe out humanity — with nothing in between — is a demoralizing false dichotomy that ignores the real, positive work still to be done.

Jordan argues that thought leaders framing AI's future purely as 'superintelligence vs. extinction' demoralize young people who want to build genuinely useful technology, and that this framing is a false binary.

transcript

Michael I. Jordan: These young folks of whom there are huge numbers are excited about technology and they want to build things that help their family and help their country. Actually, more of their family than their country, honestly. And they they they see real opportunities in doing that and they're kind of being told by the leaders, well, we had our fun. We developed a bunch of algorithms.

Claim◆

Framing AI's future as a stark choice between superintelligence and human extinction is a false and demoralizing binary that obscures the many positive, human-scale things AI could actually do.

Jordan rejects the extinction-vs-superintelligence dichotomy pushed by prominent 'thought leaders' as harmful, especially to young people who are being told there is nothing meaningful left to build.

transcript

Michael I. Jordan: Super intelligence versus extinction. Those are your two options. And god damn it those aren't the only two options. There's a huge number of very positive things that can be done at human scale. And let's hope that enough of the young mentalities kind of get behind that.

Claim◆

The framing that AI's only possible futures are utopian superintelligence or human extinction is a false, demoralizing dichotomy that obscures a wide range of positive, human-scale outcomes AI could enable.

Jordan calls the 'superintelligence versus extinction' framing pushed by prominent AI voices a false and demoralizing dichotomy, insisting there is a large space of positive, human-scale possibilities being ignored.

transcript

Michael Jordan: Super intelligence versus extinction. Those are your two options. Um and god damn it those aren't the only two options. There's a huge number of very positive things that can be done at human scale. And let's hope that enough of the young mentalities kind of get behind that.

Example

Real-world uncertainty is often not resolved by individual expected-value optimization but by population-level hedging that forms an equilibrium — as shown by ducks foraging in ratios matching resource distribution rather than all choosing the single best option.

Jordan uses a duck-foraging example to argue that good decision-making under uncertainty isn't a purely individual Bayesian calculation — real ducks hedge their foraging in proportions matching resource ratios, which is a Nash equilibrium at the population level.

transcript

Michael Jordan: The actual ducks don't do that. They go to probably 2/3 to that side of the lake and one third to the other side. They're hedging... but they're actually getting the right ratio. And the explanation is that you weren't thinking about the context of this uncertainty — if they all have that same uncertainty, then they can sample with probability 2/3, and that's actually a Nash equilibrium of the bigger system.

rebuts · 1

Example◆

Real ducks don't maximize expected value by always going to the side of the lake with more grain; instead they hedge in the exact ratio (2/3 to 1/3) that forms a Nash equilibrium across the population, showing that individual uncertainty must be reasoned about in a social/game-theoretic context rather than in isolation — a kind of contextual reasoning current LLMs lack.

Using the example of foraging ducks that hedge their choices in a ratio matching a Nash equilibrium rather than always picking the higher-expected-value option, Jordan argues true uncertainty reasoning requires social/game-theoretic context — something LLMs, which only mimic humans' stated confidence, don't actually do.

transcript

Michael I. Jordan: The actual ducks don't do that. They go to probably 2/3 to that side of the lake and one third to the other side. They're hedging... they're actually getting the right ratio. And the explanation is that you weren't thinking about the context of this uncertainty. If they all have that same uncertainty, then they can sample with probability 2/3, and that's actually a Nash equilibrium of the bigger system.

Highlight slides

Related episodes

Video · a16z Podcast

Ben Horowitz on the Global Race for Tech, Power, and Influence

Exploring the J-Space

Foundation Models Are Coming for Enterprise Databases [Jure Leskovec] - 768