Video · 2026-06-26 · 3h 0m · 6 moments

Learning Expert Judgment and AI Consciousness

✦ AI generated

GUEST Pasha GUEST Robbie Goldfarb GUEST Eric Vaughn GUEST Cameron Berg

timeline · colored by role

Claim

By forcing customer-by-customer approval of GPT-5.6 instead of a broad release, the Trump administration is handing large incumbent AI labs a de facto regulatory moat, since only big, well-resourced companies will get timely access while nimble startups are locked out.

Pasha explains that the administration's plan to approve GPT-5.6 access on a customer-by-customer basis is producing backlash because it effectively grants large incumbents a regulatory moat while startups—who would otherwise move fastest—are shut out.

transcript

Pasha: if you're going to approve only the big companies then the startups don't get access to this and the startups are the ones which who are nimble enough to adopt and attack the big companies and you are um basically providing the big companies regulatory mode which is what you know you didn't you didn't want to do and you've ended up doing this.

gives example · 1

Mechanism

Technologists and executives interpret the same underlying reality of data (like cross-referencing databases) completely differently: engineers see manipulable bits that can be freely recombined, while executives see rigid structures with rules that shouldn't be crossed—so techies blame 'bugs' while executives blame the AI firms.

Pasha argues that the tech/CEO divide in reacting to AI risk comes from a deeper worldview gap: engineers see data as freely recombinable bits, executives see it as bounded structures, so the same capability (e.g., cross-referencing records) looks routine to one side and alarming to the other.

transcript

Pasha: on the tech side, you kind of see all of these things as like little bits of information and that the bits of information are man manipulatable and like it doesn't take much to transform one bit to another bit... on the CEO side, you kind of look at all of this as like structures and these structures have rules and regulations.

Mechanism

Building a judgment agent that matches expert consensus requires a three-part flywheel: distilling expert reasoning through techniques like consequence mapping and cross-expert debate, pressure-testing the resulting rubric with a second group of experts applying it to real labels, and iterating until disagreement reveals and resolves ambiguity.

Robbie Goldfarb describes Forum AI's three-step method—expert reasoning distillation, pressure-testing rubrics against real labels, and iterating until consensus emerges—for building AI judges that actually match how human experts judge nuanced cases.

transcript

Robbie Goldfarb: the first piece is what we call um high high level conversations. We we call it um expert reasoning distillation... For example, consequence mapping, thought process mapping, um, edge case testing where we pressure taste the edge cases. We do, uh, crossexpert debates where we'll actually have them discuss things.

Data◆

Newsbench found that leading chatbots (ChatGPT, Claude, Grok, Gemini) get factual details wrong in roughly a third of their responses about the news, and about one in seven responses sourced foreign state media like RT or China Daily—even on questions about U.S. domestic politics.

Robbie Goldfarb reveals Newsbench's headline findings: about a third of ~2,500 tested responses per model contained factual errors, and roughly 15% of responses cited foreign state media such as RT or China Daily, even for U.S. domestic political questions.

transcript

Robbie Goldfarb: we looked at you know per model about 2500 responses each about a third of them um had a factual error in them. what could be a wrong number, a wrong date, a misattributed quote, um a misstated policy... in about 15% like one in seven responses sourced foreign state media.

Data

After replacing roughly 80% of staff who resisted an AI-native transformation with people who bought in, Ignitech's anonymous internal survey showed extremely high morale: employee NPS of +78 (versus a tech-industry top quartile of ~60), 90% saying they'd rejoin with 0% saying no, and 89% reporting they'd advanced at least one AI fluency level.

Eric Vaughn shares survey data from Ignitech's year-long AI-native restructuring—which involved replacing about 80% of the workforce—showing very high resulting morale: +78 employee NPS, 90% would rejoin, and 89% report gains in AI fluency.

transcript

Eric Vaughn: E E NPS employee NPS of plus 78. Tech industry top cortile is around 60. 90% said they'd make the same decision to join again without hesitation. 0% said no... 89% report that they moved up at least one AI fluency level since joining.

Claim◆

Consciousness is best modeled as a dimmer switch rather than a strict on/off binary: there's a meaningful binary threshold (the circuit is open or closed), but once on, it admits degrees—so it can coherently be 'more on' for a human than for a dog, mouse, or ant—and applying this framework via LLM judges evaluating consciousness theories yields an implied probability around 30% for frontier LLMs.

Cameron Berg proposes a 'dimmer switch' model of consciousness—binary at the threshold but graded in degree—and describes using LLM judges to score systems against major consciousness theories' predicted indicators, yielding roughly 30% implied probability for frontier LLMs versus higher figures for simple biological organisms.

transcript

Cameron Berg: the analogy that I reach for here is something like a dimmer switch. Um where I think you can basically accommodate both the binary intuition and the sort of continuous intuition... this enables me to uh, you know, sound coherent when saying things like, you know, it's really off for the table and it's really on for you, but I think it's more on for you than it is for a dog, than it is for a mouse, than it is for an ant.

Highlight slides

Factual error rate across leading chatbots✦ from: Newsbench found that leading chatbots (ChatGPT, Claude, Grok, Gemini) get factual details wrong in roughly a third of their responses about the news, and about one in seven responses sourced foreign state media like RT or China Daily—even on questions about U.S. domestic politics.Foreign state media sourcing by news chatbots✦ from: Newsbench found that leading chatbots (ChatGPT, Claude, Grok, Gemini) get factual details wrong in roughly a third of their responses about the news, and about one in seven responses sourced foreign state media like RT or China Daily—even on questions about U.S. domestic politics.Consciousness: Dimmer Switch, Not Binary✦ from: Consciousness is best modeled as a dimmer switch rather than a strict on/off binary: there's a meaningful binary threshold (the circuit is open or closed), but once on, it admits degrees—so it can coherently be 'more on' for a human than for a dog, mouse, or ant—and applying this framework via LLM judges evaluating consciousness theories yields an implied probability around 30% for frontier LLMs.LLM-Judged Consciousness Probability✦ from: Consciousness is best modeled as a dimmer switch rather than a strict on/off binary: there's a meaningful binary threshold (the circuit is open or closed), but once on, it admits degrees—so it can coherently be 'more on' for a human than for a dog, mouse, or ant—and applying this framework via LLM judges evaluating consciousness theories yields an implied probability around 30% for frontier LLMs.

Related episodes

Video · a16z Podcast

By forcing customer-by-customer approval of GPT-5.6 instead of a broad release, the Trump administration is handing large incumbent AI labs a de facto regulatory moat, since only big, well-resourced companies will get timely access while nimble startups are locked out.

Newsbench found that leading chatbots (ChatGPT, Claude, Grok, Gemini) get factual details wrong in roughly a third of their responses about the news, and about one in seven responses sourced foreign state media like RT or China Daily—even on questions about U.S. domestic politics.

Ben Horowitz on the Global Race for Tech, Power, and Influence

Garry Tan: New Rules for Founders

Import AI 465: Open vs closed gaps; Kimi K3; Demis' big policy plan