Article · 2026-08-15 · 3 moments

EP222: What is Google’s TPU?

A TPU (Tensor Processing Unit) is Google’s custom AI chip, designed from scratch for the giant matrix multiplications that modern models live on. GPUs were built for graphics first. ✦ AI generated

Read original ↗

GUEST system-design-refresh

Definition◆

A TPU is Google's custom AI chip designed from scratch for the giant matrix multiplications that modern models live on, unlike GPUs which were built for graphics first

TPUs are purpose-built AI chips for matrix math in modern models, distinct from GPUs which originated for graphics workloads.

transcript

system-design-refresh: A TPU (Tensor Processing Unit) is Google's custom AI chip, designed from scratch for the giant matrix multiplications that modern models live on. GPUs were built for graphics first. TPUs were built for deep learning from day one.

Fact◆

Google's 8th generation TPU ships in two flavors: TPU 8t for training (raw throughput) and TPU 8i for inference (latency and chip-to-chip speed)

Google's 8th gen TPU introduces two purpose-built variants: TPU 8t optimized for training throughput and TPU 8i optimized for inference latency.

transcript

system-design-refresh: At Cloud Next '26, Google unveiled its 8th generation, and for the first time it ships in two flavors. TPU 8t is built for training, where raw throughput wins. TPU 8i is built for inference, where latency and chip-to-chip speed matter most.

Highlight slides

TPUs: Built for AI from day one✦ from: A TPU is Google's custom AI chip designed from scratch for the giant matrix multiplications that modern models live on, unlike GPUs which were built for graphics first Google's 8th Gen TPU: Two Purpose-Built Flavors✦ from: Google's 8th generation TPU ships in two flavors: TPU 8t for training (raw throughput) and TPU 8i for inference (latency and chip-to-chip speed)

Related episodes

Video · The Cognitive Revolution

Intelligence on the Edge: Liquid AI's Ramin Hasani on the Search for Device-Native Foundation Models

→

Video · The Cognitive Revolution

Fable's Back, AI Engineer Recap, & SambaNova

→

Video · Syntax

I Built an LLM From Scratch

→

A TPU is Google's custom AI chip designed from scratch for the giant matrix multiplications that modern models live on, unlike GPUs which were built for graphics first

Both TPU 8t and TPU 8i share the same Axion CPUs, liquid cooling, and software stack, so code written for one runs on the other

Google's 8th generation TPU ships in two flavors: TPU 8t for training (raw throughput) and TPU 8i for inference (latency and chip-to-chip speed)

Intelligence on the Edge: Liquid AI's Ramin Hasani on the Search for Device-Native Foundation Models

Fable's Back, AI Engineer Recap, & SambaNova

I Built an LLM From Scratch