MechanismAudio · 10:00 — 11:30

Word vectors represent words as points in a multi-dimensional space, enabling mathematical operations like similarity measurement and analogy that reveal unexpected relationships.

Parrish explains word vectors as a way of representing words as points in multi-dimensional space, allowing operations like measuring similarity and solving analogies (e.g., France is to Paris as Germany is to Berlin). ✦ AI generated

Allison Parrish · CoRecursive · 2019-11-16 · original ↗

Share card ↓Export .md Export .txt

plays this moment only · 10:00 — 11:30

Elicited by

“So what are word vectors?”

A word vector is exactly that. It's like a way of representing a word as a point in space of some number of dimensions... you can say like, what are all of the words that are most similar to blue? And it would give you like green and sad and purple and sky... the big finding from the famous Word2Vec paper was you could do things like analogies, like the same line that connects France and Paris, if you transpose that line to start on Germany, the line will point to Berlin.

verbatim transcript · starts at 10:00

▦ Google ✳ word2vec

Around this claim

Mechanism · 1

Semantic word vectors work by building co-occurrence matrices: words that appear in similar contexts share similar meanings, though the dimensions themselves don't have stable semantic labels.Allison Parrish · CoRecursive · conf 85%

In practice · 1

Language needs to be destabilized and cut down to size so we realize the ways in which it is arbitrary, and computers are an effective tool for doing that.Allison Parrish · CoRecursive · conf 90%

This moment responds to

gives example → Word2vec's embedding vectors capture real semantic relationships purely from word co-occurrence statistics, so that mathematically subtracting 'man' from 'king' and adding 'woman' lands closest to 'queen', without anyone teaching the model about gender or royalty.CJ · Syntax gives example → Computational tools can reveal bizarre and unusual questions about texts and the world that other techniques cannot, by letting the natural affordances of computation lead inquiry.Allison Parrish · CoRecursive gives example → Word embedding vectors trained via word2vec capture real semantic relationships, like gender and royalty, purely from word co-occurrence statistics, without anyone explicitly teaching the model those concepts.CJ · Syntax