Language needs to be destabilized and cut down to size so we realize the ways in which it is arbitrary, and computers are an effective tool for doing that.

(00:00:00) Computer programming is beautiful and useless. (00:00:02) That's the reason that you should want to do it. (00:00:04) It's not because it's meant to get you a job, not because it has a particular utility, but simply for the same reasons that you would pick up oil paints or do origami or something. (00:00:15) It's something that has like an inherent beauty to it that is worthy of studying. (00:00:22) Hello, this is Adam Gordon-Bell. (00:00:24) Join me as I learn about building software. (00:00:30) This is Code Recursive. (00:00:33) That was Allison Parrish. (00:00:35) I've been trying to get her on the podcast for a while. (00:00:38) She is a teacher at NYU. (00:00:40) She teaches a class called Computational Approaches to Narrative. (00:00:43) She's also a game maker. (00:00:44) She's a computer programmer, and she's a poet. (00:00:46) Her poetry is generative. (00:00:48) She uses the tools of NLP and computers to create her poetry. (00:00:53) So today we talk about word vectors and also about poetry and art. (00:00:57) Allison was nice enough to answer my somewhat (00:01:00) blunt questions about the nature of art. (00:01:02) And she also does a couple readings of her computer-generated poetry. (00:01:06) And we end with some tips on how people can get started creating things with text. (00:01:11) Her thoughts on computer programming for just the pure beauty of it, they also really resonate with me. (00:01:17) So I hope you enjoy the interview. (00:01:24) Allison, thanks for coming on the podcast. (00:01:26) So I have this quote from you here that I think this is reason enough for me to try to get you on the podcast. (00:01:32) It says, I want to punch language in the face and I want to use a computer to do it. (00:01:38) So what does that mean? (00:01:39) I don't know. (00:01:40) I think it's pretty clear. (00:01:42) There are a couple of parts to that quote. (00:01:45) So I call myself a poet. (00:01:46) And one of the roles of poetry, in my opinion, is to sort of destabilize language or call to attention the ways that language shapes the way that we think, the ways that language enforces particular worldviews, the ways that language shapes the world, right? (00:02:09) The English language in particular has a history of being a force for particular forms of power, and that comes in lots of different forms, notably in things like movements in the United States of making English the first, or the official language of the country is one example of language being used as a platform for politics in particular kinds of projections of power. (00:02:34) that ability of language to do things, to facilitate power, the kinds of power that I'm not interested in facilitating, that's the kind of language that I want to punch in the face, right? (00:02:48) Anything about language that forces us to see the world in a particular way, maybe not with our consent, anything that limits the expressiveness of language, that's what I want to punch in the face. (00:03:01) Though with the computer part, that's a little bit more difficult to explain. (00:03:06) I think there's, among poets, there's kind of, I would say, a Luddite-ism, which I think in a lot of ways is justified. (00:03:15) Like, computers historically have also been tools that have been used to project certain forms of power and not good forms of power. (00:03:24) So I understand, like, the skepticism (00:03:28) surrounding computation and its use in the arts. (00:03:32) But, on the other hand, I like computers, I like programming. (00:03:36) That is my expressive medium. (00:03:38) When I do things intuitively, the way that I do things intuitively is through computer programming. (00:03:43) And I also think that... (00:03:45) computational thinking and the tools that it gives you for breaking the world down and thinking through processes, rethinking processes, I think that has the potential to be a really interesting artistic tool. (00:03:56) When I say I want to punch language in the face and I want to do it with computers, that's what I'm getting at. (00:04:02) I think language needs to be punched in the face. (00:04:04) It needs to be cut down to size so that we realize the ways in which it is arbitrary. (00:04:10) And I want to do it computers because that's what I like. (00:04:13) That's what I think is an effective tool, especially for the kinds of ways the language is used in this contemporary context. (00:04:21) So were you a computer programmer first who became a poet? (00:04:24) Is that your medium or how did you come to this approach? (00:04:28) I got my first computer as a Christmas present when I was five, and it came with a programming manual, and I started doing computer programming for it. (00:04:39) I've always been interested in language since I was a kid. (00:04:42) My dad gave me a copy of The Hobbit to read when I was 10, and I loved reading it. (00:04:48) And the thing that I loved especially about it was Tolkien's Inventive Languages. (00:04:53) And I think those two interests and like, you know, I was captivated with computer programming and I loved the way that Tolkien used language in his books. (00:05:01) That plus in high school, my creative writing teacher had us read Gertrude Stein, the modernist poet. (00:05:10) And so I think like those three interests like combined together to make this the inevitable outcome of my artistic practice. (00:05:18) So I wouldn't say like none of the things like came before any of the other things, just sort of like this bubbling up of all of these interests simultaneously. (00:05:25) I have the computer science degree from a really, really long time ago, but my main undergraduate degree was in linguistics. (00:05:33) And I think only more recently in the past like 5 to 10 years have I been (00:05:37) calling myself a poet, you know, like actually as like things that I would put in my bio, but my interest in language and creative uses of language comes way, way, way before that. (00:05:46) How do you combine the worlds of poetry and computer programming? (00:05:50) Mad Libs, I think, sort of count in the realm of computational language. (00:05:56) If you look at the history of computational poetics, some of the earliest projects (00:06:01) were sort of canonicalized, that became part of art history and literary history are things that superficially resemble Mad Libs. (00:06:08) Well, actually deeply resemble Mad Libs. (00:06:10) They don't superficially resemble Mad Libs. (00:06:12) They resemble Mad Libs formally, but have a very different tone. (00:06:16) Like Allison Knowles. (00:06:18) House of Dust is a poem that was generated by like a Fortran program. (00:06:23) So things like randomizing words is a really simple technique that you can do computationally, and that just adds new energy to a text. (00:06:31) But it's also a simple language model. (00:06:34) So a language model is like a statistical model that predicts what the next word in a text is going to be. (00:06:40) So something like autocomplete on your phone is an example of that, or GPT-2 is a really sophisticated example of that. (00:06:47) It's only goal is to take an existing corpus and predict the next word. (00:06:51) Tristan Sara's How to Make a DaDa's Poem is also a language model. (00:06:54) It just predicts the next word by the frequency of all of the words in the text. (00:07:00) So those techniques are sort of a number of well-known techniques, so something like a Madlib or word replacement. (00:07:07) things like language models and then more mechanical operations on language, like randomizing and stuff like that. (00:07:12) Those are all of the techniques that you have at your disposal in one form or another. (00:07:17) Does that answer help, or does that just make it work confusing? (00:07:20) Can I say both? (00:07:21) Yeah, sure. (00:07:22) That's probably the best answer, actually. (00:07:24) No, it is helpful. (00:07:25) What makes something poetry? (00:07:26) You're not going to be able to answer this in our time, but if I select randomly from a bag of words made out of an article, what makes it art? (00:07:35) That's a difficult and fraught question. (00:07:38) I don't care if you're being kind about it. (00:07:41) With this question, it can be asked in a kind way, it can be asked in an aggressive way, but I have confidence in my techniques as an artist. (00:07:49) I think that the things that I do are art, and I'm also not attached to the label of art. (00:07:54) It's still important and interesting, even if people choose to say that it's not art, but art is literally things that you have made, right? (00:08:02) Art is the same root as an artificial. (00:08:04) It's like a thing that has been created, right? (00:08:06) So anything that meets that bare definition counts as art. (00:08:09) It's like a word that it has like connotations of being like high society or high status. (00:08:15) And that I think is kind of a useless thing to attach to it. (00:08:18) Poetry is the same way. (00:08:20) Like the word poetry has this like sort of genteel, academic, educated feeling to it. (00:08:27) And a way that you might complement a text is by saying that it's poetic, right? (00:08:32) It's like considered to be this like higher form of language. (00:08:34) That's not what I mean when I talk about poetry. (00:08:37) When I talk about poetry, I'm talking about forms of language that call attention to language's materiality. (00:08:45) They call attention to like the surface form of language rather than to just its semantics, like semantics being like the meaning, like the so-called meaning of a text. (00:08:56) There are multiple ways for me to ask you to pass me the salt shaker, right? (00:09:02) I can mean that same thing in multiple ways. (00:09:05) And I would say like narrative art forms, like the novel and things like that, call attention to the underlying story structure and like characters and plots and things like that. (00:09:16) So to contrast like narrative creative writing from poetry as a creative writing form, poetry would be forms of creative writing that (00:09:25) aren't about underlying structures of meaning necessarily, but are about what actually happens on the page when language is written, what actually happens in the mouth when language is spoken out loud, things like that. (00:09:36) I think creative language is more poetic to the extent that it calls attention to those dimensions of language by that definition. (00:09:45) There are lots of poets who don't care about that stuff, and their practice, I'm not saying that it's not poetry, but for me, when I'm talking about poetry, that's what I'm talking about. (00:09:54) The talk I saw of yours, you used word vectors, which I wasn't familiar with to do some neat things with text. (00:10:01) So what are word vectors? (00:10:03) One of the main operations that people want to do in natural language processing, and this has been the case even since like the '50s, is determine how similar two documents are or two units of language, whatever that unit happens to be. (00:10:20) or they want to be able to say, These documents are all like one another. (00:10:26) They form a cluster versus this other set of documents or units of language or whatever form a different cluster. (00:10:32) And the main mathematical operations that we have for operating on things like similarity and clustering and classes for categorization is Euclidean distance or other measures of distance between points, right? (00:10:49) So if you want to know (00:10:50) how similar two points on a Cartesian plane are, you just get the distance or the length of the line that connects those points, right? (00:11:00) And that math is the same thing if you wanted to. (00:11:03) Any data that you can represent as two spreadsheet columns, as two dimensions of data, (00:11:09) you can use that same math to figure out whether those two items are similar or whether things are in clusters or so forth. (00:11:15) So that's like the underlying statistical and mathematical need is to take phenomena in the world and represent them as points on a plane, or points in three dimensions, or points in however many dimensions there are, however many features there are of the data that you're looking at. (00:11:34) So a word vector is exactly that. (00:11:36) It's like a way of representing a word as a point in space of some number of dimensions. (00:11:42) And there are lots of ways of making word vectors. (00:11:44) Like I said, if you go back through the literature, you can find stuff from the '50s, '60s, and '70s that are trying to hand assign values to spreadsheets of words, say like, put a one in this column if it's a word for an animal, put a one in this column if it's a noun. (00:12:02) so that they can do these distance operations to do things like clustering. (00:12:06) In the more contemporary machine learning, those word vectors aren't constructed by building the data set carefully. (00:12:13) They're constructed through processes like a neural network, building a generative model and taking the calculated latent space value for each of those words, or just doing something like principal components analysis on (00:12:25) word co-occurrence matrix or something like that to give you a vector representation of that word. (00:12:30) So that you could say, for example, once you have a vector representation for the word, you can say like, what are all of the words that are most similar to blue? (00:12:38) And it would give you like green and sad and purple and sky, all of the words that occur in similar contexts. (00:12:48) And then you can do weird mathematical operations on them, like classification and categorization is one thing. (00:12:55) But the big finding from the famous Word2Vec paper from the Google researchers was you could do things like analogies, like the same line that connects France and Paris. (00:13:07) If you draw a line from France and Paris and you transpose that line to start on Germany, (00:13:13) the line will point to Berlin. (00:13:16) There's this space that gets generated. (00:13:18) They were presenting this as a finding specifically about their method of making word vectors, but I think it probably applies more generally. (00:13:24) Some of the research that I've been doing has been using much less sophisticated techniques that still show some of those properties. (00:13:30) We cast all this thing into this multi-dimensional space, and the arrow in that space represents some sort of relationship. (00:13:38) Is that what your Berlin example is? (00:13:40) Yeah. (00:13:40) Imagine on a two-dimensional plane, it's the line that connects to (00:13:43) points plus the direction, so like the direction and the magnitude of that line. (00:13:48) You can easily think of it as just like the line that connects two points. (00:13:52) That's the way that I usually explain it at least. (00:13:54) Yeah, it makes sense. (00:13:55) It's not a way I had thought about words before. (00:13:58) Right. (00:13:58) That's what's interesting. (00:14:00) Yeah. (00:14:01) talk that I saw you gave, you started off with a very simple example, which was RGB colors. (00:14:07) I think that you took each color and mapped it to an RGB value, and then you answered some strange questions about what is the average color of the book Dracula? (00:14:16) And I remember thinking, that doesn't seem like a type of question that, first of all, I would know to ask or that could have a logical answer. (00:14:23) Right, and for me, that's when I say that I want to punch language in the face with computers. (00:14:30) That's the utility of doing these things computationally, is that if you're critical about what computers can do and about where these data sets come from, you can let kind of the natural affordances of computation lead you to these unusual questions that can reveal things about the world and about texts like Dracula, but just anything in the world. (00:14:52) It can reveal things about those things that you might not have otherwise known to look for. (00:14:56) But the way that a computer program or the way that a dataset casts those phenomena, their results of particular decisions that people have made, but in that combination, in that synthesis, you can end up with these really amazing and bizarre questions and ways of looking at the world that other techniques can't do. (00:15:15) It's their worldviews and things that you notice that I think are particular to computation in some ways. (00:15:21) So are there other interesting questions you have asked of text? (00:15:26) One of the things that I talked about in that talk that's on my mind because I was preparing a little bit for this interview is a phonetic similarity between texts. (00:15:35) That's one of the big things that I've been working on in my research for the past couple of years is how to determine whether two stretches of text, whether it's a word or a sentence or a (00:15:46) line in poetry sound similar to each other. (00:15:49) And that's something like approaching that question without computation, I think would be interesting, but difficult. (00:15:55) Like the way that answering these questions might happen like back in the Middle Ages is you would just get a whole monastery of monks on the question. (00:16:04) Like if you wanted to do something like find me every verse of the Bible that has to do with clouds, you would have to like (00:16:13) seed a monk for a couple of days to go through the book and find all of the verses that have to do with clouds. (00:16:19) If I wanted to answer the question, what sounds similar to something else without a computer, I would have established criteria and then make a database of those words and then pay a grad student to go through and say, Does each of these lines match these criteria? (00:16:31) With computation, I can do it much faster and I can iterate on ideas of what sounding similar means, right? (00:16:41) And because it's kind of easy to do that, or more or less easy, it's a one-person job instead of a monastery's job, I can pick and choose my techniques and develop the technique in a way that ends up being most aesthetically pleasing to me, aesthetically pleasing because I'm a poet and an artist, and that's what I'm interested in. (00:16:59) Phonetic similarity has been the big thing lately. (00:17:01) It's the kind of thing that I don't think I would've been able to do as easily without computation. (00:17:06) It was definitely informed by the computational tools that I had at hand when I was developing algorithms to do that kind of thing. (00:17:13) Do you have wave samples of words and you map them, or how do you determine what a word sounds like? (00:17:18) If you want to know how to pronounce a word, let's say in a paper dictionary or on Wiktionary or something like that, there's the phonetic transcription that shows you how to pronounce the word. (00:17:29) look up a word like torque, for example, T-O-R-Q-E. (00:17:33) How do you pronounce that word? (00:17:34) Well, you don't sound it out letter by letter, really, because that doesn't work for those last two letters. (00:17:41) Those letters are silent. (00:17:42) How do you know that? (00:17:43) Well, you learn it in school, and if you're not sure, you look it up in the dictionary and you look at the phonetic transcription. (00:17:49) There's a big database of phonetic transcriptions for English words called the CMU Pronouncing Dictionary. (00:17:55) And I use that as a data set for a lot of this work because it's computationally readable. (00:18:00) It has the word and then also a language for transcribing the phonetics of those words. (00:18:06) My own personal view and what's interesting to me is that we can talk about sounds and qualities of sounds. (00:18:12) without resorting to audio files. (00:18:15) I can tell you this poem has a really whispery feel, or that talk was really ... (00:18:21) They used a lot of words that were really heavy, or something like that. (00:18:26) Or we can even talk about individual sounds, like the mmm sound in English, or the nasal sounds of French, or the trilled R of Spanish. (00:18:35) We don't have to resort to audio files to know what we're talking about. (00:18:39) We have a language for discussing that's not (00:18:42) purely based on recall or imitation. (00:18:45) So there's this level of thinking about language that's more about an internal mental representation of those sounds than it is about the audio itself. (00:18:56) So what do you do with that? (00:18:57) If you have this mapping of words, I mean you personally, what excites you and what can you do with this information? (00:19:03) So I use it a lot. (00:19:04) It's a file that I basically always have open on my computer in one form or another. (00:19:10) I think for the stuff that was in the talk that you watched, I was just making word vectors using the features in the CMU pronouncing dictionary as the dimensions of the spreadsheet. (00:19:25) So counting basically co-occurrence of phonemes, but not just phonemes, but underlying phoneme features as a way of basically making a spreadsheet where every word had a different vector associated with it. (00:19:40) And that gives me a really easy way to tell whether two words sound similar or whether two sentences sound similar, just based on like counting up the phonetic features from the phonetic transcription of the words that comprise that sentence or that line of poetry or that individual work. (00:19:56) Other ways that I've used it, one is just like finding words that rhyme in English, right? (00:20:02) Which sort of the naive solution to finding words that rhyme is like, well, we find words that have like the same (00:20:09) last couple of letters. (00:20:12) Here and there, those are really good examples. (00:20:15) Here and there both end with the same four letters, but they don't rhyme. (00:20:19) To know whether two words rhyme, you need a phonetic transcription, and then you can look at everything from the last stressed syllable up to the end of the word. (00:20:27) If two words share that, then they rhyme. (00:20:30) The CMU pronouncing dictionary gives you the ability to find words that look like that. (00:20:35) Are you willing to share an example of something with us? (00:20:38) One of the things that you can do with word vectors is tint a word. (00:20:44) The way that you can do that is say that I have, if we were working with semantic word vectors, like word2bec or glove, I could say, I want a word like basketball, but I want it to be basketball as an example. (00:21:04) I want a word like computer, but I want it to be a little bit more sad or something. (00:21:11) So you'd find the word for computer and the word for sad, and then basically draw a line between the two and find a point on that line and see if there are any words that are close to that point on the line. (00:21:21) And then you might end up with like, I can't think of a word that's like computer, but more sad, maybe like abacus or calculator or something like that. (00:21:31) And I call this like tinting, tinting the words. (00:21:34) With the word vectors that I've made, they're based on sound. (00:21:36) So you can tint a word by saying, I want you to find a word that's like this word, but it sounds more like some other word. (00:21:46) The words that I use as examples a lot for tinting are kiki and uba. (00:21:51) And these are nonsense words, but they've been shown in different studies, like anthropological and psychological studies, to have this sort of constant (00:22:01) emotional valence across cultures. (00:22:04) So the word kiki is perceived as sharp, and the word buba is perceived as round. (00:22:11) And people match up pictures to these words. (00:22:14) It doesn't matter where you grew up or what language you spoke when you grew up, they're almost always seen in that way. (00:22:21) You can hear it, right? (00:22:22) Like kiki sounds very crunchy or angly, I don't know. (00:22:26) Yeah, and buba seems like round, bulbous, right? (00:22:30) So there may be these universals in phonesthetics where language has particular synesthetic properties, which I think is super interesting. (00:22:37) So one of the exercises that I did is that I took Robert Frost's The Road Not Taken, which is the two roads diverged in a yellow wood. (00:22:45) I took the one that has been less traveled by and that has made a little difference. (00:22:48) Well known if you're primary or secondary education in the United States and took an English class here, it's one of canonical texts. (00:22:57) and rewrote it with the phonetic word vectors, replacing words that sound, two versions of this, one version where it's replacing every word with a word that sounds more like kiki, and then another version where it's replacing words with words that sound more like uba. (00:23:15) So we'll do a little experiment here, and I'll read all three versions. (00:23:19) So the first is without any modifications, and it goes like this. (00:23:23) Two roads diverged in a yellow wood, and sorry I could not travel both and be one traveler, long I stood and looked down one as far as I could to where it bent in the undergrowth. (00:23:34) Then took the other, as just as fair, and having perhaps the better claim because it was grassy and wanted wear. (00:23:41) Though as for that, the passing there had warned them really about the same. (00:23:45) And both that morning equally lay in leaves no step had trodden black. (00:23:49) Oh, I kept the first for another day. (00:23:52) of knowing how way leads on to way I doubted if I should ever come back. (00:23:57) I shall be telling this with a sigh somewhere ages and ages hence. (00:24:01) Two roads diverged in a wood and I, I took the one less traveled by and that has made all the difference. (00:24:09) So that's the original. (00:24:11) Thank you, Robert Frost. (00:24:13) So here is the version of that poem plus the word kiki, so tinting all of the words to sound a little bit more like kiki, and it goes, (00:24:22) Kooky roads diverged in a yellow wood pi, and sarti, I goki, pikan, kibble booth, and pi, one traveler, long I stuki, and loci down one as far as I goki, tuki, waikiki, eek, beek in the undergrowth. (00:24:35) Then kupek the other as cheeky as pichera, and having perhaps the bekikame, piki, eek was kisi and waikiki, wacky. (00:24:44) Though as for peek, the kiki and kiki, kaki warned them, ki li kubuki, the safety thing. (00:24:49) And booth peak mourning kiki lay in teeth's no tecky hacky tea garden in black. (00:24:54) Oh, I kacky thee thirsty for another gee. (00:24:57) The kiti kion haway teeks on tucky way, I tiki if I should keeber come backy. (00:25:04) I kishy pee lecky keeth withy a psyche, squeaky kizi and kizi hence. (00:25:09) Kooky roads diverged in a woodkee at I, I kupak the one lecky keevel be, and peak has peeg all the difference. (00:25:17) It's wild, right? (00:25:18) I don't know. (00:25:19) It makes me smile. (00:25:20) I don't know. (00:25:20) I don't know how to process that. (00:25:22) It's super interesting, though. (00:25:23) It makes me smile. (00:25:25) It definitely like highlights the sound things. (00:25:27) I don't know. (00:25:28) Do you feel as though language has been punched in the face? (00:25:32) I definitely feel like that you have computationally represented some sort of mouth sound. (00:25:38) You're saying it's above the level of mouth sound, but like that's how I appreciate it is like some sort of... (00:25:44) You've kicked the poem in a certain direction in some weird dimensional space. (00:25:47) Well, that's part of what I think is so interesting about it, right? (00:25:50) I didn't use any audio data to make this happen, right? (00:25:53) I just used the phonetic transcription. (00:25:55) It's the data of the phonetic transcription. (00:25:57) Yeah, I think it's cool. (00:25:58) Should I read the other one? (00:25:59) Yeah, let's do it. (00:26:00) The Buba one? (00:26:01) Let's do it. (00:26:02) Okay, so this is the same poem plus Buba. (00:26:05) Jubu Roads, Barbersville and Yellowwood, and Bari, Ikoba, Knob, Travel Both, and Bobby One, Bosler, Dom, Istova, and Jukebox Bode, One as Fad as (00:26:15) Koba to bowel it font in the Bogard. (00:26:18) Babu bow cook the bother as Babu is fair, and having perhaps the Babet claim, Bagas it's abawa barisi and wamba bowel. (00:26:27) Though as for Bogach, the Babu Babu hab warm them bodly abu the same. (00:26:32) And both Bogach booming equally lay, and bob's no bobet hab baden blob. (00:26:37) Oh, I bobcat the first for bother joy, Babet knowing Baha way bob's on to way. (00:26:43) I bowed if I should ever kebab bob. (00:26:46) I shall bobby-bobu-bogas with a sebu, bobby-bobbies and bobbies hence, chubu roads, barbers fill in a wood, and I, I bo crook the one bob who traveled tha, and bo gach has by vavo all the ballots. (00:27:00) That's very good. (00:27:01) Thank you so much. (00:27:01) Yeah, you're welcome. (00:27:03) I had a computer when I was a kid, too. (00:27:05) I built some things, and I feel like the world is a little bit... (00:27:09) The world of people who computer program is constrained, and everything's about building to-do apps, or making super optimized web services or something. (00:27:19) And so you're doing something super different. (00:27:21) And that impresses me. (00:27:23) And I don't know how-- do you think that the world should be more creative in the use of computer programming? (00:27:30) I'm trying to pick through that question. (00:27:32) Because, I mean, the answer is obviously yes, right? (00:27:35) And it's yes in a way that I have come to see or I've come to feel is fairly unproblematic, right? (00:27:42) I have strong opinions about this that are maybe not backed up by anything practical. (00:27:46) But one of the things that I tell my students, I teach in a program that's a combination of design and technology and the arts, and I teach our introduction to computer programming class. (00:27:57) It's called Introduction to Computational Media. (00:28:00) One of the things that I tell them on the first day is, computer programming is beautiful and useless. (00:28:05) That's the reason that you should want to do it. (00:28:07) It's not because it's going to get you a job, not because it has a particular utility, but simply for the same reasons that you would pick up oil paints or do origami or something. (00:28:19) It's something that has an inherent beauty to it. (00:28:23) that is worthy of studying. (00:28:25) And that beauty stems from a whole bunch of things. (00:28:28) One is just like the pure mathematics of computer programming, I think are super interesting. (00:28:32) Like to the extent that pure mathematics is a field of practice that lends itself to that same kind of artistic thinking that is joyful just for the purpose of just for doing it. (00:28:43) I think computer programming is like that. (00:28:45) And I mean, when you're making a to-do app, you are doing something creative in the sense that you're like applying (00:28:52) your skills and your interests in making those kinds of decisions. (00:28:58) It's just they're attached to this very uninteresting problem, right? (00:29:01) The same way that you might be really good at oil painting, and you could still use oil painting to paint Portrait of a Dictator, right? (00:29:09) That's not a good use of that skill, but it is a use that you can put it towards. (00:29:14) So yes, I think in general, the world would be a better place if we used computer programming (00:29:19) for what to me seems like its intended purpose of being artistic, of being creative, of building communities, of being citizens of the world, of trying to make the world like a good and beautiful place to be. (00:29:33) And I think it's like a real shame that this delicate artistic process, delicate even though I'm proposing it can be used to punch someone in the face, you can punch someone in the face delicately and still do a lot of damage, I have to say. (00:29:45) has been turned towards these other applications that, let's say, at best are uninteresting and at worst are much worse than that. (00:29:54) That's a great answer. (00:29:55) So I know a ton of people who would be writing computer programs, whether you pay them two or not, but they get paid well to do it. (00:30:03) It's like the world of oil painting, if everybody could get a job. (00:30:07) you know, for a hundred plus thousand dollars doing portraits of people on tourist portraits, right? (00:30:12) That would negatively influence the world of oil paintings, I assume. (00:30:16) I mean, maybe. (00:30:17) It's hard to speculate about that precisely. (00:30:20) I'm certainly not trying to say if computer programming is something that you do for a job, that it also needs to be your passion. (00:30:26) That's not part of this equation. (00:30:28) It can be interesting to you for other reasons other than it's something that you feel fixated on for whatever reason. (00:30:35) And that's also true for artists, right? (00:30:36) There are lots of professional artists who feel passionate about their medium, but who don't necessarily feel like they have to do it after office hours or after their day at the office is over. (00:30:47) Something can be your professional practice and you can feel really passionate about it without it being something that consumes your soul in this very romantic (00:30:55) romantic era artist kind of way. (00:30:57) I just mean that I like it's not a bad thing to just program for your job. (00:31:01) Oh yeah, totally. (00:31:02) And also I would say like you can have very practical things that you do and they can be super fun and interesting and intellectually stimulating. (00:31:10) Oh yeah, absolutely. (00:31:11) Absolutely. (00:31:12) But for my purpose as an artist and as like someone who teaches programming to artists and designers. (00:31:18) I want to emphasize that it's not only a vocational thing. (00:31:23) It's not only a way for building things like to-do apps. (00:31:27) For that matter, it's not only a way to write useful applications that help to organize communities or help to do scientific work and other good applications of programming and software engineering. (00:31:39) But there is this very essential, very core part of computer programming that is just (00:31:44) joyful. (00:31:45) That's about understanding your own mind in different ways and understanding the world in different ways. (00:31:51) Yeah, that's a great sentiment. (00:31:53) That was my big tangent. (00:31:54) That was a vector space of the sounds that we did. (00:31:58) What other vector spaces are useful in creation? (00:32:02) Usually, the way that word vectors are used is for semantics, because most often, for whatever reason, people want to (00:32:11) say, cluster documents or sentences based on how similar they are in meanings. (00:32:16) You can say like, Here are all of the tweets that mention Pepsi and here are all the tweets that mention Coke, right? (00:32:22) And you can do that without paying someone to read every tweet. (00:32:24) So it's more about meaning than about other aspects of language. (00:32:28) So that's like the original, like the word2vec research or other word vector stuff like the blood vectors from the Stanford Natural Language Processing Lab. (00:32:38) Or more recent things like Google's Universal Sentence Encoder, which is a neural network that turns similar sentences with similar meanings into similar vectors. (00:32:48) So that's the actual academic research in this era, academic and corporate research in that area. (00:32:54) And I just sort of hijacked some of those techniques to make my phonetic vectors because it was like that was more interesting to me as a poetic thing. (00:33:02) But I've also done stuff with word-to-back, pre-trained vectors, like the glug vectors and so forth. (00:33:07) We're doing things like grouping lines of poetry by meaning and doing things just like chatbots are easy to make. (00:33:13) Once you have a way of judging the semantic similarity of two sentences, you can make a really easy chatbot just by having a corpus of conversation and then always responding to the user. (00:33:25) with a response that is semantically similar to whatever the response was for something that's semantically similar to that user's contribution to the conversation. (00:33:34) It's a convoluted thing to try to explain, but that's basically how it works, right? (00:33:38) So it's like, if I said to the chatbot, I'm feeling sad, then it would find some... (00:33:42) statement near that and say like, I'm feeling down or something. (00:33:46) Is that the idea? (00:33:47) Well, it would find whatever the response was to something that is similar to what you said. (00:33:53) So if you're saying I'm feeling sad, it looks through its corpus and it says like, what's the closest sentence to I'm feeling sad? (00:33:59) And that might be like, I'm feeling down. (00:34:02) and then it responds with whatever the line that comes after that was. (00:34:06) If you trained it on a movie script, instead of saying, it wouldn't say, I'm feeling down, it would say, Buck up, chap, or whatever the response was to that, and then you can go back and forth like that. (00:34:16) That's a very elementary application of this technology, but still shows how effective it can be. (00:34:22) One thing I don't get is how do you get meaning out of this? (00:34:26) How do you group words by their meaning in this multi-dimensional space? (00:34:32) way that it's generally done is through co-occurrence matrixes. (00:34:36) So it'd be like, if you go through an entire text, make a really big spreadsheet that has one row for every word and then one column for every possible context that a word can occur in, that might just be like the word before and the word after. (00:34:52) And then just calculate for every word, how many times does it occur between the and of, how many times does it occur between (00:35:00) abacus and alphabet, how many times does it occur between this and going or whatever. (00:35:08) So you have all of the possible contexts and then every word, and then you just count how many times the word occurs in that context. (00:35:14) That number then in every row is then a vector that corresponds to what words occur in similar contexts, right? (00:35:23) So if two words have similar contexts, (00:35:26) That's like a key according to this theory of semantics, that those two words share a meaning or have similar semantic characteristics. (00:35:35) And that bears out if you think of words like days of the week, right? (00:35:39) So I'll say something like, This Tuesday, I am. (00:35:45) Or, Last Wednesday, we will. (00:35:48) So it was context of like this and last and next. (00:35:52) Those are shared by days of the week and maybe not with other things. (00:35:55) You would never say, This puppy, I'm going to the store, or, This yesterday, I'm going to the store. (00:36:01) With those day of the week examples, though, they don't actually mean the same thing. (00:36:07) They, in fact, mean different things, but I guess they're related. (00:36:10) They are semantically related. (00:36:12) This is one of the benefits of word vectors that once you calculate these things automatically, one of the drawbacks is that there's no way for it to understand those really subtle differences between the meanings (00:36:22) words, except by making the window of the context much bigger. (00:36:28) So the idea would be if the context is just one word before and after, I'm never going to get that meaning. (00:36:34) But if the context was a million words before and after, then you would, by necessity, almost come to capture the unique context of those words. (00:36:44) I mean, a million is impractical, but there are other ways of getting around it. (00:36:48) All of the more recent research in this field has basically been about how do you find tricks to make that window bigger and bigger without actually using up all of the RAM in the universe. (00:37:00) It makes me wonder if in this vector representation where you have a whole bunch of numbers, if there's one, the 73rd one in is whether it's a time and place or something. (00:37:11) Yeah, there's been some research with the pre-built vectors because the dimensions don't actually mean anything on their own. (00:37:18) The researchers have shown that some of the individual dimensions of the vector do actually correspond to particular semantic properties, which is interesting to think about. (00:37:26) But it's just sort of like an epiphenomenon of the way that the dimensional reduction works. (00:37:32) whether it's done with a neural network or with a technique like principal components analysis or something like that. (00:37:37) Oh, there's no stability to what these dimensions are. (00:37:39) Each process produces some different- Yeah, exactly. (00:37:43) Exactly. (00:37:43) Yeah, because there's the Big Five personality test. (00:37:46) It has five dimensions, and I believe that the way that they were come up with was using principal component analysis, and then they retrospectively gave them names, like this one is extroversion. (00:37:57) Yeah, that kind of thing I imagine is fairly common, even (00:38:02) Something like t-SNE, you're retroactively assigning. (00:38:05) These dimensions don't actually mean anything, but you're assigning, well, this means the x-axis and this means the y-axis, right? (00:38:11) With what? (00:38:12) Sorry, what was your example? (00:38:13) t-SNE. (00:38:14) It's the dimensional reduction algorithm that's used commonly for doing 2D visualizations of bi-dimensional data, and it makes weird swirly patterns. (00:38:23) It was popular four years ago, I guess. (00:38:26) Often, when you're doing a visualization of GAN results or something, and you want to show them (00:38:32) Well, GAN isn't a good example because that's a cleaner latent space, but you're doing results of your autoencoder for handwriting digits or whatever, and the underlying model has 50 dimensions, but obviously our screens only show two dimensions, so you want to show it in two dimensions. (00:38:52) t-SNE is a good way of taking 50 dimensional data and reducing it to just two dimensions. (00:38:59) So you can easily show it as a visualization. (00:39:01) Oh, very cool. (00:39:02) Yeah, I didn't invent it or anything like that. (00:39:04) It's just something that I know about. (00:39:06) So another thing you do is make Twitter bots. (00:39:09) That is a form of your art? (00:39:11) I haven't made Twitter bots in a while actually, but for a while it was pretty important. (00:39:16) The benefit of a Twitter bot is that it's like a really easy way to publish (00:39:21) Developish any kind of writing, really, but especially generative, like computer-generated writing. (00:39:26) It's a really easy way to publish that, because it has sort of a low barrier of attention, but it can still reach a wide audience. (00:39:33) What is Every Word Twitter bot? (00:39:35) So Every Word is a Twitter bot that I started when I was in grad school in 2007. (00:39:41) I was inspired by a piece called Every Icon by John Simon, which is a piece, it's like a 32 by 32 monochrome grid, and it's gradually iterating through every possible permutation of that grid. (00:39:57) So basically like every possible combination of those pixels being on or off, and it's doing like one every thousandth of a second, and it's going to keep doing it obviously for millions of years because (00:40:08) A 32 by 32 grid is 2 to the 32 by 32, so it's a huge, huge number. (00:40:14) So we were learning about that in class, and that was about at the same time that Twitter was kind of kicking off. (00:40:19) And then people are saying, well, Twitter is useless. (00:40:21) People just talk about their sandwiches or whatever dismissive thing people are saying about it then. (00:40:26) So I was like, well, (00:40:27) I'm going to do this really small project, and it's going to be like every icon, but instead it's going to be every word. (00:40:31) I'll tweet every word. (00:40:32) So if people think language on Twitter is useless, we'll see whether that's the case. (00:40:36) I'll just tweet every possible word and thereby make every possible statement. (00:40:40) So it lasted for seven years. (00:40:42) It ended in June 2014. (00:40:45) It went through every word in a word list that I happened to find somewhere. (00:40:48) I don't remember where the word list was. (00:40:50) At its peak, it had a little bit more than 100,000 followers, which doesn't seem like a big deal now because-- Seems like a big deal to me. (00:41:00) But the standard for Twitter now is much higher. (00:41:03) Barack Obama has 100 million followers or something like that, right? (00:41:07) But for an experimental writing project made by a grad student, it was a pretty big following. (00:41:12) So yeah, and then I published a book version of every word with Instar Press a couple of years ago that has every word. (00:41:20) the Twitter bot tweeted along with the number of favorites and retweets that it got. (00:41:24) I feel like I need to unpack this. (00:41:26) So you have on this bot 100 times the number of followers that I have and you tweet each word and then you also published it. (00:41:33) Like it sounds a little bit a successful joke that's taken off. (00:41:37) I don't know. (00:41:40) Yeah, it was a tongue-in-cheek project, right? (00:41:42) It was like a lot of arts projects that are kind of in the avant-garde. (00:41:46) It has its roots in a little bit of satire, a little bit of just like, what if we did this, right? (00:41:51) Let's see what happens. (00:41:52) So yeah, it was definitely a joke and it was funny and I had fun. (00:41:55) And I think it... (00:41:57) It showed a lot of different things. (00:42:00) It was a successful experiment as well, I think, in the sense that it was sort of like this ultimate project in the decontextualization of words. (00:42:10) What do words actually do when you forcefully take them out of language? (00:42:14) How do people respond to them in that context? (00:42:17) The fact that people had used every word, people still favorite and retweet the words even to today. (00:42:23) What was interesting while it was running is that, this was back before Twitter really leaned into the algorithmically moderated feeds, is that the tweets would just come up in the middle of your feed, like your friend. (00:42:35) might be tweeting about their sandwich or... (00:42:37) It's hard to imagine anybody using Twitter for anything except either spreading fascism or being unhappy about fascism right now. (00:42:46) But back in the day, if you cast your minds back to 2009, people actually used Twitter for actual purposes and not for self-promotion and trying to either destroy or save the world. (00:42:59) So these words had come up between two people's tweets, and you might have the tweet for happy might come up. (00:43:05) And the next tweet after that might be a friend that got good news, or a tweet might come up and it might be like, Lunch. (00:43:10) And you'd be like, Oh, maybe I do want to get lunch. (00:43:13) So it's sort of this heartbeat. (00:43:15) It was tweeting every half hour. (00:43:16) So it's kind of this weird heartbeat that was injecting your Twitter feed with a little bit of, not randomness, but serendipity. (00:43:24) So I think it was successful from that perspective. (00:43:26) I would totally take a clock that just had, instead of actual time, with just words. (00:43:33) I think that would be interesting. (00:43:34) And it was also about how does reading on Twitter actually work? (00:43:39) Can Twitter be used as an artistic medium? (00:43:41) Every Word was one of the first, maybe not one of the first Twitter bots, but is sometimes recognized as one. (00:43:48) one of the first specifically artistic Twitter bots. (00:43:51) So it kind of was participating in this idea of can social media be a canvas for artworks? (00:43:58) Yeah, that's amazing. (00:44:00) I remember there used to be this thing. (00:44:02) I mean, it probably still exists. (00:44:03) I was just trying to Google it. (00:44:04) It was like Garkov. (00:44:05) It was like Garfield with Markov chain written text. (00:44:09) Have you ever- Yes. (00:44:10) I think I know at least Josh Millard made a version of that. (00:44:14) There might be others, but yeah. (00:44:16) Yeah, I know you're right. (00:44:17) It's Josh Millard. (00:44:18) Yeah, okay. (00:44:18) Yeah, Josh is brilliant. (00:44:20) Josh is another internet prankster/artist who makes really great work. (00:44:26) You teach students and if somebody is fluent in computer programming and wants to make things using text, where would they start? (00:44:36) Do you have a recommended what they should play around with? (00:44:38) What should they try to create? (00:44:40) There are a couple of really interesting and easy resources. (00:44:44) All of my class material I put online. (00:44:48) If you (00:44:48) to decontextualize.com, which is my website. (00:44:51) There's a whole bunch of materials, mainly in Python. (00:44:53) That's the language that I work with the most. (00:44:55) For a really easy and really powerful tool for working with generative text, I would recommend Tracery, which is a tool that Kate Compton made. (00:45:05) Kate Compton's a recent PhD graduate from UC Santa Cruz's computational media program, and also just a brilliant all-around person. (00:45:13) So she made this kind of a simple programming language. (00:45:17) called Tracery that makes it really easy to write Madlib style text generators, but also text generators that have like recursive syntactic linguistic structure. (00:45:28) And there's a tool that goes along with that called Cheap Bots Done Quick, which makes it really easy to turn your Tracery grammar into a Twitter bot. (00:45:35) And that workflow of like, I learned Tracery, and then I learned Cheap Bots Done Quick, and then I made Twitter bots, that's like, to me, it's sort of like the gateway drum path of, (00:45:44) getting involved in this kind of work. (00:45:46) I think that gives me more context to some things you said earlier, because that would be very easy for me to just take somebody I don't like and build some bot that constantly harasses them. (00:45:56) Yeah, I mean, that's what you have just described is contemporary global politics. (00:46:01) And that's part of the reason I don't make Twitter bots anymore is, first of all, because Twitter closed down the developer rules to make it more difficult to make Twitter bots for good reasons, right, because bots are like a huge vector for (00:46:14) harassment and gaming engagement algorithms and stuff like that, that it just made it harder for me as an artist. (00:46:21) And also because Twitter is not good anymore. (00:46:24) Twitter is not a good influence on the world, and I didn't want to be in the business of adding legitimacy and beauty to a platform that I don't think is actually contributing to the world being a better place. (00:46:37) But it really is easy to get started making Twitter bots with tracery and cheap bots and quick. (00:46:42) And it's still, I think, an important place for arts and for experimental artists to publish their work and for artists to intervene and make things that question the way that social media is supposed to be used. (00:46:55) And you mentioned this word computational media. (00:46:57) If I want to learn more, is that what I should be Googling or what's the term or phrase for this area? (00:47:05) I don't know. (00:47:06) I should have that. (00:47:08) Computational media is more of a broad phrase that includes just anywhere the computation and media intersect. (00:47:15) My own interest is in generative stuff. (00:47:18) That's the word that's usually used with it, even though that term has different meanings and different contexts. (00:47:24) generative art is art that's like generated with computer algorithms. (00:47:27) So that's sort of like the phrase that I would go for is like generative text, generative poetry, things like that. (00:47:33) Generative art. (00:47:34) That's awesome. (00:47:35) I heard the old story about Brian Eno, and he would get sound loops that all had different lengths, and he would get them all playing. (00:47:44) It was just constantly generating music, and a lot of times it would be bad, but sometimes it would get in a weird offset and produce something cool. (00:47:51) Yeah, exactly. (00:47:52) The thing that I like to emphasize when talking about this kind of work is that it's not new. (00:47:56) You're talking about Brian Eno, who was making art that used these techniques before he had his hands on computers, but then even before that, you had Steve Reich. (00:48:05) You had (00:48:05) Artists like John Cage, Allison Knowles, who I mentioned earlier, Jackson Mackla, Tristan Zahra. (00:48:11) Going back even before the 20th century, you had artists working not necessarily with computers, but with techniques that we could label as computational. (00:48:20) So it's not like this new thing. (00:48:22) The first computer-generated poem didn't happen yesterday when the newest model from Google came out. (00:48:28) It happened like a hundred years ago. (00:48:31) interest in Zara was pulling words out of a hat and maybe even earlier than that. (00:48:35) Oh, that's awesome. (00:48:36) I mean, it's obviously not new, but it's a bit new to me. (00:48:39) So I'm learning. (00:48:40) And I think it's something. (00:48:41) Oh, that's fine. (00:48:41) Yeah. (00:48:43) I think I just want people to make more weird stuff. (00:48:45) That's my perspective. (00:48:47) Yeah, I agree. (00:48:48) So you mentioned your book, and I think we're running out of time. (00:48:51) I was wondering, is there anything you'd like to share with us from the book? (00:48:55) Okay, yeah, so I will read a short selection from Articulations, which is the book that I wrote. (00:49:01) It's part of Nick Montford's Using Electricity series, which is a series of books that are computer-generated, so it's from Counterpath Press. (00:49:12) You can buy it online at Smallpress Distribution or Amazon, and it's just a short section from it. (00:49:19) A shape of the shapeless night, the spacious round of the creation shake, the seashore, the station of the Grecian ships. (00:49:27) In the ship, the men she stationed, between the shade and the shine, between the sunlight and the shade, between the sunset and the night, between the sunset and the sea, between the sunset and the rain, (00:49:38) A tint in the sweet air with the setting sun, the setting sun. (00:49:42) The setting day, a snake said. (00:49:44) It's a cane. (00:49:45) It's a kill. (00:49:45) It's like a stain, like a stream, like a dream. (00:49:49) And like a dream sits like a dream, sits like a queen, shine like a queen, when like a flash, like a shell, fled like a shadow, like a shadow still, lies like a shadow still. (00:50:00) Aye, like a flash, oh by, shall I like a fool, quoth he. (00:50:04) You shine like a lily, like a mute, shall I languish, and still I like Alaska. (00:50:10) Lies like a lily white, is like a lily white, like a flail, like a whale, like a wheel, like a clock, like a pea, like a flea, like a mill, like a pill, like a pill, like a pall, hangs like a pall, hands like a bowl, bounds like a swallow, falls like a locust swarm on boughs whose love was like a cloak for me. (00:50:31) whose form is like a wedge, but I was saved like a king, was lifted like a cup, or leave a kiss, but in the cup, the cup she fills again, up she comes again, till she comes back again, till he comes back again, till I come back again. (00:50:46) That was great. (00:50:47) Thank you very much, Alison. (00:50:48) Thank you for punching language with computers. (00:50:52) Sure, anytime. (00:50:56) All right, that was the talk with Allison. (00:50:58) I hope you liked it. (00:50:58) Back in 2017, I went to Strange Loop Conference. (00:51:03) It was the first time I've been there, the only time so far. (00:51:08) And I was kind of doing a little bit of podcasting then, but not for Code Recursive. (00:51:13) I did some episodes for SE Radio, and so podcasting was kind of on my mind. (00:51:18) And the conference was super interesting 'cause there was a lot of people talking about like scaling X or like type level programming, but then also people just doing like interesting things, you know, just building fun things with computer programming and like showing the code and walking through it. (00:51:34) And Allison was one of those people. (00:51:36) She did a talk that included manipulating text using pronunciations and, you know, some of the readings that we did today. (00:51:44) And at the time I was thinking like, this is just something that's very well suited to an audio format, like a reading, manipulating the pronunciations of words, coming up with this multi-dimensional representation of words, and then kind of like moving things in certain directions in that multi-dimensional space, and then hearing the results. (00:52:04) I thought, you know, this is something that would work great in an audio format. (00:52:07) So I hope everybody enjoyed it. (00:52:09) I think it was a little bit of a different type of show, but I hope it encourages people to play around with (00:52:14) creating art with computers, computer programming without a specific purpose. (00:52:20) Yeah, let me know what you think. (00:52:21) I thought it was a great episode.