The Future of Voice AI: Agents, Dubbing, and Real-Time Translation with ElevenLabs Co-Founder Mati Staniszewski

Mati Staniszewski: And the big thing was like, instead of having it just translated, could you have the original voice, original emotions, original intonation carried across? So like, imagine having this podcast, but say people could switch it over to Spanish and they still hear Sarah, they still hear Mati and the same voice, the same delivery, which is kind of exactly what we did with Lex back when he interviewed Narendra Modi and you could kind of immerse yourself in that story a lot better. So that was the original kind of insight.

Mati Staniszewski: Originally, what we did was try to actually use existing models that were in the market and kind of optimize them for our first use case was actually starting with combination of narration and dubbing and then on the creative side. And we realized pretty quickly that the models that existed just produce such a robotic and not good speech that people didn't want to listen to it. And that's where my co-founder's genius came in, where he was able to assemble the team and do a lot of the research himself to actually create a new version of creating that work.

Mati Staniszewski: I still think actually this market will be immense because it's not going to be only the static delivery in movies, but if you travel around the world and want to communicate in real time, like the full babblefish idea from Hitchhiker's Guide of the Galaxy, this will happen. It will be like the biggest, like the whole breaking down language barriers, the barriers to communication, to creation, like all of that will break. And that will be like the foundational real-time dabbing concept.

Mati Staniszewski: I think that second exciting piece within that domain, which is happening is the shift from effectively a reactive customer support, I have a problem, I'm reaching out to customer support, into more of like a proactive part of the experience customer support. So to make it explicit, we work with the biggest e-commerce a shop in India, Michau, where they started working on the customer support side, where I want a refund, I want to see the tracking of the package, to actually having an agent be a front part of the experience. So if you go to the website, you have the widget, you can engage it through voice, and you can ask it, hey, can you help me navigate to item X, item Y, or can you explain to me what's the right thing for me to give up for a gift for this period of time? And then it will actually help you based on your questions, based on what is on the offer, show you those items, navigate to the right parts of the piece, maybe go all the way through the checkout.