The agent autonomously figured out how to transcribe and translate an audio message it was never given the capability to handle, by inspecting the file header, converting it with ffmpeg, and using a discovered OpenAI API key with curl.
Peter describes the moment his early WhatsApp-agent prototype received a voice message it wasn't built to handle, and autonomously figured out how to detect the file type, convert it, and use a found API key to transcribe and translate it. ✦ AI generated
Peter Steinberger · Lex Fridman · 2026-02-12 · original ↗
starts at this moment · 16:23
“So, oh, you did an audio message?”
I literally went, 'How the fuck did he do that?' And it was like, 'Yeah, the mad lad did the following. He sent me a message but it only, only was a file and no file ending.' So I checked out the header of the file and it found that it was, like, opus so I used ffmpeg to convert it and then I wanted to use whisper but it didn't had it installed. But then I found the OpenAI key and just used Curl to send the file to OpenAI to translate and here I am.
verbatim transcript · starts at 16:23
16:03- And it's not supposed to work because- - No - ... you didn't give it that- - No, literally - ... capability. - I literally went, "How the fuck did he do that?" And it was like, "Yeah, the mad lad did the following. He sent me a message but it only, only was a file and no file ending." So I checked out the header of the file and it found that it was,
16:23like, opus so I used ffmpeg to convert it and then I wanted to use whisper but it didn't had it installed. But then I found the OpenAI key and just used Curl to send the file to OpenAI to translate and here I am. Just looked at the message I'm like, "Oh wow." - You didn't teach it any of those things and the agent just figured it out, did all those conversions,
16:47the translations. It figured out the API, it figured out which program to use, all those kinds of things. And you were just absent-mindedly just sent an audio message when it came back. - Yeah, like, so clever even because he would have gotten the whisper local path, he would have had to download a model. It would have been too slow. So like, there's so much world knowledge in there, so much creative problem solving. A lot of it
17:08I think mapped from... If you get really good at coding that means you have to be really good at general purpose problem solving. So that's a skill, right? And that just maps into other domains. So it had the problem of like, what is this file with no file ending? Let's figure it out. And that's when it kind of clicked for me. It's like, I was like very
17:29impressed. And somebody sent a pull request for Discord support and I'm like, "This is a WhatsApp relay. That doesn't, doesn't fit at all." - At that time it was called WA Relay. - Yeah. And so I debated with me like, do I want that? Do I not want that? And then I thought, well maybe, maybe I do that because that could be a cool way to show people. Because I... So far I did it in WhatsApp
18:00as like groups you know but don't really want to give my phone number to every internet stranger. - Yeah. - Um, journalists manage to do that anyhow now so that's a different story. So I merged it-... from Shadow, who helped me a lot with the whole project. So, thank you. And, and I put my, my bot in there. - On Discord? - Yeah. No security because I didn't... I hadn't built sandboxing in
- ·Received a file with no extension or format info
- ·Inspected file header, identified it as Opus audio
- ·Used ffmpeg to convert the file itself
- ·Wanted Whisper, but it wasn't installed
- ·Found an OpenAI API key on its own
- ·Used curl to send file for transcription and translation
- ·Used curl to send audio to OpenAI for translation
- ·Fully autonomous chain with no human instruction
- ·Peter Steinberger: 'the mad lad did the following'