Video · 2026-07-17 · 14m · 6 moments

Can This Reachy Mini Be My Personal Assistant?

✦ AI generated

timeline · colored by role

Claim◆

To make Reachy a true personal assistant requires wake word detection, back-and-forth conversation, and memory, tool calling, coding tasks, Home Assistant control, and calendar reading.

Scott lays out the full checklist of capabilities Reachy Mini needs to count as a genuine personal assistant, from wake-word detection to memory and calendar access.

transcript

Scott: I need to, one, connect it to my Hermes. I need to get wake word detection, so that way I can talk to it directly starting out of nowhere. I need to have conversation going back and forth. And as far as the personal assistant tasks go, I would love for this thing to have a memory.

Mechanism

Reachy Mini's voice pipeline uses local voice activity detection and Whisper speech-to-text running on a Mac mini to detect speech before checking for the wake word.

Scott explains the mechanics behind the robot's listening pipeline: local voice activity detection feeding into a local Whisper transcription step before the wake word is checked.

transcript

Scott: Basically, the Richie software has voice activity detection. It decides when speech is actually starting and ending based on the audio level that's coming in. If the voice activity detection detects speech or silence, it then starts to transcribe via the local SST, the speech to text that's done being Whisper and it's happening on my Mac mini.

Mechanism

Hermes agents are slower for this robot use case because they carry a lot of additional tokens in every prompt compared to quick, lightweight prompts.

Scott identifies token overhead in Hermes's default agent prompts as a key source of the conversational latency he's trying to fix.

transcript

Scott: one of the reasons why Hermes is less good for this is because by default a Hermes agents have a ton of additional tokens in here so the responses take a long time because we're sending a lot of information in each prompt rather than quick little fast prompts.

Anecdote◆

The first version of the spelling test app failed in practice — the robot consistently marked the child's correct spelling as wrong because the listening window timing was unclear.

Scott's first spelling-test build flopped during a live test with his 9-year-old: the child spelled words correctly but the robot kept marking them wrong due to timing issues.

transcript

Scott: what you didn't see was him spelling the words correctly over and over again and the robot consistently getting them wrong. Specifically because the window for when he was supposed to talk wasn't clear.

Mechanism◆

To fix the spelling app's bugs, Scott set up a computer to autonomously run the spelling test against Reachy Mini via audio and microphone loopback, using OpenAI models to check debug logs and iteratively fix and retest until it worked.

Rather than manually iterating, Scott built a self-healing test loop where a computer conversed with Reachy over audio while OpenAI models diagnosed and fixed bugs automatically.

transcript

Scott: what I actually ended up doing was setting the Richie Mini in front of a computer, having that computer take the spelling test on the Richie Mini back and forth with audio and microphones, and then that way the system was able to self-heal and improve the latency using Open AI models to run the test, check the debugging information, and then fix it, test it, reiterate.

gives example · 1

Prediction

The Reachy Mini personal assistant project was a great success, and the next step is using its camera and an LLM to teach and play Warhammer 40K by identifying pieces on the battlefield.

Scott declares the project a success and previews the next phase: using the robot's camera and LLM vision to help him and his son play Warhammer 40K.

transcript

Scott: I think this was a great success. The robot is now my personal assistant. It's taking spelling tests with our kids and it's only going to get better from here because I still haven't really implemented that much with the camera. The next step on the docket is to use the camera to teach my son and I how to play Warhammer 40K.

Highlight slides

The Reachy Personal Assistant Checklist✦ from: To make Reachy a true personal assistant requires wake word detection, back-and-forth conversation, and memory, tool calling, coding tasks, Home Assistant control, and calendar reading.From Voice Wake-Up to Calendar Control✦ from: To make Reachy a true personal assistant requires wake word detection, back-and-forth conversation, and memory, tool calling, coding tasks, Home Assistant control, and calendar reading.Live test failure: robot vs. 9-year-old✦ from: The first version of the spelling test app failed in practice — the robot consistently marked the child's correct spelling as wrong because the listening window timing was unclear.The gap between engineering and real use✦ from: The first version of the spelling test app failed in practice — the robot consistently marked the child's correct spelling as wrong because the listening window timing was unclear.Self-Healing Test Loop for Reachy Mini✦ from: To fix the spelling app's bugs, Scott set up a computer to autonomously run the spelling test against Reachy Mini via audio and microphone loopback, using OpenAI models to check debug logs and iteratively fix and retest until it worked.Autonomous Iteration Replaced Manual Debugging✦ from: To fix the spelling app's bugs, Scott set up a computer to autonomously run the spelling test against Reachy Mini via audio and microphone loopback, using OpenAI models to check debug logs and iteratively fix and retest until it worked.

Related episodes

Video · The Cognitive Revolution

Intelligence on the Edge: Liquid AI's Ramin Hasani on the Search for Device-Native Foundation Models

→

Video · My First Million

Ray Dalio: the single biggest portfolio flaw keeping you poor

→

Audio · All-In Podcast

Charles & Chase Koch on How They Quietly Built a $150B Empire

→