Hermes agents are slower for this robot use case because they carry a lot of additional tokens in every prompt compared to quick, lightweight prompts.
Scott identifies token overhead in Hermes's default agent prompts as a key source of the conversational latency he's trying to fix. ✦ AI generated
Scott · Syntax · 2026-07-17 · original ↗
starts at this moment · 6:09
one of the reasons why Hermes is less good for this is because by default a Hermes agents have a ton of additional tokens in here so the responses take a long time because we're sending a lot of information in each prompt rather than quick little fast prompts.
verbatim transcript · starts at 6:09
6:09basically just a conversational prompt. Now we also have some stop word detection where we say go to sleep and Reggie will stop the conversation. Again, this is still things that I'm tweaking and I set out to create a whole host of profiling tools so that I could understand where the latency is. Now one of the reasons why Hermes is less good for this is because by default a Hermes
6:30agents have a ton of additional tokens in here so the responses take a long time because we're sending a lot of information in each prompt rather than quick little fast prompts. It also depends of course which model you're using to have it respond to. No, I just have this connected via my Codex subscription so it is using some chunky models rather than some highly tuned fast models. That's something that I
6:54will be experimenting with more in the future. So, did this work? Well, let's start a wake listening and we're going to see if all of these things work and if we can check them off our list. I have Marshmallow here. We have wake word listening. We have conversation and we're going to first give this a try with memory and see if Marshmallow can understand. By the way, this is the
7:19custom UI that I've been building. Hey Marshmallow. >> Hey, I'm listening. >> What do you know about me? >> You're Scott. You like concise, direct technical progress, live checks, and family-friendly Reiki mini apps. You also prefer short robot prompts, and you've got two kids who may use me for spelling practice. I am listening. >> Go to sleep. >> Okay, I'll listen for hey Marshmallow. >> So, you can see that the conversation
8:08definitely works. Now, the latency is certainly there. It's something that I'm working on improving. Okay, now let's try some of the other things we had in mind. Now, I have several different profiles in Hermes that all have different abilities. And the Marshmallow profile is kid-friendly and kid-safe. However, I have a default profile that has access to things like my calendar, as well as can do coding. So, we're