People mostly evaluate AI agents by 'vibes' instead of rigorous benchmarks because agent behavior (its tool choices, plans, and execution paths) varies more run-to-run than the underlying hardware does, making controlled hardware comparisons noisy and hard to trust.

6:34responsiveness. So, while this experiment started as a hardware comparison, it's clear that the software stack plays a huge role in the overall experience. Another thing that this experiment really highlights for me is why people mostly talk about vibes when evaluating agents. Real benchmarking of these systems is hard. In this case, the biggest source of variation wasn't the thing I was trying to investigate, the hardware, but rather it was the agent

7:01itself. Different tool choices, different plans, and different execution paths often had a bigger impact on runtime, a much bigger impact on runtime in fact, than whether the task was running on a VPS or the Rubik's Pi. And beyond the agent, at least for these workloads, the harness matters just as much as the hardware. So, where does this leave me? For me, there are three practical takeaways. First, I'm probably not switching to a

7:26setup like this for my everyday interactive AI usage. For that, running Chat GPT, Claude, or Gemini directly on my laptop or phone is simply faster and more convenient. That said, I do see a lot of value in systems like Open Claw for asynchronous workloads, things that run on schedules, things that run on triggers, things that gather information, summarize content, or perform background automation while I'm doing something else. For those kinds of

7:50tasks, responsiveness matters much less, and the Rubik Pi performed pretty well. In fact, based on what I've seen so far, I will be shutting down my $28 a month VPS and running my Open Cloud style agents on the Rubik Pi. A Rubik Pi costs about $280, so figure a 10-month payback period, which isn't too bad. Finally, this experiment raised at least as many questions as it answered. I'm

8:13particularly interested in comparing different agent frameworks and understanding where the real bottlenecks are. If that's something you'd like to see more of, let me know in the comments. Hope you found this useful, and I'll see you in the next one.