Evaluation is the single biggest bottleneck to deploying AI because people don't understand how to define value, and every business in the world will unambiguously need it.
Anastasios asserts that evaluation — defining and measuring value — is the largest obstacle to AI deployment, making it an unambiguous necessity for every business, since cost is easy to define but performance depends on each business and use case. ✦ AI generated
Anastasios · 20VC · 2026-08-03 · original ↗
starts at this moment · 51:12
“Do you think the evaluation business is better than the data business?”
I think every business in the world is going to need evaluation unambiguously and that is the single biggest bottleneck to deploying AI because people don't understand how to define value. all this co all this like stuff around cost for value. It's like how do you define value? It's easy to cook costs. I can tell you to go use you know Gemini Flash and that's going to be like way more efficient in terms of token spend.
verbatim transcript · starts at 51:12
50:52>> Why didn't you build a data business with that? Well, we've built an evaluation business around this that allows people to understand the strengths and weaknesses of models and therefore improve them. The labs can improve their models based on, you know, the insights and data that we give them. But we also want to help businesses with this. >> Do you think the evaluation business is better than the data business?
51:12>> I think every business in the world is going to need evaluation unambiguously and that is the single biggest bottleneck to deploying AI because people don't understand how to define value. all this co all this like stuff around cost for value. It's like how do you define value? It's easy to cook costs. I can tell you to go use you know Gemini Flash and that's going to be like
51:33way more efficient in terms of token spend. >> Isn't value entirely subjective? Like for one it's speed and for other it's accuracy. For one do you know what I mean? >> Right. Absolutely. So you can try to decompose it. I I think about it as three a three um threepronged uh value proposition. There's performance and then there's cost and latency. Cost and latency are easier to define but
51:58performance is the tough one because the definition of performance depends on the business, depends on the use case. So at Arena we built this pretty sophisticated pipeline for extracting organic performance measurements from agentic traces. And that's exactly where I would say that the the value lies. and helping businesses take advantage of their own data instead of having to purchase data in order to say which AI works best for