AI proof-of-concept trials need strict end dates and defined success criteria upfront to prevent them from turning into endless science projects.
Andy warns that AI POCs risk becoming open-ended science projects due to rapid capability advances, and recommends boxing them in with firm end dates and pre-agreed success metrics. ✦ AI generated
Andy · a16z Podcast · 2026-08-13 · original ↗
starts at this moment · 22:30
“How do you even think about the delivery mechanism for some of these trial periods, POCs?”
I think one of the real dangers today is these things turn into like science projects... you run the risk of these trials or proof of concepts going on forever. And so it just takes a lot of discipline, I think, in today's day and age to really box that in and say, 'Listen, here's what our solution does, and we're going to define it this way, and we're both going to agree that if it has done this after 45 days, it's success and you're going to move forward with the purchase.'
verbatim transcript · starts at 22:30
22:21them, but I want to do it in a in a period of time that doesn't go on forever, right? And so what you what you have to stay away from, and I think one of the real dangers today is these things turn into like science projects, right? Like I'm going to deploy this, well, can it do this? Can it do this? Can you show me this? Can you show me
22:34this? And of course things are advancing every day, so the answer is probably yes, I could, but then you run the risk of these trials or proof of concepts going on forever. >> Yeah. >> And so it just takes a lot of discipline, I think, in today's day and age to to really box that in and say, "Listen, here's what our solution does, and we're going to, you know, define it
22:52this way, and we're both going to agree that if it, you know, has done this after 45 days, it's success and you're going to move forward with the purchase." >> so there's there is a lot of that today. >> And did you um like I guess would you recommend having like kind of auto converts on these as much as you can or Yeah, like I don't know how like other
23:09like kind of learnings and lessons from this type of like 30-45 day And then we can talk about like the time you know the the right amount of time that you're giving people with the product. >> I think I think the timing depends a little bit on the complexity of your product, right? If it's going to take, you know, 2 weeks to set up, then you can't make it a 2-week trial type thing.
23:25But I think the two biggest things are make sure you have an end date, right? >> Yeah. >> It's a 30-day trial, it's a 45-day trial, it's a 60-day trial, period, end of story. And then the second one is you have to define the success criteria up front. Here is what we are proving that we can do for you, right? And in some of these companies, you know, you can't do a proof of
23:43concept because, you know, maybe it is regulatory, maybe there's, you know, too much risk in there, and they're not going to let you do it. But where you can, I think you want to make sure you have both an end date and you have the the success criteria clearly defined. >> Yeah. I think this is so tricky right now where, you know, so many, you know, like you have to basically define the
24:02scope that you're going after, and the success criteria, but often times if you're like automating something that has never been automated before, there's like significant amount of configuration, and that like costs money. And then your product could could work, but it might be deployed improperly, and and and or the results take longer than 30 or 45 days. And so how do you actually kind of There's like