Proof of Concept (POC) trials for AI products must have clearly defined success criteria and strict end dates to avoid becoming open-ended 'science projects'.
Andy emphasizes the importance of setting clear success criteria and end dates for AI POCs to prevent them from dragging on indefinitely as 'science projects'. ✦ AI generated
Andy · a16z Podcast · 2026-08-13 · original ↗
starts at this moment · 22:24
I think the two biggest things are make sure you have an end date, right? ... It's a 30-day trial, it's a 45-day trial, it's a 60-day trial, period, end of story. And then the second one is you have to define the success criteria up front. Here is what we are proving that we can do for you... I think one of the real dangers today is these things turn into like science projects... you run the risk of these trials or proof of concepts going on forever.
verbatim transcript · starts at 22:24
22:04>> Yeah. I I think it's become more challenging, right, in the world of AI because number one, things are moving so fast. Like things are changing daily. And if you think about like a a proof of of concept or a trial, you know, the whole idea, if you're on the sales side, is I want the customer to experience this, I want to prove that it works for
22:21them, but I want to do it in a in a period of time that doesn't go on forever, right? And so what you what you have to stay away from, and I think one of the real dangers today is these things turn into like science projects, right? Like I'm going to deploy this, well, can it do this? Can it do this? Can you show me this? Can you show me
22:34this? And of course things are advancing every day, so the answer is probably yes, I could, but then you run the risk of these trials or proof of concepts going on forever. >> Yeah. >> And so it just takes a lot of discipline, I think, in today's day and age to to really box that in and say, "Listen, here's what our solution does, and we're going to, you know, define it
22:52this way, and we're both going to agree that if it, you know, has done this after 45 days, it's success and you're going to move forward with the purchase." >> so there's there is a lot of that today. >> And did you um like I guess would you recommend having like kind of auto converts on these as much as you can or Yeah, like I don't know how like other
23:09like kind of learnings and lessons from this type of like 30-45 day And then we can talk about like the time you know the the right amount of time that you're giving people with the product. >> I think I think the timing depends a little bit on the complexity of your product, right? If it's going to take, you know, 2 weeks to set up, then you can't make it a 2-week trial type thing.
23:25But I think the two biggest things are make sure you have an end date, right? >> Yeah. >> It's a 30-day trial, it's a 45-day trial, it's a 60-day trial, period, end of story. And then the second one is you have to define the success criteria up front. Here is what we are proving that we can do for you, right? And in some of these companies, you know, you can't do a proof of
23:43concept because, you know, maybe it is regulatory, maybe there's, you know, too much risk in there, and they're not going to let you do it. But where you can, I think you want to make sure you have both an end date and you have the the success criteria clearly defined. >> Yeah. I think this is so tricky right now where, you know, so many, you know, like you have to basically define the
24:02scope that you're going after, and the success criteria, but often times if you're like automating something that has never been automated before, there's like significant amount of configuration, and that like costs money. And then your product could could work, but it might be deployed improperly, and and and or the results take longer than 30 or 45 days. And so how do you actually kind of There's like