Judging AI's success can't just rely on usage growth — Booking has to rigorously track the real cost of AI (token consumption, compute, back-and-forth interactions) against ROI per booking and long-term customer lifetime value before scaling it further.

20:03that you know you think are most common for Penny? Are there um you know specific things that it doesn't do well that you just need the underlying models to get better for? I'm a little bit curious about >> well the actual we have some areas where we're coming out with some new things on it but here's something really important asked me the questions you say it's so

20:22great but you tell me it's not really doing much you know the numbers really show up much in the numbers yet because it's still really really small in terms of the absolute number. We got to talk scale here. You know, last year we didund 86 billion dollars worth of travel. That's a lot of travel. Okay? You know, we did over a billion room nights. So, you know, the actual numbers

20:42there's a lot. Well, part of it is we're not pushing it really fully. Now, this one of the issues that I always want to think about is what's the cost of this? And that's not so easy to understand. What is the cost? That's what's developable of running it. How many tokens are we consuming? where or how many times are they coming back and forth? we're doing tell me how much was

21:04the cost of us getting that trip for that person and what is our ROI going to be then the next thing is well what's the return long term lifetime value do they come back the loyalties up what how much and will it change these are things that we don't know yet that we're going to have to continue to work on develop so we know and by the way the whole

21:24thing of token economics now which model should we be using for which purpose and when and stuff And uh you know obviously you can get tokens a lot cheaper in certain regards to different models can be a lot cheaper and that's something that we have to look at very closely too. So it is fascinating that we can do things that I I'm so thrilled that we do

21:46and for example things like using customer service right now customer service where we're using AI it's it's great it happens much faster you know instead of having to staff your humans and at peak time somebody's going have to wait for somebody to pick up. We've all been in that line that queue waiting for someone to pick up and we hate it. But now with AI, the computer can pick pick it up. It's

22:11not a problem. And solve the problem even better and faster. You never will have in the future where you finally you talk to a human. The human says, "I'm sorry. You have to be on hold again while I get somebody else who can solve that problem, which he by then you want to throb somebody. AI will solve that problem." But here's the question again about that is sometimes though you want