Kumo's relational graph embeddings, appended to Reddit's already highly-optimized hand-engineered advertising features, produced a nearly double-digit increase in click-through rate, a gain far beyond the typical ~1% annual improvement from manual feature engineering.
Jure describes Kumo's production deployment at Reddit, where graph embeddings for users, subreddits, and ads were combined with Reddit's existing hand-engineered features and yielded a nearly double-digit click-through-rate lift, dramatically outpacing typical incremental gains. ✦ AI generated
Jure Leskovec · The TWIML AI Podcast · 2026-05-21 · original ↗
starts at this moment · 46:49
“I think I saw somewhere that the system is deployed at like places like Door Dash and and others. Can you talk a little bit about the process for deploying it?”
Another another great client we work with is Reddit. So the advertising models on on Reddit are built on top of or are built with Kuma. Um and it was um nearly a double digit uh increase in at uh clickthrough rates. So basically the the revenue the yeah it's like unbelievable right usually an entire team you know like increases maybe 1% that accuracy year over year right because clickthrough rate
verbatim transcript · starts at 46:49
46:49and we've seen you know uh revenue impacting hundreds of millions of dollars um another another great client we work with is Reddit. So the advertising models on on Reddit are built on top of or are built with Kuma. Um and it was um nearly a double digit uh increase in at uh clickthrough rates. So basically the the revenue the yeah it's like unbelievable right usually an entire team you know like increases
47:20maybe 1% that accuracy year over year right because clickthrough rate >> and this is your original point about like uh you know domain expertise and like manual features like you would imagine that they've been working on this for a long time and they've kind of squeezed a lot of the juice out of that lemon but you know here comes the machine. >> Exactly. Exactly. And it's actually
47:41interesting and I mean the you know we have a great collaboration and great relationship with with Reddit team and and they're amazingly sophisticated and of course they build their own uh super optimized feature engineered pipeline and then and then the way we did we do it there actually is that we said okay let's take your data represent it as a graph and let's create embeddings for users subreddits ads and things like
48:05that. So now these embeddings actually get appended to their to their own features, right? And even with that there was there was a huge increasing the click-through rate because this signal that the neural network learn was kind of complementaryary to what the human feature engineering uh already had. So actually the the model that is in production is combined from the neural network embeddings by by graph embeddings by Kumo as well as the manual
48:36feature engineering. Uh so that's been that's been a great a great collaboration. So it's add add the recommendation clickthrough rate prediction if you want to think of it that way. you know, or have you looked at like if their manual features really make a difference? Like, is that a feel-good thing like you left them in there because they had them? Uh, or do they provide, you know, lift that's been