Unlike computer vision and NLP, structured/tabular data has never been transformed by deep learning because practitioners still rely on manual feature engineering (SQL/ETL) rather than learning directly on raw data.
Leskovec argues that while vision and NLP were revolutionized by neural networks learning directly on raw pixels/tokens, tabular ML is still stuck doing manual feature engineering on a single flattened table. ✦ AI generated
Jure Leskovec · The TWIML AI Podcast · 2026-05-21 · original ↗
starts at this moment · 11:40
“You talk a little bit about how the these two areas of of research and practice relate to one another.”
AI has not transformed this structured data space in the same way as uh computer vision or natural language understanding have been fundamentally transformed by AI... the transformers they just learn over tokens, no grammar, no syntax... the same thing is actually not happening on structured tabular data. There we don't learn on raw data, we run all these SQL queries, all this ETL, all this feature engineering to then come up with a set of signal.
verbatim transcript · starts at 11:40
11:20feature engineer this single table of your features, you add a label, and now you train some supervised model that from the features predicts predicts that label, right? And we've been doing that over and over again. And maybe this predictive model, you know, it's a it's a deep model we would call, but um it's a neuronet network. But what I would argue is it that AI has not transformed
11:47this structured data space in the same way as uh computer vision or natural language understanding have been fundamentally transformed by AI. Okay. And let let me let me quantify what do I mean by that, right? Like what was the big breakthrough both in computer vision as well as in natural language understanding? It was about let's build neural networks that learn directly on the raw data. Right? In the old days you
12:13would do in computer vision you would do all kind of feature engineering sift features gabbor filters and you'd be like I'll describe this image as well as I can so I can then predict you know is there a is there a car on the image or not right? um in in NLP was similar, right? Like we we you know IBM uh won Jeopardy with uh with their system, but
12:36it was all super handgineered, manual and so on, but you know, kind of it worked, right? But it took 300 people to build build it and was very, you know, was was great, but was very kind of uh brittle, right? So again the transformers they just learn over tokens no no no no no grammar no syntax no it's just you know learn over tokens right again a neural network directly on the
13:01raw data the same thing is actually not happening on structured tabular data right there we don't learn on raw data we run all these SQL queries all this ETL all this feature engineering to then come up with a set of signal from which we let's say try to predict something and and when we uh came up with um with this idea of relational deep learning our goal was to
13:28fundamentally disrupt this and say hey why can't I just learn directly over row relational data um and why why do we always have to learn over a sing data in a single table and the point is that as I take this multitabular data and just to be very precise right what's a good example of list. It could be like I have a set of customers. I said have a set of
- ·Vision and NLP were revolutionized by learning on raw data
- ·Transformers learn directly over tokens — no grammar, no syntax
- ·Structured/tabular data still relies on manual feature engineering
- ·Practitioners run SQL queries and ETL pipelines
- ·They manually engineer features before modeling
- ·No equivalent of end-to-end raw-data learning exists