The relational foundation model produces predictions via a single forward pass over in-context examples, with no training, backpropagation, gradients, agent loop, or memory involved.
Leskovec clarifies RFM2 is not an agent with memory or scratchpad reasoning — it's purely a single forward pass of a frozen pre-trained transformer-like network using in-context examples. ✦ AI generated
Jure Leskovec · The TWIML AI Podcast · 2026-05-21 · original ↗
starts at this moment · 34:43
“Does the model require some type of memory structure blackboard or something in order to um you know do a scratch work to to come up with a representation or is this all like thought traces or something like that?”
No, this is not this is not an agent. This is a single forward pass of a transformer-like neural network. Right? So this is purely inside the neural network. There is no agent. There is no memory. There is no scratch pad... The answer is truly a single forward pass of a neural network. There is no loop.
verbatim transcript · starts at 34:43
34:43or is this all like thought traces or something like that? >> No no no this is not this is not an agent. This is a single forward pass of a transformer-l like neural network. Right? So this is purely inside the neural network. There is no agent. There is no memory. There is no scratch pad. There is no let me do this, let me do that. Right? The answer is truly a
35:07single forward pass of a neural network. There is no loop. Nothing like that. Right? So you you get the answer in I don't know 2 seconds, half a second uh whatever the time be, right? It's really a single forward pass of a pre-trained frozen neural network. There is no language model here, right? This is kind of technology that's parallel or or complementaryary to language models, right? You cannot textify a database and
35:35then go to chat GP and say, "Hey, what do you think? How how likely is this transaction to be fraudulent? You get horrible results, right? So, this is uh yeah, frozen pre-trained architecture that allows you that allows you to do that. I feel like I've gone the full cycle from that's an outlandish claim to oh yeah, I can see how that will work to I don't know. It's still
35:57kind of crazy that it works. >> Yeah. No, it's interesting, right? And when we when we test this on data sets that are locked away and hidden and the model has never been trained on and on task tasks that we haven't even thought about, we see we see a we see a gain over best supervised models out there. Right? If you would go and say, I I'll
36:20hire a data scientist. They'll spend several weeks building the model, tuning the model, the latest neural networks, whatever, this, it's still a couple of percentage points worse. Um and then if you fine-tune the let's say the foundation model uh on on on more data for the specific task then you get to this superhuman accuracy performance that you know present manual or semianual or agentic solutions uh are
- ·No training, backpropagation, or gradients
- ·No agent loop, memory, or scratchpad reasoning
- ·Purely a frozen transformer-like network
- ·Predictions use in-context examples only
- ·Relational foundation model (RFM2) is pre-trained once
- ·New tasks supplied as in-context examples at inference
- ·Single forward pass produces the prediction directly
- ·No iterative refinement or multi-step reasoning