Separating image generation into an Architect (planning composition) and an Artist (rendering pixels) — mirroring how human artists work — makes complex multi-subject scenes more tractable than end-to-end generation.
The RT-CAM paper splits generation into two components: an Architect that outputs spatial composition (face locations as XY coordinates) without generating pixels, and an Artist (a fine-tuned diffusion model) that renders photorealistic output from that structure. Both are trained with GRPO, using rewards for composition accuracy, pose, identity, and count. ✦ AI generated
Fati Periqi · The TWIML AI Podcast · 2026-08-12 · original ↗
starts at this moment · 22:21
“Talk us through the RT-CAM paper.”
Instead of asking model to do everything, we separate planning from rendering similar to how a human artist would work. So we have two components there. Architect which is it doesn't generate pixels but instead it creates the structure or composition for the scene. Then it decides where for instance if there's a person is going to be in this image where it should appear and if there are multiple people how they should be arranged you know that it should look natural and realistic then artist starts with that composition structure and generates the final photorealistic image while of course from one objective also preserve identities if we provide identity. So yeah this is like a planning an architect then an artist type of framework.
verbatim transcript · starts at 22:21
22:21this model kind of breaks down the problem of uh text to image generation. Talk us through that. >> So what uh motivated about that one? Disco is great. We as a pivotal example we of course talk about this identity diversity. Um but then uh in the other paper we are making the point that maybe uh it's too much for a model to try to do everything. Going back to you know
22:53agentic flow again uh maybe it's easier to approach some of these red AI challenges like a human being. We do not like try to solve everything ourselves. Well, not just ourselves, but even like even in this domain, you know, art like you an artist approaching this problem wouldn't necessarily think about it pixel by pixel. They think about like if you know what's the subject, what's the background and
23:24>> there's a planning, right? >> There's a planning aspect to it. Yeah. Yeah. >> In this paper, we build on it. Build on that idea. Instead of asking model to do everything, we separate uh planning from rendering similar to how a human artist would work. Um so we have two components there. Architect which is uh it doesn't generate pixels but instead it creates the structure or composition for the
23:52scene. Then uh kind of it decides where for inance if there's a person is going to be in this image where it should appear and if there are multiple people how they should be arranged you know that it should look natural and you know realistic then artist starts with that composition structure and generates the final photorealistic image while of course uh from one objective also preserve identities if we provide
24:19identity. So yeah this is like a um planning an architect then an artist type of framework. >> The underlying technical approach is also using RL just like disco. It's also gpo based approach. >> You are right. Um we also have this reward function leveraging on different objectives that we want to optimize. In this goal we had in intraim image and intra group like interim image at human