Video · 2026-08-12 · 56m · 11 moments

Why Image Generation Needs More Than Bigger Models | Fatih Porikli

✦ AI generated

GUEST Sam Sherington GUEST Fatih Porikli

timeline · colored by role

Claim◆

Looking good and being correct are not the same thing in text-to-image generation — models can produce photorealistic images while failing to follow specific instructions about composition, identity, number of subjects, or diversity.

Text-to-image models have reached impressive visual quality, but they still fail at precision tasks like generating distinct faces, maintaining correct subject counts, and following specific compositional instructions.

transcript

Sam Sherington: Once a frontier research problem in computer vision, text to image generation has reached a point where almost anyone can ask an AI system for a picture and get something that looks remarkably good. But looking good and being correct are not the same thing. Ask for several different people and the model may generate variations of the same face. Ask for a specific composition, identity or number of subjects, and it may ignore those details.

Claim◆

Current image generation models may be doing too much by asking a single model to simultaneously handle all aspects of scene creation — from understanding prompts and determining subject count to composition, identity preservation, and rendering.

Fatih argues that image generation models are overloaded — a single model is expected to parse prompts, decide subject count, compose scenes, reason about interactions, preserve identities, and render everything at once.

transcript

Fatih Porikli: Maybe we are also asking a single model to solve too many difficult problems at once. Think about what happens when you generate a scene with several people. Now the model has to understand the problem that I'm asking to model and then decide how many people should appear, determine where they should be placed like the composition of the scene, reason about their interactions because if there's a person, if there's another person most likely there is some connection, preserve the identity of the person — we can give okay this is my daughter this is my son and I want them to be in the picture not like any random person — and finally render everything together in all a single process. So maybe this is too much.

supports · 2

Mechanism◆

Image generation models produce identical faces because training objectives focus on realism and prompt matching but do not explicitly encourage identity diversity — adding diversity as an optimization objective via reinforcement learning solves this.

DISCO addresses the problem of models generating identical faces by adding diversity as an explicit optimization objective through reinforcement learning, achieving 98-99% unique face accuracy versus much lower scores from base models.

transcript

Fatih Porikli: This is not an image quality problem, the problem that I mentioned before like we are asking the model to generate faces and certain number of faces and it keeps generating same faces identical almost identical faces over and over again. So quality wise, image quality wise, if you look at the pixels and noise and everything, it looks realistic. But the missing piece was that those models, base models, amazing models had not really learned to create truly distinct identities because existing training objectives focus heavily on the realism and matching the user prompt, but they don't explicitly encourage diversity between people and that is very important. So that observation led us to a simple question what if the identity or facial appearance or any diversity itself becomes an optimization objective.

provides context · 2

Claim◆

Text-to-image models fail to generate distinct identities when producing multiple people because existing training objectives focus on realism and prompt matching but do not explicitly encourage diversity.

Fatih Porikli explains that despite high pixel-level quality, text-to-image models generate nearly identical faces for multiple people because their training objectives prioritize realism over inter-person diversity.

transcript

Fatih Porikli: this is not an image quality problem the problem that I mentioned before like we are asking the model to generate faces and and certain number of faces and it keeps generating same faces identical almost identical faces over and over again. So quality wise, image quality wise, if you look at the pixels and noise and everything, it looks realistic. But the missing piece was that those models, base models, amazing models had not really learned to create truly distinct identities because existing training objectives focus heavily on the realism and matching the user prompt, but they don't explicitly encourage diversity between people and that is very important.

Claim

The image generation community may be asking single models to solve too many problems at once, and a broader lesson is that models fail not because of architecture limitations but because of how objectives and training data are provided.

Porikli reflects that many image generation failures stem from missing or misaligned training objectives rather than architectural shortcomings, and suggests the community should reconsider whether single models should attempt to solve everything.

transcript

Fatih Porikli: a broader lesson from the disco paper sometimes the model simply miss the right objective in training... and also it is not mainly an architecture limitation but it is how you are providing this objective and training data to the algorithm.

extends · 1gives example · 4supports · 1

Mechanism◆

Asking a single model to simultaneously handle prompt understanding, scene composition, interaction reasoning, identity preservation, and rendering for multi-person scenes is too much; separating planning from rendering improves results.

The R2-Canvas paper separates image generation into an Architect (planning composition) and Artist (rendering pixels), mimicking how human artists approach scene creation rather than doing everything in one pass.

transcript

Fatih Porikli: maybe we are also asking a single model to solve too many difficult problems at once. Think about what happens when you generate a scene with several people. The model has to understand the prompt, decide how many people should appear, determine where they should be placed like the composition, reason about their interactions, preserve the identity of the person... and finally render everything together in a single process. Maybe this is too much. Instead of asking one model to do everything what if we separated planning from rendering similar to what human artists might do.

extends · 1provides context · 2

Mechanism

Separating planning from rendering in image generation — using an architect to determine composition and an artist to generate pixels — mirrors how human artists work and produces more natural, controllable results.

R2C splits image generation into two stages: an architect that plans composition and subject placement, and an artist that renders the final image, allowing each component to be independently optimized for better results.

transcript

Fatih Porikli: In this paper, we build on that idea. Instead of asking model to do everything, we separate planning from rendering similar to how a human artist would work. So we have two components there. Architect which doesn't generate pixels but instead it creates the structure or composition for the scene. Then it decides where for instance if there's a person is going to be in this image where it should appear and if there are multiple people how they should be arranged, that it should look natural and realistic then artist starts with that composition structure and generates the final photorealistic image while of course from one objective also preserve identities if we provide identity.

provides context · 2

Mechanism

High-resolution image generation (4-16 megapixels) can be achieved efficiently by generating a base image, upsampling it, then refining latent space patches with guided semantic noise — running 35x faster than full-resolution approaches.

PixelRush achieves high-resolution (4-16MP) image generation by cascading base generation with upsampling and latent-space patchification, achieving 35x speedup while maintaining quality through guided semantic noise injection.

transcript

Fatih Porikli: Now we are saying that can we even push it to the next level because before we have been talking about let's say 1K resolution and that's the current sort of even the cloud models are kind of limited to that resolution. But the question is now can we do like 4 megapixel image generation, 16 megapixel image generation. The challenge is not only how fast you can run the model... but there's a memory challenge also because when the image resolution gets larger we need to retain this diffusion process the latent features in the memory somewhere on the device... when I say much faster, it's not like two times faster. It's maybe 35 times faster. From let's say 10 minutes to around 20 seconds type of acceleration.

Mechanism

High-resolution image generation (4-16 megapixels) on memory-constrained mobile devices requires patch-based latent space processing rather than scaling the entire latent space proportionally to output resolution.

PixelRush enables 4-16 megapixel generation on mobile devices by generating a base 1K image, upsampling it, then processing patches independently in latent space with guided semantic noise, achieving 35x speedup (10 minutes to 20 seconds).

transcript

Fatih Porikli: we are saying that can we even push it to the next level because before we have been talking about let's say 1K resolution... but the question is now can we do like 4 megapixel image generation, 16 megapixel image generation. The challenge is not only how fast you can run the model... but there's a memory challenge also because when the image resolution gets larger we need to retain this diffusion process the latent features in the memory somewhere on the device... we partition the input image into like original size patches let's say I like 1K 1K image. Now I have like 4K 4K... so I divide it into 16 parts... we allow generating noise in the background... it's maybe 35 times faster.

provides context · 2

Mechanism

Image inpainting models produce visible boundary artifacts because they initialize only the masked region with random noise, losing coherence with the surrounding image context.

InverField improves inpainting by first inverting the entire clean image to its noise representation, then adding new random noise only in the masked region, enabling seamless harmonization without boundary artifacts.

transcript

Fatih Porikli: I can still see you create the texture sometimes okay it is meaningful but then there's those artifacts that you see in the background like the sand on the beach where you remove the person who didn't need to be in the picture is kind of funky looking and maybe the texture is not really compliant with the rest of the image or I see literally the artifacts around the boundary. We can take this input image and map into noise. So we are progressively inverting a real clean image into noisy versions. This noise is not random noise anymore. It is specific to the input image. So if I change the input image the noise is going to be different. Now I added noise there for the bird because I want to allow algorithm to generate a new bird compliant with my text prompt... there's no boundary artifacts anymore because we have two noise like no random noise and inverted noise.

provides context · 2

Mechanism

Image inpainting can eliminate boundary artifacts by using inverted noise (noise specific to the input image) rather than random noise in the masked region, allowing seamless harmonization between retained background and generated content.

InverField improves inpainting by inverting the input image to its noise representation, then using that structure-guided noise plus new noise in the mask region, avoiding boundary artifacts that plague random-noise approaches.

transcript

Fatih Porikli: We can take this input image and map into noise. So we are progressively inverting a real clean image into noisy versions... this noise is not random noise anymore. It is specific to the input image. So if I change the input image the noise is going to be different... Now I changed the way that we create this image. But I don't need to change the original model... when we start with this inverted noise plus the mask and new noise within the mask. Now first of all we can retain background but we also allow background to slightly impact the foreground like the mask itself. Allows seamless harmonization and it generates high quality images but most importantly there's no boundary artifacts anymore because we have two noise like no random noise and inverted noise within the same when we started the image we are not starting from just noise within the mask.

Highlight slides

Visual Quality ≠ Accuracy in Text-to-Image✦ from: Looking good and being correct are not the same thing in text-to-image generation — models can produce photorealistic images while failing to follow specific instructions about composition, identity, number of subjects, or diversity.Precision Failures Persist✦ from: Looking good and being correct are not the same thing in text-to-image generation — models can produce photorealistic images while failing to follow specific instructions about composition, identity, number of subjects, or diversity.The Gap Between Quality and Fidelity✦ from: Looking good and being correct are not the same thing in text-to-image generation — models can produce photorealistic images while failing to follow specific instructions about composition, identity, number of subjects, or diversity.Image Generation Models Are Overloaded✦ from: Current image generation models may be doing too much by asking a single model to simultaneously handle all aspects of scene creation — from understanding prompts and determining subject count to composition, identity preservation, and rendering.What One Model Must Handle at Once✦ from: Current image generation models may be doing too much by asking a single model to simultaneously handle all aspects of scene creation — from understanding prompts and determining subject count to composition, identity preservation, and rendering.The Core Question✦ from: Current image generation models may be doing too much by asking a single model to simultaneously handle all aspects of scene creation — from understanding prompts and determining subject count to composition, identity preservation, and rendering.The Identity Problem✦ from: Text-to-image models fail to generate distinct identities when producing multiple people because existing training objectives focus on realism and prompt matching but do not explicitly encourage diversity.The Identical Face Problem✦ from: Image generation models produce identical faces because training objectives focus on realism and prompt matching but do not explicitly encourage identity diversity — adding diversity as an optimization objective via reinforcement learning solves this.Root Cause: Training Objectives✦ from: Text-to-image models fail to generate distinct identities when producing multiple people because existing training objectives focus on realism and prompt matching but do not explicitly encourage diversity.Root Cause: Missing Diversity Objective✦ from: Image generation models produce identical faces because training objectives focus on realism and prompt matching but do not explicitly encourage identity diversity — adding diversity as an optimization objective via reinforcement learning solves this.DISCO: Adding Diversity as an Objective✦ from: Image generation models produce identical faces because training objectives focus on realism and prompt matching but do not explicitly encourage identity diversity — adding diversity as an optimization objective via reinforcement learning solves this.The Problem: Too Much for One Model✦ from: Asking a single model to simultaneously handle prompt understanding, scene composition, interaction reasoning, identity preservation, and rendering for multi-person scenes is too much; separating planning from rendering improves results.The Solution: Separate Planning from Rendering✦ from: Asking a single model to simultaneously handle prompt understanding, scene composition, interaction reasoning, identity preservation, and rendering for multi-person scenes is too much; separating planning from rendering improves results.

Related episodes

Video · The Cognitive Revolution

Intelligence on the Edge: Liquid AI's Ramin Hasani on the Search for Device-Native Foundation Models

→

Video · The Cognitive Revolution

LTX CEO on Video Gen + Can we beat AI superforecasters?

→

Article · Interconnects

Kimi K3: The open-weights escalation

→