Explicitly optimizing for identity diversity as a reinforcement learning objective — including both intra-image diversity (distinct faces within one image) and inter-image diversity (different faces across runs) — dramatically improves facial distinctness without requiring large datasets.

7:38becomes an optimization objective. So that led into the idea behind disco. Instead of you know creating a completely new T2I model, we kept the underlying model and fine-tune it with reinforcement learning. Um we designed rewards that encourage several things simultaneously. For instance, different people within an image should have distinct identities. You don't want to, you know, kind of duplicate create duplicate faces. That's something we call in the paper as intraim image

8:13diversity and across different runs of the same model uh with similar prompts same prompt uh we should not generate uh we should not keep producing generating the same faces. So this is interr run inter image diversity. So these are explicit uh new objectives when we train fine-tune the model and also you know we want model to uh generate the correct number of people if I'm asking generate

8:42two people that should be two not three uh we are ex also you know kind of incorporating that thing in the objective and of course we still have the previous uh you know uh image quality objective there there so putting all together Um uh to put everything all together we use reinforcement learning something called as group p relative uh policy optimization gr. Taking a step back, it strikes me that

9:13you know you, yes, this as an objective, you know, diversity of faces is uh an important one if you're generating images with multiple faces, but it strikes me as, you know, one of many, you know, possible ways that you might want to or characteristics of an image that you might want to influence. And it seems like tuning the objective for all of the possible ways that you might want images

9:46to kind of generate correctly seems uh you know not just difficult but like anti-bitter lesson like is it you know why not just collect more data with lots of faces and train the model with better data >> in a way that that is what we are doing but we are we show that you do not need a lot of data and you are right you know kind of there might be many attributes

10:11diversity facial you know kind of uh diversity is one objective uh number of people is another one but then let's say we want to generate a certain action of a person or pose of the face or location of the person in the image or knock out their um body pose and everything so for those um but we are saying that well you can incorporate such objectives in

10:40addition to overall let's say perception or image quality uh objective when we are training such models but you do not need a lot of data if you the knowledge of the paper for in this work have to incorporate incorporate everything into a reinforcement learning framework uh that would make it uh possible and training fine-tuning such models would be also uh kind of very uh affordable. By the way, there is a difference

11:12between the quality of the input images going into such models. There is another thing called this curriculum learning starting from simpler scenes and gradually increasing complexity makes learning this reinforcement learning much more stable. I remember conversations I've had, you know, years, many years ago about curriculum learning and it was always a, you know, theoretical improvement. It's exciting to see it being incorporated into uh practical algorithms, training