Professor Chan-Hyun Youn’s Research Team Developed a Technique to Prevent Abnormal Data Generation in Diffusion Models
<(From left) Professor Chan-Hyun Youn, Jinhyeok Jang Ph.D. candidate, Changha Lee Ph.D. candidate, Minsu Jeon Ph.D. >
Professor Chan-Hyun Youn’s research team from the EE department has developed a momentum-based generation technique to address the issue of abnormal data generation frequently encountered by diffusion model-based generative AI.
While diffusion model-based generative AI, which has recently garnered significant attention, generally produces realistic images, it often generates abnormal details, such as unnaturally bent joints or horses with only three legs.
Figure 1 : The generated images by Stable Diffusion with the proposed technique
To address this problem, the research team reformulated the generative process of diffusion models as an optimization problem, such as gradient descent. Both the generative process of diffusion models and gradient descent can be expressed as a Generalized Expectation-Maximization problem, and visualization revealed the presence of numerous local minima and saddle points in the generative process.
This observation demonstrated that inappropriate outcomes are akin to local minima or saddle points. Based on this insight, the team introduced the widely used momentum technique from optimization into the generative process.
Various experiments confirmed that the generation of inappropriate images significantly decreased without additional training, and the quality of generated images improved even with reduced computational cost. These results suggest a new insight about the generative process of diffusion models as a progressive optimization problem and show that introducing the momentum technique into the generative process reduces inappropriate outcomes.
This new research outcome is expected to not only improve generation results but also provide a new interpretation of generative AI and inspire various follow-up studies. The research findings were presented in February at the 38th Annual AAAI Conference on Artificial Intelligence (AAAI 2024) in Vancouver, Canada, one of the leading international conferences in the AI field, under the title “Rethinking Peculiar Images by Diffusion Models: Revealing Local Minima’s Role.”
