Classifier-free guidance (CFG) is a fundamental tool in modern diffusion models for text-guided generation. Although effective, CFG has notable drawbacks. For instance, DDIM with CFG lacks invertibility, complicating image editing; furthermore, high guidance scales, essential for high-quality outputs, frequently result in issues like mode collapse. Contrary to the widespread belief that these are inherent limitations of diffusion models, this paper reveals that the problems actually stem from the off-manifold phenomenon associated with CFG, rather than the diffusion models themselves. More specifically, inspired by the recent advancements of diffusion model-based inverse problem solvers (DIS), we reformulate text-guidance as an inverse problem with a text-conditioned score matching loss and develop CFG++, a novel approach that tackles the off-manifold challenges inherent in traditional CFG. CFG++ features a surprisingly simple fix to CFG, yet it offers significant improvements, including better sample quality for text-to-image generation, invertibility, smaller guidance scales, reduced mode collapse, etc. Furthermore, CFG++ enables seamless interpolation between unconditional and conditional sampling at lower guidance scales, consistently outperforming traditional CFG at all scales. Moreover, CFG++ can be easily integrated into high-order diffusion solvers and naturally extends to distilled diffusion models. Experimental results confirm that our method significantly enhances performance in text-to-image generation, DDIM inversion, editing, and solving inverse problems, suggesting a wide-ranging impact and potential applications in various fields that utilize text guidance. Project Page: https://cfgpp-diffusion.github.io/.
Rethinking the Spatial Inconsistency in Classifier-Free Diffusion Guidance
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；SenseTime Research（商汤科技研究院）；The University of Hong Kong（香港大学）；The Chinese University of Hong Kong（香港中文大学）
Guidance Matters: Rethinking the Evaluation Pitfall for Text-to-Image Generation
The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；Cogniser Information Technology（Cogniser信息科技）
Adaptive Guidance: Training-free Acceleration of Conditional Diffusion Models
King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学（KAUST））；Center for Research and Formation in Artificial Intelligence, Universidad de los Andes（安第斯大学人工智能研究与培训中心）；GenAI, Meta（Meta 生成式人工智能部门）
Improving Classifier-Free Guidance of Flow Matching via Manifold Projection
Department of Mathematics, The Hong Kong University of Science；IAS Center for AI for Scientific Discoveries, The Hong Kong University of Science；School of Mathematics；Statistics \& Institute of Interdisciplinary Research for Mathematics；Applied Science \& Hubei Key Laboratory of Engineering Modeling；Scientific Computing, Huazhong University of Science；Department of Statistics；Data Science, Southern University of Science
SafeCFG: Controlling Harmful Features with Dynamic Safe Guidance for Safe Generation
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；University of Science and Technology of China（中国科学技术大学）；University of Rochester（罗切斯特大学）
Hexaglobe；EleutherAI；University of Geneva（日内瓦大学）；Information Sciences Institute（信息科学研究所）；University of Southern California（南加州大学）；Sightful；IIIT Hyderabad（海得拉巴国际信息技术研究所）；Booz Allen Hamilton（博思艾伦汉密尔顿）
VAGS: Velocity Adaptive Guidance Scale for Image Editing and Generation
Harvard AI and Robotics Lab（哈佛人工智能与机器人实验室）；Harvard University（哈佛大学）；School of Computing and Data Science（计算与数据科学学院）；The University of Hong Kong（香港大学）；Kempner Institute for the Study of Natural and Artificial Intelligence（自然与人工智能研究学院）
