Compositional Inversion for Stable Diffusion Models
the Hong Kong Polytechnic University（香港理工大学）；Center for Artificial Intelligence and Robotics, HKISI, CAS（中国科学院香港创新研究院人工智能与机器人创新中心）；Sichuan University（四川大学）；UCAS（中国科学院大学）；State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA（中国科学院自动化研究所多模态人工智能系统国家重点实验室）
Inversion methods, such as Textual Inversion, generate personalized images by incorporating concepts of interest provided by user images. However, existing methods often suffer from overfitting issues, where the dominant presence of inverted concepts leads to the absence of other desired concepts. It stems from the fact that during inversion, the irrelevant semantics in the user images are also encoded, forcing the inverted concepts to occupy locations far from the core distribution in the embedding space. To address this issue, we propose a method that guides the inversion process towards the core distribution for compositional embeddings. Additionally, we introduce a spatial regularization approach to balance the attention on the concepts being composed. Our method is designed as a post-training approach and can be seamlessly integrated with other inversion methods. Experimental results demonstrate the effectiveness of our proposed approach in mitigating the overfitting problem and generating more diverse and balanced compositions of concepts in the synthesized images. The source code is available at https://github.com/zhangxulu1996/Compositional-Inversion.
Stable and Near-Reversible Diffusion ODE Solvers for Image Editing
Department of Computing, Imperial College London, London, United Kingdom（帝国理工学院伦敦分校计算机系，伦敦，英国）；Department of Mathematics, Imperial College London, London, United Kingdom（帝国理工学院伦敦分校数学系，伦敦，英国）
University of California, Santa Barbara（加州大学圣巴巴拉分校）；University of California, Santa Cruz（加州大学圣克鲁兹分校）；Google（谷歌）
Large-scale diffusion models have achieved state-of-the-art results on text-to-image synthesis (T2I) tasks. Despite their ability to generate high-quality yet creative images, we observe that attribution-binding and compositional capabilities are still considered major challenging issues, especially when involving multiple objects. In this work, we improve the compositional skills of T2I models, specifically more accurate attribute binding and better image compositions. To do this, we incorporate linguistic structures with the diffusion guidance process based on the controllable properties of …
Google Research（谷歌研究院）；The Blavatnik School of Computer Science, Tel Aviv University（特拉维夫大学布拉瓦尼克计算机科学学院）
Recent text-guided diffusion models provide powerful image generation capabilities. Currently, a massive effort is given to enable the modification of these images using text only as means to offer intuitive and versatile editing. To edit a real image using these state-of-the-art tools, one must first invert the image with a meaningful text prompt into the pretrained model's domain. In this paper, we introduce an accurate inversion technique and thus facilitate an intuitive text-based modification of the image. Our proposed inversion consists of two novel key components: (i) Pivotal inversion …
Evaluating Design Video Generation: Metrics for Compositional Fidelity
Lica World（Lica世界）；San Francisco, United States of America（美国旧金山）；ICML’26 Workshop on Human-AI Co-Creativity, Seoul, South Korea（ICML’26 人类-人工智能协同创作研讨会，韩国首尔）
Semantic 3D-aware Portrait Synthesis and Manipulation Based on Compositional Neural Radiance Field
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；ByteDance Ltd（字节跳动公司）；Nanjing University（南京大学）
University of Southern California（南加州大学）；Microsoft Research（微软研究院）
We propose a new paradigm to automatically generate training data with accurate labels at scale using the text-toimage synthesis frameworks (e.g., DALL-E, Stable Diffusion, etc.). The proposed approach decouples training data generation into foreground object mask generation and background (context) image generation. For foreground object mask generation, we use a simple textual template with object class name as input to DALL-E to generate a diverse set of foreground images. A foreground-background segmentation algorithm is then used to generate foreground object masks. Next, in order to gene…
