Recent advances in text-to-image diffusion models have substantially improved the quality of image customization, enabling the synthesis of highly realistic images. Despite this progress, achieving fast and efficient personalization remains a key challenge, particularly for real-world applications. Existing approaches primarily accelerate customization for human subjects by injecting identity-specific embeddings into diffusion models, but these strategies do not generalize well to arbitrary object categories, limiting their applicability. To address this limitation, we propose a novel framework that employs a learned network to predict object-specific textual inversion embeddings, which are subsequently integrated into the UNet timesteps of a diffusion model for text-conditional customization. This design enables rapid, zero-shot personalization of a wide range of objects in a single forward pass, offering both flexibility and scalability. Extensive experiments across multiple tasks and settings demonstrate the effectiveness of our approach, highlighting its potential to support fast, versatile, and inclusive image customization. To the best of our knowledge, this work represents the first attempt to achieve such general-purpose, training-free personalization within diffusion models, paving the way for future research in personalized image generation.
Personalization as a Shortcut for Few-Shot Backdoor Attack against Text-to-Image Diffusion Models
Nanyang Technological University（南洋理工大学）；New York University（纽约大学）；Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局）；East China Normal University（华东师范大学）
ProSpect: Prompt Spectrum for Attribute-Aware Personalization of Diffusion Models
Institute of Automation, CAS（中国科学院自动化研究所）；School of Artificial Intelligence, UCAS（中国科学院大学人工智能学院）；Institute of Computing Technology, CAS（中国科学院计算技术研究所）；Kuaishou Technology（快手科技）；National Cheng-Kung University（国立成功大学）；University of Konstanz（康斯坦茨大学）
Meta-LoRA: Meta-Learning LoRA Components for Domain-Aware ID Personalization
Pixery Labs İstanbul, Türkiye（Pixery实验室伊斯坦布尔，土耳其）；METU Dept. of Computer Engineering（METU计算机工程系）；METU Dept. of Electrical and Elec. Engineering（METU电气与电子工程系）
EDITOR: Effective and Interpretable Prompt Inversion for Text-to-Image Diffusion Models
University of Massachusetts, Amherst（马萨诸塞大学阿姆赫斯特分校）；Georgia Institute of Technology（佐治亚理工学院）；Rutgers University（罗格斯大学）；University of Utah（犹他大学）；Dolby Laboratories（杜比实验室）
Text-to-image models offer unprecedented freedom to guide creation through natural language. Yet, it is unclear how such freedom can be exercised to generate images of specific unique concepts, modify their appearance, or compose them in new roles and novel scenes. In other words, we ask: how can we use language-guided models to turn our cat into a painting, or imagine a new product based on our favorite toy? Here we present a simple approach that allows such creative freedom. Using only 3-5 images of a user-provided concept, like an object or a style, we learn to represent it through new "wor…
InstaStyle: Inversion Noise of a Stylized Image is Secretly a Style Adviser
Beijing University of Posts and Telecommunications（北京邮电大学）；University of California, Santa Barbara（加利福尼亚大学圣巴巴拉分校）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of AI, UCAS（中国科学院大学人工智能学院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Kuaishou Technology（快手科技）
The artistic style within a painting is the means of expression, which includes not only the painting material, colors, and brushstrokes, but also the high-level attributes including semantic elements, object shapes, etc. Previous arbitrary example-guided artistic image generation methods often fail to control shape changes or convey elements. The pre-trained text-to-image synthesis diffusion probabilistic models have achieved remarkable quality, but it often requires extensive textual descriptions to accurately portray attributes of a particular painting. We believe that the uniqueness of an …
LocInv: Localization-aware Inversion for Text-Guided Image Editing
University of Chinese Academy of Sciences（中国科学院大学）；Computer Vision Center（计算机视觉中心）；College of Computer Science（计算机学院）；Nankai University（南开大学）；Universitat Autonoma de Barcelona（巴塞罗那自治大学）
Contextualized Visual Personalization in Vision-Language Models
Department of Electrical and Computer Engineering, Seoul National University, Seoul, South Korea（电气电子工程系，首尔国立大学，首尔，韩国）；Interdisciplinary Program in Artificial Intelligence, Seoul National University, Seoul, Korea（人工智能交叉学科项目，首尔国立大学，首尔，韩国）
Conditioning Matters: Stabilizing Inversion and Attention in Diffusion Image Editing
State Key Laboratory of Blockchain and Data Security, Zhejiang University（浙江大学区块链与数据安全全国重点实验室）；HangZhou High-Tech Zong (Binjiang) Institute of Blockchain and Data Security（杭州高新技术产业开发区（滨江）区块链与数据安全研究院）；College of Computer Science, Zhejiang University（浙江大学计算机科学与技术学院）；University at Buffalo, State University of New York（纽约州立大学布法罗分校）
Compositional Inversion for Stable Diffusion Models
the Hong Kong Polytechnic University（香港理工大学）；Center for Artificial Intelligence and Robotics, HKISI, CAS（中国科学院香港创新研究院人工智能与机器人创新中心）；Sichuan University（四川大学）；UCAS（中国科学院大学）；State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA（中国科学院自动化研究所多模态人工智能系统国家重点实验室）
Google Research（谷歌研究院）；The Blavatnik School of Computer Science, Tel Aviv University（特拉维夫大学布拉瓦尼克计算机科学学院）
Recent text-guided diffusion models provide powerful image generation capabilities. Currently, a massive effort is given to enable the modification of these images using text only as means to offer intuitive and versatile editing. To edit a real image using these state-of-the-art tools, one must first invert the image with a meaningful text prompt into the pretrained model's domain. In this paper, we introduce an accurate inversion technique and thus facilitate an intuitive text-based modification of the image. Our proposed inversion consists of two novel key components: (i) Pivotal inversion …
A Comprehensive Survey on Visual Concept Mining in Text-to-image Diffusion Models
School of Computer Science, Nanjing University of Information Science and Technology（南京信息工程大学计算机学院）；School of Computer Science, Nanjing University of Science and Technology（南京理工大学计算机学院）
