Currently, personalized image generation methods mostly require considerable time to finetune and often overfit the concept resulting in generated images that are similar to custom concepts but difficult to edit by prompts. We propose an effective and fast approach that could balance the text-image consistency and identity consistency of the generated image and reference image. Our method can generate personalized images without any fine-tuning while maintaining the inherent text-to-image generation ability of diffusion models. Given a prompt and a reference image, we merge the custom concept into generated images by manipulating cross-attention and self-attention layers of the original diffusion model to generate personalized images that match the text description. Comprehensive experiments highlight the superiority of our method.
Tuning-Free Visual Customization via View Iterative Self-Attention Control
Tsinghua University（清华大学）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；Nanchang University（南昌大学）；Jiluan Academy, Nanchang University（南昌大学吉鸾学院）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）
针对 LoRA 等方法依赖训练和多张参考图的问题，本文提出免训练的 VisCtrl，通过 DDIM 反演与迭代自注意力特征注入，实现单参考图的图像、视频及 3D 场景个性化编辑。
Faster Diffusion: Rethinking the Role of the Encoder for Diffusion Model Inference
Nankai University（南开大学）；Computer Vision Center（计算机视觉中心）；Universitat Autònoma de Barcelona（巴塞罗那自治大学）；Mohamed bin Zayed University of AI（穆罕默德·本·扎耶德人工智能大学）；Linkoping University（林雪平大学）
