While generative models produce high-quality images of concepts learned from a large-scale database, a user often wishes to synthesize instantiations of their own concepts (for example, their family, pets, or items). Can we teach a model to quickly acquire a new concept, given a few examples? Furthermore, can we compose multiple new concepts together? We propose Custom Diffusion, an efficient method for augmenting existing text-to-image models. We find that only optimizing a few parameters in the text-to-image conditioning mechanism is sufficiently powerful to represent new concepts while enabling fast tuning (~6 minutes). Additionally, we can jointly train for multiple concepts or combine multiple fine-tuned models into one via closed-form constrained optimization. Our fine-tuned model generates variations of multiple new concepts and seamlessly composes them with existing concepts in novel settings. Our method outperforms or performs on par with several baselines and concurrent works in both qualitative and quantitative evaluations while being memory and computationally efficient.
VMC: Video Motion Customization using Temporal Attention Adaption for Text-to-Video Diffusion Models
Kim Jaechul Graduate School of AI（金宰澈人工智能研究生院）；Bio and Brain Engineering（生物与脑工程系）；Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院（KAIST））
Crafting Your Evolving Dreams: Concept-Incremental Versatile Customization
Mohamed bin Zayed University of Artificial Intelligence（Mohamed bin Zayed大学人工智能学院）；University of Trento（特伦托大学）；Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University（香港理工大学土木与环境工程系）；South China University of Technology（华南理工大学）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Institute of Big Data, Fudan University（复旦大学大数据研究院）；Shanghai Jiao Tong University（上海交通大学）
Towards Robust Defense against Customization via Protective Perturbation Resistant to Diffusion-based Purification
MAIS & NLPR, Institute of Automation, Chinese Academy of Sciences（自动化研究所、中国科学院）；School of Artificial Intelligence, University of Chinese Academy of Sciences（人工智能学院、中国科学院大学）
Disrupting Diffusion: Token-Level Attention Erasure Attack against Diffusion-based Customization
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
Diff-PC: Identity-preserving and 3D-aware Controllable Diffusion for Zero-shot Portrait Customization
School of Electronic Science and Engineering, Nanjing University（南京大学电子科学与工程学院）；School of Business, Hohai University（河海大学商学院）；School of Artificial Intelligence, Nanjing University of Information Science and Technology（南京信息工程大学人工智能学院）
How to Continually Adapt Text-to-Image Diffusion Models for Flexible Customization?
Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；The Hong Kong Polytechnic University（香港理工大学）；Fudan University（复旦大学）；Australian National University（澳大利亚国立大学）；Linköping University（林雪平大学）；Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所）
MotionCrafter: One-Shot Motion Customization of Diffusion Models
MAIS, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所多模态人工智能系统全国重点实验室）；School of AI, UCAS（中国科学院大学人工智能学院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Kuaishou Technology（快手科技）
LILAC: Layer-Wise Independent LoRAs and Cascaded Conditioning for Multi-Concept Customization of Diffusion Models
Adobe Research, Romania（罗马尼亚Adobe研究院）；Department of Computer Science, University of Bucharest, Romania（罗马尼亚布加勒斯特大学计算机科学系）；International Computer High School of Bucharest, Romania（罗马尼亚布加勒斯特国际计算机高中）
Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluation
University of Toronto（多伦多大学）；Yonsei University（延世大学）；University of Texas at San Antonio（德克萨斯大学圣安东尼奥分校）；National Tsing Hua University（国立清华大学）；Apple（苹果公司）
MotionAdapter: Video Motion Transfer via Content-Aware Attention Customization
Hangzhou Dianzi University（杭州电子科技大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；The Hong Kong University of Science and Technology（香港科技大学）；Ocean University of China（中国海洋大学）；Singapore Management University（新加坡管理大学）
SketchAnimator: Animate Sketch via Motion Customization of Text-to-Video Diffusion Models
PRIS, School of Artificial Intelligence, Beijing University of Posts（PRIS人工智能学院，北京邮电大学）；SketchX, CVSSP, University of Surrey（SketchX，计算机视觉与模式识别研究所，萨里大学）
Multi-party Collaborative Attention Control for Image Customization
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Department of Radiology, The First Affiliated Hospital of Army Medical University（中国人民解放军军医大学第一附属医院放射科）
Delving into the Temporal Challenges of Unified Video Protection Against Image-to-Video and Fine-Tuning-based Customization
The University of Sydney（悉尼大学）；University of Melbourne（墨尔本大学）；City University of Hong Kong（香港城市大学）；Wuhan University（武汉大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
PersonaMagic: Stage-Regulated High-Fidelity Face Customization with Tandem Equilibrium
Ocean University of China（中国海洋大学）；Shanghai Jiao Tong University（上海交通大学）；Singapore Management University（新加坡管理大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；South China University of Technology（华南理工大学）
