Diffusion models achieve remarkable fidelity in image synthesis, yet precise control over their outputs for targeted editing remains challenging. A key step toward controllability is to identify interpretable directions in the model's latent representations that correspond to semantic attributes. Existing approaches for finding interpretable directions typically rely on sampling large sets of images or training auxiliary networks, which limits efficiency. We propose an analytical method that derives semantic editing directions directly from the pretrained parameters of diffusion models, requiring neither additional data nor fine-tuning. Our insight is that self-attention weight matrices encode rich structural information about the data distribution learned during training. By computing the eigenvectors of these weight matrices, we obtain robust and interpretable editing directions. Experiments demonstrate that our method produces high-quality edits across multiple datasets while reducing editing time significantly by 60% over current benchmarks.
Diffusion models have shown remarkable capabilities in generating high quality and creative images conditioned on text. An interesting application of such models is structure preserving text guided image editing. Existing approaches rely on text conditioned diffusion models such as Stable Diffusion or Imagen and require compute intensive optimization of text embeddings or fine-tuning the model weights for text guided image editing. We explore text guided image editing with a Hybrid Diffusion Model (HDM) architecture similar to DALLE-2. Our architecture consists of a diffusion prior model that …
Image generation has recently seen tremendous advances, with diffusion models allowing to synthesize convincing images for a large variety of text prompts. In this article, we propose DiffEdit, a method to take advantage of text-conditioned diffusion models for the task of semantic image editing, where the goal is to edit an image based on a text query. Semantic image editing is an extension of image generation, with the additional constraint that the generated image should be as similar as possible to a given input image. Current editing methods based on diffusion models usually require to pr…
Peking University（北京大学）；The Hong Kong University of Science and Technology（香港科技大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
HairDiffusion: Vivid Multi-Colored Hair Editing via Latent Diffusion
School of Computer Science & Software Engineering, Shenzhen University（深圳大学计算机与软件学院）；Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳市人工智能与机器人研究院）；National Engineering Laboratory for Big Data System Computing Technology, Shenzhen University（深圳大学国家大数据系统计算技术工程实验室）；Guangdong Provincial Key Laboratory of Intelligent Information Processing（广东省智能信息处理重点实验室）
MADiff: Text-Guided Fashion Image Editing with Mask Prediction and Attention-Enhanced Diffusion
Southeast University（东南大学）；Advanced Ocean Institute of Southeast University（东南大学先进海洋研究院）；Northwestern Polytechnical University（西北工业大学）；Binjiang Institute of Artificial Intelligence, ZJUT（浙江工业大学滨江人工智能研究院）；Alibaba Group（阿里巴巴集团）
University of Science and Technology of China（中国科学技术大学）；Microsoft Research Asia（微软亚洲研究院）
Language-guided image editing has achieved great success recently. In this paper, for the first time, we investigate exemplar-guided image editing for more precise control. We achieve this goal by leveraging self-supervised training to disentangle and re-organize the source image and the exemplar. However, the naive approach will cause obvious fusing artifacts. We carefully analyze it and propose an information bottleneck and strong augmentations to avoid the trivial solution of directly copying and pasting the exemplar image. Meanwhile, to ensure the controllability of the editing process, we…
Task-Oriented Diffusion Inversion for High-Fidelity Text-based Editing
The University of Hong Kong（香港大学）；Shanghai AI Lab（上海人工智能实验室）；Ocean University of China（中国海洋大学）；Singapore Management University（新加坡管理大学）；South China University of Technology（华南理工大学）
MotionEditor: Editing Video Motion via Content-Aware Diffusion
Fudan University（复旦大学）；Shanghai Collaborative Innovation Center of Intelligent Visual Computing（上海智能视觉计算协同创新中心）；Microsoft Research Asia（微软亚洲研究院）；Carnegie Mellon University（卡内基梅隆大学）；Huya Inc.（虎牙公司）
AttentionDrag: Exploiting Latent Correlation Knowledge in Pre-trained Diffusion Models for Image Editing
Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University（上海数据科学关键实验室，计算机科学学院，复旦大学）；Rajax Network Technology (ele.me), Alibaba Group（RAJAX网络技术（饿了么），阿里巴巴集团）
