Large-scale Text-to-Image (T2I) diffusion models have revolutionized image generation over the last few years. Although owning diverse and high-quality generation capabilities, translating these abilities to fine-grained image editing remains challenging. In this paper, we propose DiffEditor to rectify two weaknesses in existing diffusion-based image editing: (1) in complex scenarios, editing results often lack editing accuracy and exhibit unexpected artifacts; (2) lack of flexibility to harmonize editing operations, e.g., imagine new content. In our solution, we introduce image prompts in fine-grained image editing, cooperating with the text prompt to better describe the editing content. To increase the flexibility while maintaining content consistency, we locally combine stochastic differential equation (SDE) into the ordinary differential equation (ODE) sampling. In addition, we incorporate regional score-based gradient guidance and a time travel strategy into the diffusion sampling, further improving the editing quality. Extensive experiments demonstrate that our method can efficiently achieve state-of-the-art performance on various fine-grained image editing tasks, including editing within a single image (e.g., object moving, resizing, and content dragging) and across images (e.g., appearance replacing and object pasting). Our source code is released at https://github.com/MC-E/DragonDiffusion.
DCI: Dual-Conditional Inversion for Boosting Diffusion-Based Image Editing
Institute of Information Science, Beijing Jiaotong University（信息科学学院，北京交通大学）；Visual Intelligence +X International Cooperation Joint Laboratory of MOE（视觉智能+X国际合作联合实验室）
ByteDance Inc（字节跳动公司）；NLPR, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所模式识别国家重点实验室）
The explosion of short videos has dramatically reshaped the manners people socialize, yielding a new trend for daily sharing and access to the latest information. These rich video resources, on the one hand, benefited from the popularization of portable devices with cameras, but on the other, they can not be independent of the valuable editing work contributed by numerous video creators. In this paper, we investigate a novel and practical problem, namely audio beat matching (ABM), which aims to recommend the proper transition time stamps based on the background music. This technique helps to e…
StreamEdit: Training-Free Video Editing via Few-Step Streaming Video Generation
The University of British Columbia（不列颠哥伦比亚大学）；ETH Zürich（苏黎世联邦理工学院）；McMaster University（麦马斯特大学）；Vector Institute（向量研究所）；Canada CIFAR AI Chair（加拿大 CIFAR 人工智能主席）
SpatialReward: Bridging the Perception Gap in Online RL for Image Editing via Explicit Spatial Reasoning
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；The Hong Kong University of Science and Technology（香港科学与技术大学）；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生学院，清华大学）
Google Research（谷歌研究院）；The Hebrew University of Jerusalem（希伯来大学）
Text-driven image and video diffusion models have recently achieved unprecedented generation realism. While diffusion models have been successfully applied for image editing, very few works have done so for video editing. We present the first diffusion-based method that is able to perform text-based motion and appearance editing of general videos. Our approach uses a video diffusion model to combine, at inference time, the low-resolution spatio-temporal information from the original video with new, high resolution information that it synthesized to align with the guiding text prompt. As obtain…
First Creating Backgrounds Then Rendering Texts: A New Paradigm for Visual Text Blending
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；College of Computer Science, Nankai University（南开大学计算机学院）
Latent Bias Alignment for High-Fidelity Diffusion Inversion in Real-World Image Reconstruction and Manipulation
Department of Electrical and Electronic Engineering, Southern University of Science and Technology（南方科技大学电子与电气工程系）；Undergraduate School of Artificial Intelligence, Shenzhen Polytechnic University（深圳理工大学人工智能本科生学院）；Huawei Technologies Co., Ltd.（华为技术有限公司）；Pengcheng Lab（鹏城实验室）
Rethinking Driving World Model as Synthetic Data Generator for Perception Tasks
Peking University（北京大学）；Xiaomi EV（小米电动车）；Huazhong University of Science and Technology（华中科技大学）；Beijing Key Laboratory of Data Intelligence and Security (Peking University)（北京数据智能与安全重点实验室（北京大学））；Zhongguancun Academy（中关村学院）
