We present a diffusion-based video editing framework, namely DiffusionAtlas, which can achieve both frame consistency and high fidelity in editing video object appearance. Despite the success in image editing, diffusion models still encounter significant hindrances when it comes to video editing due to the challenge of maintaining spatiotemporal consistency in the object's appearance across frames. On the other hand, atlas-based techniques allow propagating edits on the layered representations consistently back to frames. However, they often struggle to create editing effects that adhere correctly to the user-provided textual or visual conditions due to the limitation of editing the texture atlas on a fixed UV mapping field. Our method leverages a visual-textual diffusion model to edit objects directly on the diffusion atlases, ensuring coherent object identity across frames. We design a loss term with atlas-based constraints and build a pretrained text-driven diffusion model as pixel-wise guidance for refining shape distortions and correcting texture deviations. Qualitative and quantitative experiments show that our method outperforms state-of-the-art methods in achieving consistent high-fidelity video-object editing.
Task-Oriented Diffusion Inversion for High-Fidelity Text-based Editing
The University of Hong Kong（香港大学）；Shanghai AI Lab（上海人工智能实验室）；Ocean University of China（中国海洋大学）；Singapore Management University（新加坡管理大学）；South China University of Technology（华南理工大学）
We present a novel image inversion framework and a training pipeline to achieve high-fidelity image inversion with high-quality attribute editing. Inverting real images into StyleGAN's latent space is an extensively studied problem, yet the trade-off between the image reconstruction fidelity and image editing quality remains an open challenge. The low-rate latent spaces are limited in their expressiveness power for high-fidelity reconstruction. On the other hand, high-rate latent spaces result in degradation in editing quality. In this work, to achieve high-fidelity inversion, we learn residua…
Tuning-Free Image Editing with Fidelity and Editability via Unified Latent Diffusion Model
Communication University of China（中国传媒大学）；National University of Singapore（新加坡国立大学）；University of California at Merced（加州大学默塞德分校）；Yonsei University（延世大学）
EditScore: Unlocking Online RL for Image Editing via High-Fidelity Reward Modeling
University of Science and Technology of China（中国科学技术大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；Zhejiang University（浙江大学）
Catalyst4D: High-Fidelity 3D-to-4D Scene Editing via Dynamic Propagation
State Key Laboratory of Complex and Critical Software Environment, Beihang University（北京航空航天大学复杂关键软件环境国家重点实验室）；School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）；School of Artificial Intelligence, Beihang University（北京航空航天大学人工智能学院）；Zhejiang Industrial Big Data and Robot Intelligent System Key Laboratory, Hangzhou Innovation Institute, Beihang University（北京航空航天大学杭州创新研究院浙江省工业大数据与机器人智能系统重点实验室）
MagicFace: High-Fidelity Facial Expression Editing with Action-Unit Control
Center for Machine Vision and Signal Analysis, Faculty of Information Technology and Electrical Engineering, University of Oulu（机器视觉与信号分析中心，信息科技与电气工程学院，奥卢大学）
Edit fidelity is a significant issue in open-world controllable generative image editing. Recently, CLIP-based approaches have traded off simplicity to alleviate these problems by introducing spatial attention in a handpicked layer of a StyleGAN. In this paper, we propose CoralStyleCLIP, which incorporates a multi-layer attention-guided blending strategy in the feature space of StyleGAN2 for obtaining high-fidelity edits. We propose multiple forms of our co-optimized region and layer selection strategy to demonstrate the variation of time complexity with the quality of edits over different arc…
GlyphMastero: A Glyph Encoder for High-Fidelity Scene Text Editing
MT Lab, Meitu Inc.（美图公司MT实验室）；Department of Computer Science and Technology, BNRist, IDG/McGovern Institute for Brain Research, Tsinghua University（清华大学计算机科学与技术系）
Alibaba DAMO Academy（阿里巴巴达摩院）；Australian Artificial Intelligence Institute, UTS（悉尼科技大学澳大利亚人工智能研究所）
The fashion industry has diverse applications in multi-modal image generation and editing. It aims to create a desired high-fidelity image with the multi-modal conditional signal as guidance. Most existing methods learn different condition guidance controls by introducing extra models or ignoring the style prior knowledge, which is difficult to handle multiple signal combinations and faces a low-fidelity problem. In this paper, we adapt both style prior knowledge and flexibility of multi-modal control into one unified two-stage framework, M6-Fashion, focusing on the practical AI-aided Fashion …
Institute of Information Science, Beijing Jiaotong University（信息科学学院，北京交通大学）；Visual Intelligence +X International Cooperation Joint Laboratory of MOE（视觉智能+X国际合作联合实验室）；MT Lab, Meitu Inc（美图实验室，美图公司）；Tsinghua University（清华大学）
VideoDirector: Precise Video Editing via Text-to-Video Models
Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；Sun Yat-sen University（中山大学）；Tsinghua University（清华大学）；National University of Defense Technology（国防科技大学）
Hong Kong University of Science and Technology（香港科技大学）；Fudan University（复旦大学）
GAN inversion and editing via StyleGAN maps an input image into the embedding spaces ($\mathcal{W}$, $\mathcal{W^+}$, and $\mathcal{F}$) to simultaneously maintain image fidelity and meaningful manipulation. From latent space $\mathcal{W}$ to extended latent space $\mathcal{W^+}$ to feature space $\mathcal{F}$ in StyleGAN, the editability of GAN inversion decreases while its reconstruction quality increases. Recent GAN inversion methods typically explore $\mathcal{W^+}$ and $\mathcal{F}$ rather than $\mathcal{W}$ to improve reconstruction fidelity while maintaining editability. As $\mathcal{W^…
ResetEdit: Precise Text-guided Editing of Generated Image via Resettable Starting Latent
Shanghai Jiao Tong University（上海交通大学）；University of Science and Technology of China（科学技术大学）；Wuhan University（武汉大学）；National University of Singapore（新加坡国立大学）
DirectEdit: Step-Level Accurate Inversion for Flow-Based Image Editing
National Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University, Wuhan, China（多媒体软件国家工程研究中心，计算机科学学院，武汉大学，中国武汉）
FED-Bench: A Cross-Granular Benchmark for Disentangled Evaluation of Facial Expression Editing
Xi’an Jiaotong University（西安交通大学）；Northwestern Polytechnical University（西北工业大学）；National University of Singapore（新加坡国立大学）；Fudan University（复旦大学）；Zhejiang Lab（之江实验室）
