Prompt-to-Prompt: Text-Based Image Editing Via Cross-Attention Mechanisms -- The Research of Hyperparameters and Novel Mechanisms to Enhance Existing Frameworks
Recent advances in image editing have shifted from manual pixel manipulation to employing deep learning methods like stable diffusion models, which now leverage cross-attention mechanisms for text-driven control. This transition has simplified the editing process but also introduced variability in results, such as inconsistent hair color changes. Our research aims to enhance the precision and reliability of prompt-to-prompt image editing frameworks by exploring and optimizing hyperparameters. We present a comprehensive study of the "word swap" method, develop an "attention re-weight method" for better adaptability, and propose the "CL P2P" framework to address existing limitations like cycle inconsistency. This work contributes to understanding and improving the interaction between hyperparameter settings and the architectural choices of neural network models, specifically their attention mechanisms, which significantly influence the composition and quality of the generated images.
CannyEdit: Selective Canny Control and Dual-Prompt Guidance for Training-Free Image Editing
Huawei Hong Kong AI Framework & Data Technologies Lab（华为香港AI框架与数据技术实验室）；The Hong Kong University of Science and Technology（香港科技大学）；Shanghai University of Finance and Economics（上海财经大学）
Refinement Is Inherently Editable: Training-Free Prompt-to-Prompt Image Editing with Generative Refinement Network
City University of Hong Kong (Dongguan)（香港城市大学（东莞））；City University of Hong Kong（香港城市大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Visual Prompting for One-shot Controllable Video Editing without Inversion
Singapore University of Technology and Design（新加坡科技设计大学）；Wuhan University（武汉大学）；Institute for Infocomm Research, Agency for Science, Technology and Research, Singapore（新加坡资讯与通信研究院）
Google Research（谷歌研究院）；Tel Aviv University（特拉维夫大学）
Recent large-scale text-driven synthesis models have attracted much attention thanks to their remarkable capabilities of generating highly diverse images that follow given text prompts. Such text-based synthesis methods are particularly appealing to humans who are used to verbally describe their intent. Therefore, it is only natural to extend the text-driven image synthesis to text-driven image editing. Editing is challenging for these generative models, since an innate property of an editing technique is to preserve most of the original image, while in the text-based models, even a small modi…
The Chinese University of Hong Kong（香港中文大学）；Adobe（奥多比）；SmartMore（思谋）
This paper presents Video-P2P, a novel framework for real-world video editing with cross-attention control. While attention control has proven effective for image editing with pre-trained image generation models, there are currently no large-scale video generation models publicly available. Video-P2P addresses this limitation by adapting an image generation diffusion model to complete various video editing tasks. Specifically, we propose to first tune a Text-to-Set (T2S) model to complete an approximate inversion and then optimize a shared unconditional embedding to achieve accurate video inve…
SceneCraft: Interactive System for Image Editing via Scene Graph
University of Science, Ho Chi Minh, Vietnam（胡志明市理科大学）；Vietnam National University, Ho Chi Minh, Vietnam（越南国家大学胡志明市）；University of Dayton, Dayton, Ohio, USA（代顿大学）
Cut-and-Paste: Subject-Driven Video Editing with Attention Control
School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机与信息工程学院）；School of Computer Science, Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）计算机学院）
EditTransfer++: Toward Faithful and Efficient Visual-Prompt-Guided Image Editing
School of Information and Communication Engineering and the State Key Laboratory of Media Convergence and Communication, Communication University of China（信息与通信工程学院和媒体融合与通信国家重点实验室，中国传媒大学）；ShowLab, National University of Singapore（新加坡国立大学ShowLab）
DPDEdit: Detail-Preserved Diffusion Models for Multimodal Fashion Image Editing
Shenzhen Technology University（深圳科技大学）；University of Washington（华盛顿大学）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究院，中国科学院）
Prompt-Softbox-Prompt: A Free-Text Embedding Control for Image Editing
Shanghai University of Finance and Economics（上海财经大学）；School of Computing and Artificial Intelligence（计算机与人工智能学院）；MoE Key Laboratory of Interdisciplinary Research of Computation and Economics（交叉计算与经济学联合实验室）
FiVE: A Fine-grained Video Editing Benchmark for Evaluating Emerging Diffusion and Rectified Flow Models
Harvard AI and Robotics Lab, Harvard University（哈佛人工智能与机器人实验室，哈佛大学）；Broad Institute（博德研究所）；Hong Kong Polytechnic University（香港理工大学）；School of Engineering and Applied Sciences, Harvard University（哈佛大学工程与应用科学学院）；City University of Hong Kong（香港城市大学）；Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University（自然与人工智能研究学院，哈佛大学）
