DiffMagicFace: 实现实时视频中人脸的恒定身份编辑
DiffMagicFace: Identity Consistent Facial Editing of Real Videos
AI总结：
本文提出DiffMagicFace框架，通过整合文本和图像控制模型，实现视频中人脸编辑时保持身份一致性和内容一致性，无需依赖视频数据集，效果在视觉和量化指标上均优于现有方法。
摘要：
文本条件图像编辑因图像扩散模型的进步而显著受益。然而，将其扩展到面部视频编辑时，保持源视频中面部身份一致性和确保编辑主体在帧间一致性的挑战增加。本文介绍DiffMagicFace，一种独特的视频编辑框架，整合了两个微调模型用于文本和图像控制。这些模型在推理过程中同时运行，以生成保持身份特征并无缝对齐编辑语义的视频帧。为确保编辑视频的一致性，我们开发了一个包含展示各种面部视角的图像数据集。数据集的创建通过渲染技术和后续优化算法实现。值得注意的是，我们的方法不依赖视频数据集，但仍能提供高质量的一致性和内容。优异的效果甚至适用于复杂的任务如虚拟人物视频和区分密切相关类别。使用我们框架编辑的视频效果与传统渲染软件制作的视频相当。通过与当前最先进方法的比较分析，我们的框架在视觉吸引力和量化指标上均表现出色。
英文摘要：
Text-conditioned image editing has greatly benefitted from the advancements in Image Diffusion Models. However, extending these techniques to facial video editing introduces challenges in preserving facial identity throughout the source video and ensuring consistency of the edited subject across frames. In this paper, we introduce DiffMagicFace, a unique video editing framework that integrates two fine-tuned models for text and image control. These models operate concurrently during inference to produce video frames that maintain identity features while seamlessly aligning with the editing semantics. To ensure the consistency of the edited videos, we develop a dataset comprising images showcasing various facial perspectives for each edited subject. The creation of a data set is achieved through rendering techniques and the subsequent application of optimization algorithms. Remarkably, our approach does not depend on video datasets but still delivers high-quality results in both consistency and content. The excellent effect holds even for complex tasks like talking head videos and distinguishing closely related categories. The videos edited using our framework exhibit parity with videos that are made using traditional rendering software. Through comparative analysis with current state-of-the-art methods, our framework demonstrates superior performance in both visual appeal and quantitative metrics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
