IP-FaceDiff：基于扩散的身份保持人脸视频编辑
IP-FaceDiff: Identity-Preserving Facial Video Editing with Diffusion
AI总结：
针对现有人脸视频编辑模型质量差、成本高、身份保持难及灵活性不足的问题，提出IP-FaceDiff框架，微调预训练T2I扩散模型，实现高质量文本驱动编辑，减时80%且保持身份与时间一致性，性能优于现有技术。
摘要：
人脸视频编辑对内容创作者而言愈发重要，可实现对面部表情与属性的操控。然而，现有模型存在编辑质量不佳、计算成本高，且在各类编辑中难以保持人脸身份的问题。此外，这些模型通常局限于编辑预定义的人脸属性，限制了其应对多样化编辑提示的灵活性。为解决这些挑战，我们提出一种新型人脸视频编辑框架，该框架利用预训练文本到图像（T2I）扩散模型的丰富潜在空间，并针对人脸视频编辑任务对其进行微调。我们的方法引入了一种定向微调方案，可实现高质量、局部化的文本驱动编辑，同时确保视频帧间的身份保持。此外，通过在推理阶段使用预训练T2I模型，我们的方法将编辑时间显著减少80%，同时在整个视频序列中保持时间一致性。我们通过在多种挑战性场景（包括不同的头部姿态、复杂动作序列和多样面部表情）下进行大量测试，评估了该方法的有效性。我们的方法始终优于现有技术，在广泛的指标和基准上展现出更优性能。
英文摘要：
Facial video editing has become increasingly important for content creators, enabling the manipulation of facial expressions and attributes. However, existing models encounter challenges such as poor editing quality, high computational costs and difficulties in preserving facial identity across diverse edits. Additionally, these models are often constrained to editing predefined facial attributes, limiting their flexibility to diverse editing prompts. To address these challenges, we propose a novel facial video editing framework that leverages the rich latent space of pre-trained text-to-image (T2I) diffusion models and fine-tune them specifically for facial video editing tasks. Our approach introduces a targeted fine-tuning scheme that enables high quality, localized, text-driven edits while ensuring identity preservation across video frames. Additionally, by using pre-trained T2I models during inference, our approach significantly reduces editing time by 80%, while maintaining temporal consistency throughout the video sequence. We evaluate the effectiveness of our approach through extensive testing across a wide range of challenging scenarios, including varying head poses, complex action sequences, and diverse facial expressions. Our method consistently outperforms existing techniques, demonstrating superior performance across a broad set of metrics and benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
