FacEDiT：通过面部运动填充实现统一的说话人脸编辑与生成
FacEDiT: Unified Talking Face Editing and Generation via Facial Motion Infilling
AI总结：
FacEDiT通过语音条件面部运动填充框架，实现说话人脸编辑与生成的统一，具备准确的语音对齐和强身份保持性。
摘要：
说话人脸编辑和人脸生成往往被研究为不同的问题。在本工作中，我们提出将两者不视为独立任务，而是作为统一框架——语音条件面部运动填充的子任务。我们探索面部运动填充作为一种自监督的预训练任务，同时作为动态说话人脸合成的统一框架。为了实现这一想法，我们提出了FacEDiT，一种通过流匹配训练的语音条件扩散变换器。受掩码自动编码器的启发，FacEDiT学会在周围运动和语音条件下合成掩码的面部运动。这种框架使能够实现局部生成和编辑，如替换、插入和删除，同时确保与未编辑区域的无缝过渡。此外，偏置注意力和时间平滑约束增强了边界连续性和嘴唇同步。为了解决缺乏标准编辑基准的问题，我们引入了FacEDiTBench，第一个说话人脸编辑的数据集，包含多样化的编辑类型和长度，以及新的评估指标。大量实验验证了说话人脸编辑和生成作为语音条件运动填充子任务的出现；FacEDiT能够产生准确、语音对齐的面部编辑，具有强的身份保持性和平滑的视觉连续性，同时有效地泛化到说话人脸生成。
英文摘要：
Talking face editing and face generation have often been studied as distinct problems. In this work, we propose viewing both not as separate tasks but as subtasks of a unifying formulation, speech-conditional facial motion infilling. We explore facial motion infilling as a self-supervised pretext task that also serves as a unifying formulation of dynamic talking face synthesis. To instantiate this idea, we propose FacEDiT, a speech-conditional Diffusion Transformer trained with flow matching. Inspired by masked autoencoders, FacEDiT learns to synthesize masked facial motions conditioned on surrounding motions and speech. This formulation enables both localized generation and edits, such as substitution, insertion, and deletion, while ensuring seamless transitions with unedited regions. In addition, biased attention and temporal smoothness constraints enhance boundary continuity and lip synchronization. To address the lack of a standard editing benchmark, we introduce FacEDiTBench, the first dataset for talking face editing, featuring diverse edit types and lengths, along with new evaluation metrics. Extensive experiments validate that talking face editing and generation emerge as subtasks of speech-conditional motion infilling; FacEDiT produces accurate, speech-aligned facial edits with strong identity preservation and smooth visual continuity while generalizing effectively to talking face generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
