保留源视频真实性:面向电影质量的高保真面部交换
Preserving Source Video Realism: High-Fidelity Face Swapping for Cinematic Quality
AI总结:
本文提出LivingSwap模型,通过关键帧和视频参考引导实现高保真面部交换,提升视频真实感与时间一致性,减少生产流程中的手动工作量。
摘要:
视频面部交换在电影和娱乐生产中至关重要,但实现高保真和时间一致性仍具挑战性。受参考引导图像编辑的启发,本文探索源视频丰富视觉属性在视频面部交换中的应用。我们提出LivingSwap,首个视频参考引导面部交换模型,通过关键帧作为条件信号注入目标身份,结合关键帧条件与视频参考引导进行时间拼接,确保稳定身份保留和高保真重建。为解决参考引导训练数据稀缺问题,我们构建了配对面部交换数据集Face2Face,并反向数据对以确保可靠的真实监督。大量实验表明,本方法在状态-of-the-art性能上实现了无缝整合目标身份与源视频的表达、光照和运动,同时显著减少生产流程中的手动工作量。项目网页:https://aim-uofa.github.io/LivingSwap
英文摘要:
Video face swapping is crucial in film and entertainment production, where achieving high fidelity and temporal consistency over long and complex video sequences remains a significant challenge. Inspired by recent advances in reference-guided image editing, we explore whether rich visual attributes from source videos can be similarly leveraged to enhance both fidelity and temporal coherence in video face swapping. Building on this insight, this work presents LivingSwap, the first video reference guided face swapping model. Our approach employs keyframes as conditioning signals to inject the target identity, enabling flexible and controllable editing. By combining keyframe conditioning with video reference guidance, the model performs temporal stitching to ensure stable identity preservation and high-fidelity reconstruction across long video sequences. To address the scarcity of data for reference-guided training, we construct a paired face-swapping dataset, Face2Face, and further reverse the data pairs to ensure reliable ground-truth supervision. Extensive experiments demonstrate that our method achieves state-of-the-art results, seamlessly integrating the target identity with the source video's expressions, lighting, and motion, while significantly reducing manual effort in production workflows. Project webpage: https://aim-uofa.github.io/LivingSwap
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。