GaussianSwap: 基于3D高斯点云的可动画视频人脸交换
GaussianSwap: Animatable Video Face Swapping with 3D Gaussian Splatting
AI总结：
GaussianSwap通过3D高斯点云构建可动画视频人脸交换虚拟角色，实现身份保持与高保真视觉效果，支持交互式应用。
摘要：
我们介绍了GaussianSwap，一种新的视频人脸交换框架，该框架从目标视频中构建基于3D高斯点云的人脸虚拟角色，同时将身份从源图像转移到虚拟角色。传统视频交换框架受限于生成基于像素格式的面部表示。所得到的交换面孔仅是一组无结构的像素，没有任何动画或交互操作的能力。我们的工作从传统的基于像素的视频生成转向创建具有交换面孔的高保真虚拟角色。该框架首先预处理目标视频以提取FLAME参数、摄像机姿态和分割掩码，然后将3D高斯点云固定到FLAME模型上，跨帧实现动态面部控制。为确保身份保持，我们提出了一种由三种最先进的面部识别模型构建的复合身份嵌入，用于虚拟角色微调。最后，我们在背景帧上渲染人脸交换的虚拟角色，以获得人脸交换的视频。实验结果表明，GaussianSwap在身份保持、视觉清晰度和时间一致性方面均优于现有方法，同时实现了此前无法实现的交互应用。
英文摘要：
We introduce GaussianSwap, a novel video face swapping framework that constructs a 3D Gaussian Splatting based face avatar from a target video while transferring identity from a source image to the avatar. Conventional video swapping frameworks are limited to generating facial representations in pixel-based formats. The resulting swapped faces exist merely as a set of unstructured pixels without any capacity for animation or interactive manipulation. Our work introduces a paradigm shift from conventional pixel-based video generation to the creation of high-fidelity avatar with swapped faces. The framework first preprocesses target video to extract FLAME parameters, camera poses and segmentation masks, and then rigs 3D Gaussian splats to the FLAME model across frames, enabling dynamic facial control. To ensure identity preserving, we propose an compound identity embedding constructed from three state-of-the-art face recognition models for avatar finetuning. Finally, we render the face-swapped avatar on the background frames to obtain the face-swapped video. Experimental results demonstrate that GaussianSwap achieves superior identity preservation, visual clarity and temporal consistency, while enabling previously unattainable interactive applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
