TalkingGaussian：基于高斯泼溅的结构持久性3D说话头合成
TalkingGaussian: Structure-Persistent 3D Talking Head Synthesis via Gaussian Splatting
AI总结：
本文提出TalkingGaussian，通过基于高斯泼溅的变形范式替代直接修改点外观，并分解面部与口腔分支解决运动不一致问题，实现高保真、高效率的3D说话头合成。
摘要：
辐射场在合成逼真的3D说话头方面已展现出令人印象深刻的表现。然而，由于难以拟合剧烈的外观变化，现有主流范式通过直接修改点外观来呈现面部运动，可能导致动态区域出现失真。为应对这一挑战，我们提出了TalkingGaussian，一个基于变形的辐射场框架，用于高保真说话头合成。借助基于点的高斯泼溅（Gaussian Splatting），在我们的方法中，面部运动可以通过对持久的高斯基元施加平滑且连续的变形来表示，而无需像以往方法那样学习困难的外观变化。由于这一简化，可以在保持高度完整的面部特征的同时合成精确的面部运动。在这种变形范式下，我们进一步发现了一种面部-嘴部运动不一致性问题，该问题会影响细节说话动作的学习。为解决这一冲突，我们将模型分解为两个分支，分别处理面部和口腔内部区域，从而简化学习任务，帮助重建更准确的嘴部区域运动和结构。大量实验表明，我们的方法能够渲染高质量的唇音同步说话头视频，与以往方法相比具有更好的面部保真度和更高的效率。
英文摘要：
Radiance fields have demonstrated impressive performance in synthesizing lifelike 3D talking heads. However, due to the difficulty in fitting steep appearance changes, the prevailing paradigm that presents facial motions by directly modifying point appearance may lead to distortions in dynamic regions. To tackle this challenge, we introduce TalkingGaussian, a deformation-based radiance fields framework for high-fidelity talking head synthesis. Leveraging the point-based Gaussian Splatting, facial motions can be represented in our method by applying smooth and continuous deformations to persistent Gaussian primitives, without requiring to learn the difficult appearance change like previous methods. Due to this simplification, precise facial motions can be synthesized while keeping a highly intact facial feature. Under such a deformation paradigm, we further identify a face-mouth motion inconsistency that would affect the learning of detailed speaking motions. To address this conflict, we decompose the model into two branches separately for the face and inside mouth areas, therefore simplifying the learning tasks to help reconstruct more accurate motion and structure of the mouth region. Extensive experiments demonstrate that our method renders high-quality lip-synchronized talking head videos, with better facial fidelity and higher efficiency compared with previous methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
