DT-NeRF：用于高保真说话人像合成的分解三平面哈希神经辐射场
DT-NeRF: Decomposed Triplane-Hash Neural Radiance Fields for High-Fidelity Talking Portrait Synthesis
AI总结：
提出DT-NeRF框架，通过分解三平面表示嘴部与面部特征，并利用audio-mouth-face transformer融合音频残差与NeRF加性体积渲染，实现高保真说话人像合成。
摘要：
本文提出了分解三平面哈希神经辐射场（DT-NeRF），该框架显著提升了说话面部的真实感渲染，并在关键评估数据集上取得了最先进的结果。我们的架构将面部区域分解为两个专门的三平面：一个专门用于表示嘴部，另一个用于更广泛的面部特征。我们引入音频特征作为残差项，并通过audio-mouth-face transformer将其作为查询向量集成到模型中。此外，该方法利用神经辐射场（NeRF）的能力，通过加性体积渲染技术丰富整个面部的体积表示。综合实验评估证实了所提方法的有效性和优越性。
英文摘要：
In this paper, we present the decomposed triplane-hash neural radiance fields (DT-NeRF), a framework that significantly improves the photorealistic rendering of talking faces and achieves state-of-the-art results on key evaluation datasets. Our architecture decomposes the facial region into two specialized triplanes: one specialized for representing the mouth, and the other for the broader facial features. We introduce audio features as residual terms and integrate them as query vectors into our model through an audio-mouth-face transformer. Additionally, our method leverages the capabilities of Neural Radiance Fields (NeRF) to enrich the volumetric representation of the entire face through additive volumetric rendering techniques. Comprehensive experimental evaluations corroborate the effectiveness and superiority of our proposed approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
