DNF-Avatar：面向实时可动画化虚拟形象重光照的神经场蒸馏
DNF-Avatar: Distilling Neural Fields for Real-time Animatable Avatar Relighting
AI总结：
本文提出DNF-Avatar，将隐式神经场知识蒸馏到二维高斯泼溅表示，并结合分割和近似与分部位环境光遮蔽探针，实现67 FPS的实时可动画化虚拟形象重光照，推理速度比教师模型快370倍。
摘要：
从单目视频中创建可重光照且可动画化的人类虚拟形象是一个日益兴起的研究课题，在虚拟现实、体育和视频游戏等领域具有广泛应用。以往工作利用神经场结合基于物理的渲染（PBR）来估计几何结构并解耦人类虚拟形象的外观属性。然而，这些方法的一个缺点是，由于昂贵的蒙特卡洛光线追踪，渲染速度较慢。为了解决这一问题，我们提出将隐式神经场（教师模型）中的知识蒸馏到显式二维高斯泼溅（学生模型）表示中，以利用高斯泼溅的快速光栅化特性。为避免光线追踪，我们采用分割和近似（split-sum approximation）来处理PBR外观。我们还提出了新颖的分部位环境光遮蔽探针用于阴影计算。阴影预测通过每个像素仅查询一次这些探针来实现，这为虚拟形象的实时重光照铺平了道路。这些技术的结合能够产生具有逼真阴影效果的高质量重光照结果。我们的实验表明，所提出的学生模型在推理时比教师模型快370倍，同时达到了67 FPS的渲染速度，并且取得了与教师模型相当甚至更好的重光照结果。
英文摘要：
Creating relightable and animatable human avatars from monocular videos is a rising research topic with a range of applications, e.g. virtual reality, sports, and video games. Previous works utilize neural fields together with physically based rendering (PBR), to estimate geometry and disentangle appearance properties of human avatars. However, one drawback of these methods is the slow rendering speed due to the expensive Monte Carlo ray tracing. To tackle this problem, we proposed to distill the knowledge from implicit neural fields (teacher) to explicit 2D Gaussian splatting (student) representation to take advantage of the fast rasterization property of Gaussian splatting. To avoid ray-tracing, we employ the split-sum approximation for PBR appearance. We also propose novel part-wise ambient occlusion probes for shadow computation. Shadow prediction is achieved by querying these probes only once per pixel, which paves the way for real-time relighting of avatars. These techniques combined give high-quality relighting results with realistic shadow effects. Our experiments demonstrate that the proposed student model achieves comparable or even better relighting results with our teacher model while being 370 times faster at inference time, achieving a 67 FPS rendering speed.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
