SplattingAvatar：采用网格嵌入高斯泼溅的逼真实时数字人化身
SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting
AI总结：
本研究提出SplattingAvatar混合3D表示，将高斯泼溅嵌入三角形网格，解耦虚拟人运动与外观，通过网格直接控制高斯元变换，兼容多种动画技术，实现高帧率实时渲染与SOTA渲染质量。
摘要：
我们提出SplattingAvatar，这是一种嵌入在三角形网格上的高斯泼溅（Gaussian Splatting）的逼真数字人化身混合3D表示，在现代GPU上渲染帧率超过300 FPS，在移动设备上可达30 FPS。我们通过显式网格几何和基于高斯泼溅的隐式外观建模，解耦虚拟人的运动与外观。高斯元由三角形网格上的重心坐标和位移定义，视作Phong曲面。我们拓展了提升优化方法，在高斯元于三角形网格上移动的同时同步优化其参数。SplattingAvatar是虚拟人的混合表示：网格承载低频运动与表面形变，高斯元负责高频几何与精细外观。现有形变方法依赖基于MLP的线性混合蒙皮（LBS）场实现运动，我们则直接通过网格控制高斯元的旋转与平移，这使其兼容多种动画技术，例如骨骼动画、混合形状和网格编辑。SplattingAvatar可从单目视频训练全身与头部化身，在多个数据集上展现出当前最优的渲染质量。
英文摘要：
We present SplattingAvatar, a hybrid 3D representation of photorealistic human avatars with Gaussian Splatting embedded on a triangle mesh, which renders over 300 FPS on a modern GPU and 30 FPS on a mobile device. We disentangle the motion and appearance of a virtual human with explicit mesh geometry and implicit appearance modeling with Gaussian Splatting. The Gaussians are defined by barycentric coordinates and displacement on a triangle mesh as Phong surfaces. We extend lifted optimization to simultaneously optimize the parameters of the Gaussians while walking on the triangle mesh. SplattingAvatar is a hybrid representation of virtual humans where the mesh represents low-frequency motion and surface deformation, while the Gaussians take over the high-frequency geometry and detailed appearance. Unlike existing deformation methods that rely on an MLP-based linear blend skinning (LBS) field for motion, we control the rotation and translation of the Gaussians directly by mesh, which empowers its compatibility with various animation techniques, e.g., skeletal animation, blend shapes, and mesh editing. Trainable from monocular videos for both full-body and head avatars, SplattingAvatar shows state-of-the-art rendering quality across multiple datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
