具有动态显式头发的头部头像
Head Avatars with Dynamic Explicit Hair
- ETH Zürich（苏黎世联邦理工学院）
- Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
- Max Planck Institute for Informatics（马克斯·普朗克信息研究所）
- Tübingen AI Center（图宾根人工智能中心）
- Technical University of Munich（慕尼黑工业大学）
- Technical University of Darmstadt（达姆施塔特工业大学）
- Microsoft（微软）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
该研究提出DynHair方法，用于跟踪和建模人类头部头像的动态头发。通过结构化3D高斯喷绘重建头像，利用基于头部角速度等的时间网络建模发丝动态变形，经联合优化获得可控头发动力学的动画头像，实验展现了最优性能。
AI中文摘要：
我们提出了DynHair，一种用于跟踪和建模人类头部头像动态头发的新方法。从视频输入中，我们使用结构化3D高斯喷绘重建具有基于显式发丝的头发表示的动态头部头像。与可以用相对于某些可表达的3D头部模型附加或生成的3D高斯建模的人类头部头像的面部区域不同，头发由于其动态运动效果而特别具有挑战性。因此，我们提出了一种新方法，使用基于头部角速度和加速度以及相对重力的时间网络对发丝的动态变形进行建模。具体来说，一个LSTM对运动历史进行编码，并通过FiLM条件调制逐点发丝特征，MLP进一步使用该特征来产生到标准发型的物理上合理的位移。我们通过基于光度、几何和物理的监督的可微高斯喷绘，联合优化头发的这种运动和外观表示以及面部区域的基于3DGS的表示。作为我们方法的结果，我们获得了训练视频数据的头发跟踪以及具有可控头发动力学的可动画化头部头像。在我们的实验中，我们在头发动力学、时间一致性和跨主体泛化方面展示了当前的最优性能。
英文摘要：
We present DynHair, a novel method for tracking and modeling dynamic hair for human head avatars. From video input, we reconstruct a dynamic head avatar with an explicit strand-based hair representation using structured 3D Gaussian Splatting. In contrast to the face region of human head avatars, which can be modeled with 3D Gaussians that are attached or generated with respect to some expressive 3D head model, hair is particularly challenging as it exhibits dynamic motion effects. Therefore, we present a novel method that models the dynamic deformations of the hair strands using a temporal network that is conditioned on angular velocity and acceleration of the head, as well as relative gravity. Specifically, an LSTM encodes the motion history and modulates per-point strand features via FiLM conditioning which further used by MLP to produce physically plausible displacements to canonical hairstyle. We jointly optimize this motion and appearance representation of the hair, with a 3DGS-based representation of the face-region, via differentiable Gaussian splatting with photometric, geometric, and physics-based supervision. As a result of our method, we retrieve hair tracking of the training video data and an animatable head avatar with controllable hair dynamics. In our experiments, we demonstrate state-of-the-art performance in terms of hair dynamics, temporal consistency, and generalization across subjects.
