SignSplat：通过高斯泼溅渲染手语
SignSplat: Rendering Sign Language via Gaussian Splatting
AI总结：
SignSplat提出一种基于高斯泼溅的手语渲染框架，通过约束网格参数、正则化高斯参数和自适应加密剪除方法，从少视角序列数据中高保真建模细微手部和面部动作，并在基准数据集上取得最先进性能。
摘要：
当前基于高斯泼溅的条件人体渲染最先进方法通常聚焦于从多视角捕捉的简单身体动作，常见场景为跳舞或行走。然而，对于手语等更复杂的使用场景，我们更关注手部和面部的细微复杂动作，而非大幅身体运动。构建高保真模型的难题因捕捉手语多视角数据的复杂性而加剧。解决方案是更好地利用序列数据，通过利用时间可变性来克服仅从少数视角获取的有限信息。尽管如此，从序列级数据学习需要极其准确且一致的模型拟合，以确保外观在复杂动作中保持一致。我们专注于如何实现这一点，通过约束网格参数，从少数视角构建一个能够建模细微人体运动的精确高斯泼溅框架。我们利用高斯参数上的正则化技术来缓解过拟合和渲染伪影。此外，我们提出了一种新的自适应控制方法，用于在网格表面上加密高斯点并剪除泼溅点。为证明方法的准确性，我们渲染手语视频的新序列，并基于神经机器翻译方法进行手语拼接。在基准数据集上，我们的方法取得了最先进的性能；在高度关节化且复杂的手语动作上，我们显著优于竞争方法。
英文摘要：
State-of-the-art approaches for conditional human body rendering via Gaussian splatting typically focus on simple body motions captured from many views. This is often in the context of dancing or walking. However, for more complex use cases, such as sign language, we care less about large body motion and more about subtle and complex motions of the hands and face. The problems of building high fidelity models are compounded by the complexity of capturing multi-view data of sign. The solution is to make better use of sequence data, ensuring that we can overcome the limited information from only a few views by exploiting temporal variability. Nevertheless, learning from sequence-level data requires extremely accurate and consistent model fitting to ensure that appearance is consistent across complex motions. We focus on how to achieve this, constraining mesh parameters to build an accurate Gaussian splatting framework from few views capable of modelling subtle human motion. We leverage regularization techniques on the Gaussian parameters to mitigate overfitting and rendering artifacts. Additionally, we propose a new adaptive control method to densify Gaussians and prune splat points on the mesh surface. To demonstrate the accuracy of our approach, we render novel sequences of sign language video, building on neural machine translation approaches to sign stitching. On benchmark datasets, our approach achieves state-of-the-art performance; and on highly articulated and complex sign language motion, we significantly outperform competing approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
