HOGSA：利用基于3D Gaussian Splatting的数据增强理解双手手物交互
HOGSA: Bimanual Hand-Object Interaction Understanding with 3D Gaussian Splatting Based Data Augmentation
AI总结：
本文提出HOGSA框架，利用基于网格的3DGS、超分辨率模块和双手姿态优化生成照片级真实的多姿态、多视角数据，并在H2O与Arctic上验证其可提升双手手物交互理解基线。
摘要：
对双手手物交互的理解在机器人和虚拟现实中发挥着重要作用。然而，由于手与物体之间存在严重遮挡，且运动具有高自由度，收集并标注高质量、大规模数据集颇具挑战，这阻碍了双手手物交互相关基线的进一步提升。在本研究中，我们提出了一种新的基于3D Gaussian Splatting的双手手物交互数据增强框架，该框架能够将现有数据集增强为具有多种手物姿态和视角的大规模照片级真实数据。首先，我们使用基于网格的3DGS对物体和手部进行建模；为了解决因使用多分辨率输入图像而导致的渲染模糊问题，我们设计了一个超分辨率模块。其次，我们将单手抓握姿态优化模块扩展到双手手物场景，以生成多样的双手手物交互姿态，从而显著扩展数据集的姿态分布。第三，我们分析了所提出的数据增强在不同方面对双手手物交互理解的影响。我们在H2O和Arctic两个基准上进行了数据增强，并验证了我们的方法能够提升基线的性能。
英文摘要：
Understanding of bimanual hand-object interaction plays an important role in robotics and virtual reality. However, due to significant occlusions between hands and object as well as the high degree-of-freedom motions, it is challenging to collect and annotate a high-quality, large-scale dataset, which prevents further improvement of bimanual hand-object interaction-related baselines. In this work, we propose a new 3D Gaussian Splatting based data augmentation framework for bimanual hand-object interaction, which is capable of augmenting existing dataset to large-scale photorealistic data with various hand-object pose and viewpoints. First, we use mesh-based 3DGS to model objects and hands, and to deal with the rendering blur problem due to multi-resolution input images used, we design a super-resolution module. Second, we extend the single hand grasping pose optimization module for the bimanual hand object to generate various poses of bimanual hand-object interaction, which can significantly expand the pose distribution of the dataset. Third, we conduct an analysis for the impact of different aspects of the proposed data augmentation on the understanding of the bimanual hand-object interaction. We perform our data augmentation on two benchmarks, H2O and Arctic, and verify that our method can improve the performance of the baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
