DivaTrack：基于加速度增强的三点追踪器的多样化身体与动作
DivaTrack: Diverse Bodies and Motions from Acceleration-Enhanced Three-Point Trackers
AI总结：
提出深度学习框架DivaTrack，利用IMU线性加速度增强三点追踪输入以预测足部接触，结合上半身姿态在两阶段模型中约束下半身，并融合双参考系预测稳定全身姿态，实现了多样化体型与动作的实时准确全身追踪。
摘要：
全身虚拟形象的存在对于数字现实中沉浸式的社交和环境交互至关重要。然而，当前的设备仅从头显和两个控制器（即三点追踪器）提供三个六自由度（DOF）姿态。由于这是一个高度欠约束的问题，从这些输入推断全身姿态具有挑战性，特别是在支持普通人群所代表的全部身体比例和用例时。在本文中，我们提出了一个深度学习框架DivaTrack，当应用于多样化的身体尺寸和活动时，它优于现有方法。我们利用来自惯性测量单元（IMU）的线性加速度来增强稀疏的三点输入，以改善足部接触预测。然后，我们在两阶段模型中利用足部接触和上半身姿态的预测来约束原本具有歧义的下半身姿态。我们通过学习融合在两个参考系中计算的预测，进一步在广泛的配置范围内稳定了推断出的全身姿态，每个参考系专为不同类型的动作而设计。我们在一个大型数据集上展示了我们设计的有效性，该数据集捕获了22名受试者执行对三点追踪具有挑战性的移动动作，包括弓步、呼啦圈和坐下。如在使用Meta VR头显和Xsens IMU的现场演示所示，我们的方法实时运行，同时准确追踪用户执行各种不同动作时的运动。
英文摘要：
Full-body avatar presence is crucial for immersive social and environmental interactions in digital reality. However, current devices only provide three six degrees of freedom (DOF) poses from the headset and two controllers (i.e. three-point trackers). Because it is a highly under-constrained problem, inferring full-body pose from these inputs is challenging, especially when supporting the full range of body proportions and use cases represented by the general population. In this paper, we propose a deep learning framework, DivaTrack, which outperforms existing methods when applied to diverse body sizes and activities. We augment the sparse three-point inputs with linear accelerations from Inertial Measurement Units (IMU) to improve foot contact prediction. We then condition the otherwise ambiguous lower-body pose with the predictions of foot contact and upper-body pose in a two-stage model. We further stabilize the inferred full-body pose in a wide range of configurations by learning to blend predictions that are computed in two reference frames, each of which is designed for different types of motions. We demonstrate the effectiveness of our design on a large dataset that captures 22 subjects performing challenging locomotion for three-point tracking, including lunges, hula-hooping, and sitting. As shown in a live demo using the Meta VR headset and Xsens IMUs, our method runs in real-time while accurately tracking a user's motion when they perform a diverse set of movements.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
