生物力学三维人体：从三维人体基础模型自监督蒸馏生物力学姿态
Biomechanical 3D Body: Self-Supervised Distillation of Biomechanical Pose from a 3D Body Foundation Model
浏览论文内容
中文总结 AI 辅助
该研究扩展SAM-3D-Body模型新增生物力学预测头，用JAX结合Equinox实现，在公开数据集训练后，在多数据集验证中优于现有生物力学回归模型，仅略逊于需高成本轨迹优化的同类方法。
中文摘要 AI 辅助
当前最先进的单目人体恢复方法会预测对应运动学树上的网格顶点和角度，但其输出缺乏临床分析、生物力学分析等下游应用所需的生物力学定义的关节角度。我们扩展了现有基础模型SAM-3D-Body，新增一个生物力学预测头，可从单张RGB图像中回归生物力学模型的关节角度与尺度。训练该模型存在挑战，因为配对图像与生物力学拟合的数据集有限。为解决此问题，我们使用Levenberg-Marquardt求解器的循环内优化目标监督生物力学输出，该求解器针对网格预测的标记执行逆运动学拟合，这使得即使从未标记图像中也能从网格头蒸馏出生物力学头。为适配MuJoCo中GPU优化的生物力学模型，整个模型用JAX结合Equinox实现。我们在公开的SAM-3D-Body数据集上训练了该蒸馏输出头，随后在两个公开的基于标记的数据集MoVi、BioCV，以及用多视角无标记运动捕捉捕获的临床队列动作上验证了该模型。所得模型在从图像直接回归生物力学方面优于现有模型，仅略逊于最先进的单目生物力学方法，该方法需执行计算成本更高的推理时全轨迹优化。
英文摘要
State-of-the-art monocular body recovery methods predict mesh vertices and angles on the corresponding kinematic tree, but their outputs lack biomechanically defined joint angles that downstream applications like clinical and biomechanical analyses require. We extend an existing foundation model, SAM-3D-Body, with an additional biomechanical prediction head that, from a single RGB image, regresses the joint angles and scales of a biomechanical model. Training this model presents a challenge, as there are limited datasets of paired images and biomechanical fits. To overcome this, we supervise biomechanical outputs with in-loop optimized targets from a Levenberg-Marquardt solver performing inverse kinematics fits against markers from the mesh predictions. This allows distilling the biomechanical head from the mesh head, even from unlabeled images. To make this work with GPU-optimized biomechanical models in MuJoCo, the entire model was implemented in JAX using Equinox. We trained this distilled output head on the publicly released SAM-3D-Body dataset. We then validated this model on biomechanical fits to two publicly available marker-based datasets, MoVi and BioCV, as well as movements from a clinical cohort captured with multiview markerless motion capture. The resulting model outperforms existing models for direct regression of biomechanics from images while only slightly underperforming the state-of-the-art monocular biomechanics method that performs more costly inference-time optimization of entire trajectories.
发表机构
- Shirley Ryan AbilityLab（雪莉·瑞安能力实验室）
- Northwestern University（西北大学）
- University of Illinois Chicago（伊利诺伊大学芝加哥分校）
- University of Texas at Austin（德克萨斯大学奥斯汀分校）
机构由 AI 辅助整理，请以论文原文为准。
