MEOM：用于人体姿态三角测量的多视图期望OKS最大化

中文摘要 AI 辅助

传统代数三角测量法从多视图2D关键点求解3D人体姿态估计（HPE），典型方法是从预测热图解码2D关键点，但该方法不可靠，因为遮挡下热图可能呈多模态，将其压缩为单个峰值会丢失空间分布信息。我们试图利用完整热图更准确地估计3D姿态，这需要解决两个问题：如何跨视图鲁棒融合热图，以及如何评估热图的可靠性。针对前者，我们提出新目标：多视图期望OKS最大化（MEOM），其定位3D关节的依据是各视图在概率质量上达成一致；针对后者，我们采用最高密度区域（HDR）校准作为该概率质量的诊断依据，且独立于基于距离的度量。所提框架涵盖有3D监督和无3D监督两种设置：无3D监督时，我们从预训练热图预测器优化3D姿态，通过最大化MEOM实现，性能与依赖更大骨干网络、时间融合及模拟3D数据的现有最优方法相当，在模糊的Human3.6M（H36MA）和被遮挡的CMU Panoptic帧上优势显著；当有3D标签时，我们以MEOM与MSE损失相结合的方式端到端训练模型，在Human3.6M上实现19.11mm的绝对MPJPE，以仅为现有最优体积方法一半的推理成本，在绝对MPJPE指标上超越该方法。

英文摘要

Conventional algebraic triangulation solves 3D human pose estimation (HPE) from multi-view 2D keypoints. The typical approach, decoding 2D keypoints from predicted heatmaps, is unreliable as heatmaps can be multimodal under occlusion, and collapsing them into single peaks discards their spatial distribution. We seek to use the entire heatmap to estimate 3D poses more accurately, which requires solving two problems: how to robustly fuse heatmaps across views, and how to assess the reliability of heatmaps. For the former, we introduce a novel objective, Multi-viewExpected-OKS Maximization (MEOM), that locates a 3D joint where the views agree in probability mass. For the latter, we adopt highest-density-region (HDR) calibration as a diagnostic of that mass, independently of distance-based metrics. The proposed framework covers two settings, with and without 3D supervision. Without 3D supervision, we optimize 3D poses from pretrained heatmap predictors by maximizing MEOM, achieving comparable performance with state-of-the-art methods that rely on larger backbones, temporal fusion, and simulated 3D data. On ambiguous Human3.6M (H36MA) and occluded CMU Panoptic frames, the advantage is substantial. When 3D labels are available, we train the model end-to-end with a combined MEOM and MSE loss, achieving 19.11 mm absolute MPJPE on Human3.6M outperforming the state-of-the-art volumetric approach on absolute MPJPE at half the inference cost.

发表机构

• Linköping University（林雪平大学）

机构由 AI 辅助整理，请以论文原文为准。