增强时空上下文用于从单目视频中恢复时间一致的鲁棒3D人体运动
Enhanced Spatio-Temporal Context for Temporally Consistent Robust 3D Human Motion Recovery from Monocular Videos
AI总结：
提出一种从单目视频恢复时间一致3D人体运动的新方法，利用身体感知特征、逐帧初始化与自注意力聚合增强时空上下文，结合LSTM细化参数，在多项指标上超越现有方法。
摘要：
从单目视频中恢复时间一致的3D人体姿态、形状和运动是一项具有挑战性的任务，原因包括（自）遮挡、光照条件差、复杂的铰接体姿态、深度模糊以及标注数据可用性有限。此外，进行简单的逐帧估计是不够的，因为它会导致抖动和不合理的结果。在本文中，我们提出了一种从单目视频中估计时间一致运动的新方法。我们的方法不使用通用的类ResNet特征，而是使用身体感知特征表示以及在时间窗口上进行独立的逐帧姿态和相机初始化，随后通过对身体感知特征和逐帧初始化使用自相似性和自注意力的组合进行新颖的时空特征聚合。这些操作共同通过考虑剩余的过去和未来帧，为每一帧生成了增强的时空上下文。这些特征被用于预测人体模型的姿态和形状参数，并进一步使用LSTM进行细化。在公开可用的基准数据上的实验结果表明，我们的方法获得了显著更低的加速度误差，并在所有关键定量评估指标上优于现有的最先进方法，包括在部分遮挡、复杂姿态甚至相对低照度等复杂场景中。
英文摘要：
Recovering temporally consistent 3D human body pose, shape and motion from a monocular video is a challenging task due to (self-)occlusions, poor lighting conditions, complex articulated body poses, depth ambiguity, and limited availability of annotated data. Further, doing a simple perframe estimation is insufficient as it leads to jittery and implausible results. In this paper, we propose a novel method for temporally consistent motion estimation from a monocular video. Instead of using generic ResNet-like features, our method uses a body-aware feature representation and an independent per-frame pose and camera initialization over a temporal window followed by a novel spatio-temporal feature aggregation by using a combination of self-similarity and self-attention over the body-aware features and the perframe initialization. Together, they yield enhanced spatiotemporal context for every frame by considering remaining past and future frames. These features are used to predict the pose and shape parameters of the human body model, which are further refined using an LSTM. Experimental results on the publicly available benchmark data show that our method attains significantly lower acceleration error and outperforms the existing state-of-the-art methods over all key quantitative evaluation metrics, including complex scenarios like partial occlusion, complex poses and even relatively low illumination.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
