MAEPose: 基于毫米波视频的人体姿态估计的自监督时空学习
MAEPose: Self-Supervised Spatiotemporal Learning for Human Pose Estimation on mmWave Video
AI总结：
提出MAEPose，一种直接处理毫米波频谱视频的掩码自编码方法，通过自监督时空学习实现鲁棒的人体姿态估计，在三个数据集上优于现有方法。
摘要：
毫米波雷达为基于RGB的人体姿态估计提供了一种更具隐私保护性的替代方案。然而，现有方法通常依赖预提取的中间表示，如稀疏点云或频谱图图像，这些方法丢弃了雷达视频流中自然存在的丰富时空信息用于模型学习，同时此类信号处理增加了系统复杂性。此外，现有解决方案主要采用端到端监督方式，未利用未标记的原始视频流来学习通用表示。在本研究中，我们提出MAEPose，一种基于掩码自编码的人体姿态估计方法，直接处理毫米波频谱视频。MAEPose从未标记的雷达视频中学习时空运动感知的通用表示，并利用其热图解码器进行多帧姿态估计预测。我们基于留一法交叉验证和严格的统计检验，在三个数据集上对其进行评估。MAEPose在MPJPE指标上始终优于最先进的基线方法，最高提升22.1%（p<0.05），并且在零样本旁观者干扰下保持鲁棒精度，误差仅增加6.5%。消融研究证实，预训练和热图解码器均有显著贡献，而模态分析表明，使用距离-多普勒视频作为输入比距离-方位角或其融合能实现更好的姿态估计性能，且计算成本更低。
英文摘要：
Millimetre-wave (mmWave) radar offers a more privacy-preserving alternative to RGB-based human pose estimation. However, existing methods typically rely on pre-extracted intermediate representations such as sparse point clouds or spectrogram images, where the rich spatiotemporal information naturally present in radar video streams is discarded for model learning, while such signal processing adds system complexity. In addition, existing solutions are mainly conducted in an end-to-end supervised manner without leveraging unlabelled raw video streams to learn generalized representations. In this study, we present MAEPose, a masked autoencoding-based human pose estimation approach that operates directly on mmWave spectrogram videos. MAEPose learns spatiotemporal motion-aware generalized representations from unlabelled radar video, and leverages its heatmap decoder for multi-frame pose estimation predictions. We evaluate it across three datasets based on leave-one-person-out cross-validation with rigorous statistical testing. MAEPose consistently outperforms state-of-the-art baselines by up to 22.1% in MPJPE p<0.05, and maintains robust accuracy under zero-shot bystander interference with only a 6.5% error increase. Ablation studies confirm that both the pre-training and the heatmap decoder contribute substantially, while modality analysis indicates that leveraging Range-Doppler video as input achieves better pose estimation performance than Range-Azimuth or their fusion, with lower computational cost.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
