Wave2Body：将毫米波人体姿态估计重新思考为雷达到身体的令牌转换

中文摘要 AI 辅助

毫米波雷达实现了对人体隐私友好的感知，但其稀疏点云是依赖视角的电磁反射的物理测量，只能间接表征身体关节。从这种部分的、依赖几何的观测中恢复完整的3D姿态存在约束不足的问题。现有方法直接从配对的雷达-姿态数据中回归关节坐标，依赖相同有限的配对监督来学习雷达感知、人体结构及其对齐。这种耦合会在模糊的雷达观测下催生特定数据集的捷径。我们提出Wave2Body，一个雷达到身体的令牌转换框架，使用自监督毫米波令牌器、预训练的组合身体令牌器（定义输出空间）以及它们之间的轻量级翻译器来解耦这些学习目标。在M4Human和mmBody上的实验表明，Wave2Body比以前的方法具有更强的跨域泛化能力，同时训练和推理的计算成本要低得多。所有代码和实验结果可在该https URL公开获取。

英文摘要

Millimeter-wave (mmWave) radar enables privacy-friendly human sensing, but its sparse point clouds are physical measurements of view-dependent electromagnetic reflections and only indirectly characterize body articulation. Recovering a complete 3D pose from such partial, geometry-dependent observations is therefore under-constrained. Existing methods directly regress joint coordinates from paired radar-pose data, relying on the same limited paired supervision to learn radar perception, human-body structure, and their alignment. This coupling can encourage dataset-specific shortcuts under ambiguous radar observations. We propose Wave2Body, a radar-to-body token translation framework that decouples these learning targets using a self-supervised mmWave tokenizer, a pretrained compositional body tokenizer that defines the output space, and a lightweight translator between them. Experiments on M4Human and mmBody show that Wave2Body achieves stronger cross-domain generalization than previous methods while incurring much lower computational costs for training and inference. All the code and experiment results are publicly available at https://github.com/Galaxywalk/Wave2Body.

发表机构

• School of Computer Science, Peking University（北京大学计算机科学学院）
• University of Pittsburgh（匹兹堡大学）
• Key Laboratory of High Confidence Software Technologies, Ministry of Education (PKU)（教育部高可信软件技术重点实验室（北京大学））

机构由 AI 辅助整理，请以论文原文为准。