AI 中文总结
研究针对毫米波雷达人体运动重建中数据收集成本高及模拟训练模型迁移差的问题，提出mmSimPrior框架，将知识分解为先验，经多模态编码器等训练，构建数据集并设无重叠设置，实验证明其在降低MPJPE上有显著效果。
AI 中文摘要
毫米波雷达为人体运动重建提供了隐私保护和抗光照的传感，但跨实际部署进行泛化的学习模型需要收集成本高昂的多样配对雷达-运动数据。模拟提供了可扩展的监督，然而在干净合成信号上训练的模型由于多径、杂波、响应统计和分辨率下降而难以迁移。我们提出了mmSimPrior，一个模拟预训练框架，将可迁移知识分解为信号、运动和雷达到运动映射先验。一个多模态信号编码器通过模拟传播和采集级变化的物理信息域随机化课程进行预训练，而联合时间分词器学习关于合理人体运动的离散先验。一个共享映射先验支持在学习的运动码本上进行分类以进行受限零样本重建，并支持连续回归以进行灵活的有限数据适配。我们进一步构建了一个420万帧、31K序列的数据集套件，并引入了无重叠设置，该设置排除了跨适配和测试的重复完整主体-环境-位置-运动配置。在mmSimPrior-Real和RT-Pose上的实验证明了一致的收益：仅使用24个配对真实序列，mmSimPrior-Reg在三种环境中比最强基线将MPJPE降低了24.7%至39.0%，而mmSimPrior-Cls在不进行微调的情况下将零样本MPJPE降低了8.5%。
英文摘要
Millimeter-wave (mmWave) radar enables privacy-preserving and illumination-robust human motion reconstruction, but training generalizable models typically requires costly paired radar-motion recordings. Simulation can scale such supervision, yet even physics-based simulators cannot fully reproduce real-world multipath, clutter, hardware-specific response statistics, or distance-dependent resolution degradation, leaving a sim-to-real gap. We present mmSimPrior, a simulation-pretrained framework that factorizes transferable knowledge into signal, motion, and radar-to-motion mapping priors. To learn transferable signal and motion priors, we pretrain a multimodal radar encoder with a physics-informed domain-randomization curriculum designed to mitigate the sim-to-real gap by approximating real-world propagation- and acquisition-level variations, while a joint-temporal tokenizer learns a discrete prior over plausible human motion. A dual-mode mapping module predicts either motion-code distributions for structurally constrained zero-shot reconstruction or continuous motion parameters for flexible adaptation from limited real data. We further construct a 4.2M-frame, 31K-sequence dataset suite and introduce a No-Overlap Setting that prevents any exact subject-environment-location-motion tuple from appearing in both the adaptation and test sets. Experiments on mmSimPrior-Real and RT-Pose demonstrate consistent gains: with only 24 paired real sequences, mmSimPrior-Reg reduces MPJPE by 24.7-39.0% over the strongest baseline across the three environments, while mmSimPrior-Cls reduces zero-shot MPJPE by 8.5% without fine-tuning.
Comments19 pages, 11 figures, including supplementary material. Project page: https://ch3ngguo.github.io/mmsimprior/
