发表机构
Max Planck Institute for Informatics; Saarbrücken Research Center for Visual Computing, Interaction and AI; Google（马克斯·普朗克信息研究所; 萨尔布吕肯视觉计算、交互与人工智能研究中心; 谷歌）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究可穿戴设备运动捕捉问题，提出优先考虑轻便设备。贡献包括提供多模态数据集，提出WHIP模型，研究传感器互补性，以实现从任意传感器子集重建运动，处理缺失模态并生成合理运动。
AI 中文摘要
现代可穿戴设备的普及带来了独特的运动捕捉问题：从给定时刻佩戴的任何传感硬件重建全身运动。然而，大多数研究假设传感器配置固定，无法通用。相比之下，我们认为运动捕捉应优先考虑如智能手机等不显眼且轻便的设备，并研究它们之间的相互作用。为此，我们有三项贡献：一是提供大规模多模态数据集；二是提出WHIP基线生成模型；三是对传感器互补性进行系统研究。代码和数据集可获取。
英文摘要
The modern-day surge in popularity of wearable devices poses a fundamentally unique motion capture problem: reconstructing full-body movement from any set of sensing hardware worn at a given moment. Yet, most research efforts assume fixed sensor configurations (e.g. IMU suits or HMD-centric rigs) and cannot generalize across them. In contrast, we argue that motion capture should prioritize unobtrusive and lightweight devices such as smartphones, smartwatches, smart glasses, and smart insoles, and study the interplay between them. To this end, we make three contributions. First, we present a large-scale multi-modal dataset synchronizing these consumer-grade sensors with ground-truth 3D motion, spanning 50 diverse activities including everyday tasks, sports, and social interactions. Second, we propose WHIP, a baseline generative model that reconstructs motion from arbitrary subsets of available sensors, robustly handling missing modalities and producing physically plausible motions. Third, we conduct a systematic study of sensor complementarity, quantifying how different modalities complement one another. Code and dataset are available at https://vcai.mpi-inf.mpg.de/projects/WHIP/
CommentsAccepted at ECCV 2026
