MoBind：用于细粒度IMU-视频姿态对齐的运动绑定
MoBind: Motion Binding for Fine-Grained IMU-Video Pose Alignment
AI总结：
MoBind通过层次对比学习框架实现IMU与视频姿态的细粒度对齐，提升跨模态检索与动作识别性能。
摘要：
我们旨在学习一种联合表示，将惯性测量单元（IMU）信号与从视频中提取的2D姿态序列联系起来，从而实现准确的跨模态检索、时间同步、主体和身体部位定位以及动作识别。为此，我们介绍了MoBind，一种层次对比学习框架，旨在解决三个挑战：（1）过滤掉无关的视觉背景，（2）建模结构化的多传感器IMU配置，（3）实现细粒度、亚秒级的时间对齐。为了隔离与运动相关的线索，MoBind将IMU信号与骨骼运动序列对齐，而不是原始像素。我们进一步将全身运动分解为局部身体部位轨迹，将每个轨迹与其对应的IMU配对，以实现语义基础的多传感器对齐。为了捕捉详细的时序对应关系，MoBind采用层次对比策略，首先对token级的时间段进行对齐，然后将局部（身体部位）对齐与全局（身体范围）运动聚合融合。在mRi、TotalCapture和EgoHumans数据集上评估，MoBind在所有四个任务中均优于强大的基线，证明了其在保持模态间粗粒度语义一致性的同时，具有稳健的细粒度时间对齐能力。代码可在https://github.com/bbvisual/MoBind获取。
英文摘要：
We aim to learn a joint representation between inertial measurement unit (IMU) signals and 2D pose sequences extracted from video, enabling accurate cross-modal retrieval, temporal synchronization, subject and body-part localization, and action recognition. To this end, we introduce MoBind, a hierarchical contrastive learning framework designed to address three challenges: (1) filtering out irrelevant visual background, (2) modeling structured multi-sensor IMU configurations, and (3) achieving fine-grained, sub-second temporal alignment. To isolate motion-relevant cues, MoBind aligns IMU signals with skeletal motion sequences rather than raw pixels. We further decompose full-body motion into local body-part trajectories, pairing each with its corresponding IMU to enable semantically grounded multi-sensor alignment. To capture detailed temporal correspondence, MoBind employs a hierarchical contrastive strategy that first aligns token-level temporal segments, then fuses local (body-part) alignment with global (body-wide) motion aggregation. Evaluated on mRi, TotalCapture, and EgoHumans, MoBind consistently outperforms strong baselines across all four tasks, demonstrating robust fine-grained temporal alignment while preserving coarse semantic consistency across modalities. Code is available at https://github.com/bbvisual/ MoBind.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
