MOSAIC: 通过快速残差适应弥合通用人形运动追踪与远程操作的仿真到现实差距
MOSAIC: Bridging the Sim-to-Real Gap in Generalist Humanoid Motion Tracking and Teleoperation with Rapid Residual Adaptation
AI总结:
MOSAIC通过快速残差适应技术,实现通用人形运动追踪与远程操作在仿真到现实中的高效过渡
摘要:
通用人形运动追踪器最近通过扩展数据和训练实现了强大的仿真指标,但在持续远程操作中往往因接口和动态引起的误差而显得脆弱。我们提出了MOSAIC,一个开源的全栈系统,用于在多个接口上的人形运动追踪和全身远程操作。MOSAIC首先通过在多源运动库上使用强化学习学习一个面向远程操作的通用运动追踪器,通过自适应重采样和奖励机制强调世界框架的运动一致性,这对于移动远程操作至关重要。为了在不牺牲通用性的情况下弥合仿真到现实的接口差距,MOSAIC随后进行快速残差适应:通过使用最小的接口特定数据训练一个接口特定的策略,并通过加法残差模块将其蒸馏到通用追踪器中,优于朴素微调或持续学习。我们通过系统性的消融分析、分布外基准测试和实际机器人实验验证了MOSAIC,展示了在现实延迟和噪声下的稳健离线运动回放和在线长视距远程操作。项目页面:baai-humanoid.github.io/MOSAIC。
英文摘要:
Generalist humanoid motion trackers have recently achieved strong simulation metrics by scaling data and training, yet often remain brittle on hardware during sustained teleoperation due to interface- and dynamics-induced errors. We present MOSAIC, an open-source, full-stack system for humanoid motion tracking and whole-body teleoperation across multiple interfaces. MOSAIC first learns a teleoperation-oriented general motion tracker via RL on a multi-source motion bank with adaptive resampling and rewards that emphasize world-frame motion consistency, which is critical for mobile teleoperation. To bridge the sim-to-real interface gap without sacrificing generality, MOSAIC then performs rapid residual adaptation: an interface-specific policy is trained using minimal interface-specific data, and then distilled into the general tracker through an additive residual module, outperforming naive fine-tuning or continual learning. We validate MOSAIC with systematic ablations, out-of-distribution benchmarking, and real-robot experiments demonstrating robust offline motion replay and online long-horizon teleoperation under realistic latency and noise. Project page: baai-humanoid.github.io/MOSAIC.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。