ActiveMimic: 基于主动感知的自我中心视频预训练
ActiveMimic: Egocentric Video Pretraining with Active Perception
AI总结：
提出ActiveMimic框架，从自我中心人类视频中恢复同步的相机和手腕轨迹，将相机运动建模为视角动作，联合学习主动感知和操作技能，使预训练模型在机器人任务上达到与机器人数据预训练相当的性能。
摘要：
自我中心人类视频为机器人数据预训练提供了一种可扩展的替代方案，但在此类视频上预训练的模型始终不如在机器人数据上预训练的模型。我们将这一差距归因于缺失的信号，即自我中心视频中的主动感知行为，其中人类在操作过程中不断重新定位视角，导致标准流程视为噪声的相机运动。为解决这一问题，我们提出了ActiveMimic，一个预训练框架，从单个身体佩戴的RGB相机中恢复同步的相机和手腕轨迹，将相机运动建模为视角动作，并在适应目标机器人之前，从野外自我中心人类视频中联合学习主动感知和操作。实验表明，在具有不同主动感知需求的任务中，ActiveMimic始终优于在人类视频上预训练的基线，并与在机器人数据上预训练的最先进模型相匹配。进一步分析提供了证据，表明主动感知能力源自自我中心人类视频预训练而非机器人特定微调，确认了主动感知是解锁自我中心人类视频用于机器人预训练的关键。
英文摘要：
Egocentric human video offers a scalable alternative to robot data for pretraining, yet models pretrained on such video consistently underperform those pretrained on robot data. We attribute this gap to a missing signal, the active perception behavior in egocentric videos, where humans continuously reposition their viewpoint during manipulation, inducing camera motion that standard pipelines treat as noise. To address this, we present ActiveMimic, a pretraining framework that recovers synchronized camera and wrist trajectories from a single body-worn RGB camera, models camera motion as a viewpoint action, and jointly learns active perception and manipulation from in-the-wild egocentric human video before adapting to a target robot. Empirically, real-world experiments across tasks with diverse active perception demands show that ActiveMimic consistently surpasses baselines pretrained on human video and matches state-of-the-art models pretrained on robot data. Further analysis provides evidence that active perception capability originates from egocentric human video pretraining rather than robot-specific fine-tuning, confirming active perception as the key to unlocking egocentric human video for robot pretraining.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
