发表机构
The Hong Kong University of Science and Technology（香港科技大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
提出WING框架，通过交互中心谱隐式引导，将人类第一人称视频中的交互知识迁移到机器人策略，在多个基准和真实任务上取得高成功率。
AI 中文摘要
学习通用机器人策略需要大规模的真实世界交互数据，然而收集机器人演示数据仍然昂贵且难以扩展。第一人称视频提供了丰富的、带有任务相关语义的人类交互经验，可用于机器人操作，但直接迁移面临两个挑战：从帧重建中推断出的隐式动作可能被诸如自我相机运动等干扰变化所主导，并且人类和机器人的行为往往表现出不同的时间动态。我们提出了WING（基于交互中心谱隐式引导的世界动作学习），一个将交互知识从第一人称视频迁移到机器人策略的框架。WING首先将观察者引起的运动与手物交互分离，并将交互中心组件提炼为隐式动作。然后，它利用跨实体任务语义集中在缓慢变化的时间结构中的观察结果，在谱域中识别第一人称隐式动作与机器人行为之间的共享低频组件，并用它们来指导动作生成。WING在LIBERO上实现了99.20%的平均成功率，在RoboTwin 2.0上实现了93.80%，在RoboCasa-GR1上实现了57.7%，并且在多样化的泛化设置下，在四个真实世界操作任务中也表现出色。这些结果表明，交互中心谱引导提供了一种有效且可扩展的方式，将人类第一人称视频中的物理交互知识迁移到机器人控制中。项目页面：此https URL
英文摘要
Learning general-purpose robot policies requires large-scale real-world interaction data, yet collecting robot demonstrations remains expensive and difficult to scale. Egocentric videos offer abundant human interaction experience with task-relevant semantics for robotic manipulation, but direct transfer is challenging for two reasons: latent actions inferred from frame reconstruction can be dominated by nuisance variation such as ego-camera motion, and human and robot behaviors often exhibit different temporal dynamics. We propose WING (World Action Learning via INteraction-Centric Spectral Latent Guidance), a framework for transferring interaction knowledge from egocentric videos to robot policies. WING first separates observer-induced motion from hand-object interaction and distills the interaction-centric component into latent actions. It then exploits the observation that cross-embodiment task semantics are concentrated in slowly varying temporal structures, identifying shared low-frequency components between egocentric latent actions and robot behaviors in the spectral domain and using them to guide action generation. WING achieves average success rates of 99.20% on LIBERO, 93.80% on RoboTwin 2.0, and 57.7% on RoboCasa-GR1, and also performs strongly across four real-world manipulation tasks under diverse generalization settings. These results show that interaction-centric spectral guidance provides an effective and scalable way to transfer physical interaction knowledge from human egocentric video to robot control. Project page: https://mikuz12.github.io/wing/
