EgoDemoGen：用于机器人操作中视角泛化的眼动演示生成
EgoDemoGen: Egocentric Demonstration Generation for Viewpoint Generalization in Robotic Manipulation
AI总结：
本文提出EgoDemoGen框架，通过EgoTrajTransfer和EgoViewTransfer生成新型眼动视角下的观察-动作演示，提升机器人操作在视角变化下的成功率。
摘要：
基于模仿学习的视觉-运动策略在机器人操作中取得了强劲表现，但往往对眼动视角变化敏感。不同于第三人称视角变化仅移动相机，眼动视角变化同时改变相机姿态和机器人动作坐标系，需要联合转移动作轨迹并合成相应观察。为此，我们提出EgoDemoGen框架，通过两个关键组件生成新型眼动视角下的观察-动作演示：1) EgoTrajTransfer通过运动技能分割、几何感知变换和逆运动学过滤将机器人轨迹转移到新型眼动坐标系；2) EgoViewTransfer是条件视频生成模型，融合新型视角投影场景视频和从转移轨迹渲染的机器人运动视频，生成逼真观察，通过自监督双重投影策略训练无需多视角数据。实验表明，EgoDemoGen在标准和新型眼动视角下均显著提高策略成功率，绝对提升分别为+24.6%和+16.9%（仿真）以及+16.0%和+23.0%（真实机器人）。此外，EgoViewTransfer在新型眼动观察生成中表现出色。
英文摘要：
Imitation learning based visuomotor policies have achieved strong performance in robotic manipulation, yet they often remain sensitive to egocentric viewpoint shifts. Unlike third-person viewpoint changes that only move the camera, egocentric shifts simultaneously alter both the camera pose and the robot action coordinate frame, making it necessary to jointly transfer action trajectories and synthesize corresponding observations under novel egocentric viewpoints. To address this challenge, we present EgoDemoGen, a framework that generates paired observation--action demonstrations under novel egocentric viewpoints through two key components: 1{)} EgoTrajTransfer, which transfers robot trajectories to the novel egocentric coordinate frame through motion-skill segmentation, geometry-aware transformation, and inverse kinematics filtering; and 2{)} EgoViewTransfer, a conditional video generation model that fuses a novel-viewpoint reprojected scene video and a robot motion video rendered from the transferred trajectory to synthesize photorealistic observations, trained with a self-supervised double reprojection strategy without requiring multi-viewpoint data. Experiments in simulation and real-world settings show that EgoDemoGen consistently improves policy success rates under both standard and novel egocentric viewpoints, with absolute gains of +24.6\% and +16.9\% in simulation and +16.0\% and +23.0\% on the real robot. Moreover, EgoViewTransfer achieves superior video generation quality for novel egocentric observations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
