Sim2real图像翻译实现从固定相机数据集的视角鲁棒策略
Sim2real Image Translation Enables Viewpoint-Robust Policies from Fixed-Camera Datasets
AI总结：
MANGO通过无配对图像翻译方法，从固定相机数据集生成视角鲁棒的机器人策略，显著提升现实任务中的视角适应能力。
摘要：
基于视觉的机器人操作策略已取得显著近期成功，但仍然对分布偏移（如摄像机视角变化）敏感。机器人演示数据稀缺且往往缺乏适当变化的摄像机视角。模拟提供了一种收集机器人演示数据的方式，能够全面覆盖不同视角，但带来了视觉sim2real挑战。为弥合这一差距，我们提出了MANGO——一种无配对图像翻译方法，包含一种新颖的分割条件化InfoNCE损失、一种高度正则化的判别器设计以及一种修改后的PatchNCE损失。我们发现这些元素对于在sim2real翻译中保持视角一致性至关重要。在训练MANGO时，我们只需要少量的固定相机现实数据，但证明我们的方法能够通过翻译模拟观测生成多样未见过的视角。在这一设置中，MANGO优于我们测试的所有其他图像翻译方法。在某些现实桌面上的操作任务中，MANGO的增强使移位视角的成功率比未经过增强训练的策略提高了超过40个百分点。
英文摘要：
Vision-based policies for robot manipulation have achieved significant recent success, but are still brittle to distribution shifts such as camera viewpoint variations. Robot demonstration data is scarce and often lacks appropriate variation in camera viewpoints. Simulation offers a way to collect robot demonstrations at scale with comprehensive coverage of different viewpoints, but presents a visual sim2real challenge. To bridge this gap, we propose MANGO -- an unpaired image translation method with a novel segmentation-conditioned InfoNCE loss, a highly-regularized discriminator design, and a modified PatchNCE loss. We find that these elements are crucial for maintaining viewpoint consistency during sim2real translation. When training MANGO, we only require a small amount of fixed-camera data from the real world, but show that our method can generate diverse unseen viewpoints by translating simulated observations. In this setting, MANGO outperforms all other image translation methods we tested. In certain real-world tabletop manipulation tasks, MANGO augmentation increases shifted-view success rates by over 40 percentage points compared to policies trained without augmentation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
