机器人视觉泛化中的不变性协同训练
Invariance Co-training for Robot Visual Generalization
AI总结：
该研究通过引入状态相似性和观测扰动不变性辅助任务，提升机器人在多样视角、光照和干扰物体下的泛化能力，实验表明其性能比现有方法提高了18%。
摘要：
从多样化的观察中进行推理是通用机器人策略在广泛环境中操作的基本能力。尽管近年来取得了进展，但许多大规模机器人策略仍然对关键的观测变化源敏感，如摄像机视角的变化、光照条件和干扰物体的存在。我们提出，这些模型的有限泛化能力源于大量多样性需求以稳健地覆盖这些准静态轴，同时当前大规模机器人数据集在这些方面表现出丰富变化的稀缺性。在本工作中，我们通过引入两个关键辅助任务，状态相似性和对观测扰动的不变性，应用于演示数据和静态视觉数据，系统地探讨了机器人需要如何在这些具有挑战性的轴上进行泛化。然后我们展示，通过这些辅助任务，利用更昂贵的机器人演示数据和更便宜但视觉丰富的非物理模拟生成的图像（例如Unreal Engine），可以显著提高对未见摄像机视角、光照配置和干扰条件的泛化能力。我们的结果表明，在这种多样化数据上进行协同训练比现有生成增强方法提高了18%的性能。更多信息和视频请访问https://invariance-cotraining.github.io
英文摘要：
Reasoning from diverse observations is a fundamental capability for generalist robot policies to operate in a wide range of environments. Despite recent advancements, many large-scale robotic policies still remain sensitive to key sources of observational variation such as changes in camera perspective, lighting, and the presence of distractor objects. We posit that the limited generalizability of these models arises from the substantial diversity required to robustly cover these quasistatic axes, coupled with the current scarcity of large-scale robotic datasets that exhibit rich variation across them. In this work, we propose to systematically examine what robots need to generalize across these challenging axes by introducing two key auxiliary tasks, state similarity and invariance to observational perturbations, applied to both demonstration data and static visual data. We then show that via these auxiliary tasks, leveraging both more-expensive robotic demonstration data and less-expensive, visually rich synthetic images generated from non-physics-based simulation (for example, Unreal Engine) can lead to substantial increases in generalization to unseen camera viewpoints, lighting configurations, and distractor conditions. Our results demonstrate that co-training on this diverse data improves performance by 18 percent over existing generative augmentation methods. For more information and videos, please visit https://invariance-cotraining.github.io
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
