VIP：面向机器人操作的视觉指令预训练
VIP: Vision Instructed Pre-training for Robotic Manipulation
AI总结：
针对机器人操作中训练数据扩展效果有限、文本指令理解困难的问题，提出视觉指令预训练（VIP）方法，利用稀疏点流细化目标描述，显著提升多样任务性能，可完成复杂操作任务。
摘要：
机器人操作中扩大训练数据的效果仍有限。操作的主要挑战在于任务多样，若任务目标未明确指定，训练出的策略会混淆。现有工作主要依赖文本指令描述目标，但我们发现当前机器人数据无法有效训练策略理解文本指令，而视觉更易理解。因此，我们引入视觉指令来指定目标。直接实现方式是训练策略预测连接当前观测与未来图像的中间动作，但单张未来图像对任务目标的描述不够详细。为此，我们提出使用稀疏点流提供更详细信息。基于真实和模拟环境设计了大量任务来评估视觉指令预训练（VIP）方法的效果，结果表明VIP显著提升了多样任务的性能，衍生策略可完成“打开密封瓶盖子”等复杂任务。
英文摘要：
The effectiveness of scaling up training data in robotic manipulation is still limited. A primary challenge in manipulation is the tasks are diverse, and the trained policy would be confused if the task targets are not specified clearly. Existing works primarily rely on text instruction to describe targets. However, we reveal that current robotic data cannot train policies to understand text instruction effectively, and vision is much more comprehensible. Therefore, we introduce utilizing vision instruction to specify targets. A straightforward implementation is training a policy to predict the intermediate actions linking the current observation and a future image. Nevertheless, a single future image does not describe the task target in insufficient detail. To handle this problem, we propose to use sparse point flows to provide more detailed information. Extensive tasks are designed based on real and simulated environments to evaluate the effectiveness of our vision instructed pre-training (VIP) method. The results indicate VIP improves the performance on diverse tasks significantly, and the derived policy can complete competitive tasks like ``opening the lid of a tightly sealed bottle''.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
