GVCCI：面向语言引导机器人操作的视觉定位终身学习
GVCCI: Lifelong Learning of Visual Grounding for Language-Guided Robotic Manipulation
AI总结：
针对语言引导机器人操作中预训练视觉定位模型领域差距大、标注成本高的问题，提出无监督终身学习框架 GVCCI，通过自生成合成指令持续训练 VG，并贡献近 252k 三元组数据集。
摘要：
语言引导机器人操作（LGRM）是一项具有挑战性的任务，因为它要求机器人理解人类指令以操作日常物体。近期的 LGRM 方法依赖预训练的视觉定位（VG）模型来检测物体，却没有针对操作环境进行适配。由于预训练数据与真实世界数据之间存在巨大的领域差距，这会导致性能下降。一个直接的解决方案是收集额外的训练数据，但人工标注成本极其高昂。本文提出 Grounding Vision to Ceaselessly Created Instructions（GVCCI），这是一个用于 LGRM 的终身学习框架，可在无人工监督的情况下持续学习 VG。GVCCI 通过物体检测迭代生成合成指令，并利用生成的数据训练 VG 模型。我们在不同环境中、针对不同 VG 模型，在离线和在线设置下验证了该框架。实验结果表明，积累来自 GVCCI 的合成数据可使 VG 稳定提升，最高达 56.7%，并使由此产生的 LGRM 性能最高提升 29.4%。此外，定性分析表明，未经适配的 VG 模型由于从预训练数据中学到的强烈偏差，往往无法找到正确物体。最后，我们引入了一个用于 LGRM 的新型 VG 数据集，其中包含来自多样化操作环境的近 252k 个图像-物体-指令三元组。
英文摘要：
Language-Guided Robotic Manipulation (LGRM) is a challenging task as it requires a robot to understand human instructions to manipulate everyday objects. Recent approaches in LGRM rely on pre-trained Visual Grounding (VG) models to detect objects without adapting to manipulation environments. This results in a performance drop due to a substantial domain gap between the pre-training and real-world data. A straightforward solution is to collect additional training data, but the cost of human-annotation is extortionate. In this paper, we propose Grounding Vision to Ceaselessly Created Instructions (GVCCI), a lifelong learning framework for LGRM, which continuously learns VG without human supervision. GVCCI iteratively generates synthetic instruction via object detection and trains the VG model with the generated data. We validate our framework in offline and online settings across diverse environments on different VG models. Experimental results show that accumulating synthetic data from GVCCI leads to a steady improvement in VG by up to 56.7% and improves resultant LGRM by up to 29.4%. Furthermore, the qualitative analysis shows that the unadapted VG model often fails to find correct objects due to a strong bias learned from the pre-training data. Finally, we introduce a novel VG dataset for LGRM, consisting of nearly 252k triplets of image-object-instruction from diverse manipulation environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
