面向语言引导的多任务机器人操作对比模仿学习
Contrastive Imitation Learning for Language-guided Multi-Task Robotic Manipulation
AI总结：
本文提出Sigma-Agent，通过对比模仿学习和多视图查询Transformer增强视觉-语言与当前-未来表示，在RLBench和真实操作任务中显著提升多任务机器人操作性能。
摘要：
开发能够在自然语言指令和复杂真实环境视觉观测引导下执行各种操作任务的机器人，仍然是机器人领域的重大挑战。这类机器人智能体需要理解语言命令，并区分不同任务的要求。在本工作中，我们提出了Sigma-Agent，一种用于多任务机器人操作的端到端模仿学习智能体。Sigma-Agent融合了对比模仿学习（contrastive IL）模块，以增强视觉-语言表示和当前-未来表示。我们引入了一种有效且高效的多视图查询Transformer（MVQ-Former），用于聚合具有代表性的语义信息。Sigma-Agent在18个RLBench任务的多种设置下相比最先进方法表现出显著提升，在10次和100次演示训练中分别平均超过RVT 5.2%和5.9%。Sigma-Agent还在5个真实世界操作任务中以单一策略实现了62%的成功率。代码将在录用后发布。
英文摘要：
Developing robots capable of executing various manipulation tasks, guided by natural language instructions and visual observations of intricate real-world environments, remains a significant challenge in robotics. Such robot agents need to understand linguistic commands and distinguish between the requirements of different tasks. In this work, we present Sigma-Agent, an end-to-end imitation learning agent for multi-task robotic manipulation. Sigma-Agent incorporates contrastive Imitation Learning (contrastive IL) modules to strengthen vision-language and current-future representations. An effective and efficient multi-view querying Transformer (MVQ-Former) for aggregating representative semantic information is introduced. Sigma-Agent shows substantial improvement over state-of-the-art methods under diverse settings in 18 RLBench tasks, surpassing RVT by an average of 5.2% and 5.9% in 10 and 100 demonstration training, respectively. Sigma-Agent also achieves 62% success rate with a single policy in 5 real-world manipulation tasks. The code will be released upon acceptance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
