基于快慢思考的语言条件机器人操作
Language-Conditioned Robotic Manipulation with Fast and Slow Thinking
AI总结：
受人类双加工理论启发，提出RFST快慢思考机器人操作框架，通过指令判别器切换快慢思考系统，结合微调视觉语言模型实现复杂推理任务，在仿真与真实场景中均验证了有效性。
摘要：
语言条件机器人操作旨在将自然语言指令转化为可执行动作，涵盖从简单的拾取放置到需要意图识别与视觉推理的各类任务。受认知科学中双加工理论（该理论提出人类决策存在快思考与慢思考两个并行系统）的启发，我们提出了Robotics with Fast and Slow Thinking（RFST，快慢思考机器人）框架，该框架模拟人类认知结构，根据指令类型对任务分类并在两个系统间进行决策。我们的RFST包含两个核心组件：1）指令判别器，用于根据当前用户指令确定应激活哪个系统；2）慢思考系统，由与策略网络对齐的微调视觉语言模型构成，可让机器人识别用户意图或执行推理任务。为评估我们的方法，我们构建了一个包含真实世界轨迹的数据集，涵盖从自发冲动性动作到需要深思熟虑的任务的各类动作。我们在仿真和真实场景中的结果均证实，该方法能出色处理需要意图识别与推理的复杂任务。项目主页为https://jlm-z.github.io/RSFT/
英文摘要：
The language-conditioned robotic manipulation aims to transfer natural language instructions into executable actions, from simple pick-and-place to tasks requiring intent recognition and visual reasoning. Inspired by the dual process theory in cognitive science, which suggests two parallel systems of fast and slow thinking in human decision-making, we introduce Robotics with Fast and Slow Thinking (RFST), a framework that mimics human cognitive architecture to classify tasks and makes decisions on two systems based on instruction types. Our RFST consists of two key components: 1) an instruction discriminator to determine which system should be activated based on the current user instruction, and 2) a slow-thinking system that is comprised of a fine-tuned vision language model aligned with the policy networks, which allows the robot to recognize user intention or perform reasoning tasks. To assess our methodology, we built a dataset featuring real-world trajectories, capturing actions ranging from spontaneous impulses to tasks requiring deliberate contemplation. Our results, both in simulation and real-world scenarios, confirm that our approach adeptly manages intricate tasks that demand intent recognition and reasoning. The project is available at https://jlm-z.github.io/RSFT/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
