arXiv 2311.112872023-11-19cs.ROcs.AI

用于高效机器人操作技能获取的触觉主动推断强化学习

Tactile Active Inference Reinforcement Learning for Efficient Robotic Manipulation Skill Acquisition

Research Center for Intelligent Robotics, School of Astronautics, Northwestern Polytechnical University（西北工业大学航天学院智能机器人研究中心）；National Key Laboratory of Aerospace Flight Dynamics, Northwestern Polytechnical University（西北工业大学航空航天飞行动力学全国重点实验室）

Zihao Liu, Xing Liu, Yizhai Zhang, Zhengxiong Liu, Panfeng Huang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出触觉主动推断强化学习（Tactile-AIRL），结合主动推断与视觉触觉感知，通过自由能最小化进行规划，显著提升了机器人操作在密集和稀疏奖励任务下的训练效率与实用性。

摘要：

机器人操作具有在执行繁琐或危险任务时取代人类的潜力。然而，由于难以在形式上描述现实中开放世界的操作，以及现有学习方法效率低下，基于控制的方法并不适用。因此，在广泛场景中应用操作面临着重大挑战。在本研究中，我们提出了一种用于机器人操作技能学习的新方法，称为触觉主动推断强化学习（Tactile-AIRL），旨在实现高效训练。为了增强强化学习（RL）的性能，我们引入了主动推断，将基于模型的技术和内在好奇心整合到RL过程中。这种整合提高了算法的训练效率和对稀疏奖励的适应性。此外，我们利用基于视觉的触觉传感器为操作任务提供详细的感知。最后，我们采用基于模型的方法，通过自由能最小化来想象并规划适当的动作。仿真结果表明，我们的方法在非抓握式物体推动任务中实现了显著的高训练效率。它使智能体能够仅凭少量交互回合在密集和稀疏奖励任务中均表现出色，超越了SAC基线。此外，我们使用该方法在夹爪拧螺丝任务上进行了物理实验，展示了算法的快速学习能力及其在实际应用中的潜力。

英文摘要：

Robotic manipulation holds the potential to replace humans in the execution of tedious or dangerous tasks. However, control-based approaches are not suitable due to the difficulty of formally describing open-world manipulation in reality, and the inefficiency of existing learning methods. Thus, applying manipulation in a wide range of scenarios presents significant challenges. In this study, we propose a novel method for skill learning in robotic manipulation called Tactile Active Inference Reinforcement Learning (Tactile-AIRL), aimed at achieving efficient training. To enhance the performance of reinforcement learning (RL), we introduce active inference, which integrates model-based techniques and intrinsic curiosity into the RL process. This integration improves the algorithm's training efficiency and adaptability to sparse rewards. Additionally, we utilize a vision-based tactile sensor to provide detailed perception for manipulation tasks. Finally, we employ a model-based approach to imagine and plan appropriate actions through free energy minimization. Simulation results demonstrate that our method achieves significantly high training efficiency in non-prehensile objects pushing tasks. It enables agents to excel in both dense and sparse reward tasks with just a few interaction episodes, surpassing the SAC baseline. Furthermore, we conduct physical experiments on a gripper screwing task using our method, which showcases the algorithm's rapid learning capability and its potential for practical applications.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。