arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2105.05484cs.RO
收藏

基于技能序列的长视野操作策略学习

Learning a Skill-sequence-dependent Policy for Long-horizon Manipulation Tasks

Xi’an Jiaotong-Liverpool University(西交利物浦大学);The Chinese University of Hong Kong (Shenzhen)(香港中文大学(深圳));Shenzhen Institute of Artificial Intelligence and Robotics for Society(深圳人工智能与机器人研究院)

Zhihao Li, Zhenglong Sun, Jionglong SU, Jiaming Zhang

摘要:

近年来,机器人社区使用深度强化学习(RL)在机器人操作方面取得了实质性进展。有效地学习长期任务仍然是一个具有挑战性的课题。典型的基于RL的方法将长时间任务近似为马尔可夫决策过程,并且仅将当前观测(图像或其他传感器信息)作为输入状态。然而,这种近似忽略了这样一个事实,即技能序列在长视野任务中也起着至关重要的作用。在本文中,我们考虑到观察和技能序列,并提出了一个技能序列依赖的分层策略,解决了一个典型的长期任务。该策略由高级技能策略(利用技能序列)和低级参数策略(响应观察)以及相应的训练方法组成,这使得学习更具样本效率。仿真实验表明,该方法成功地解决了一个长期的任务,并显着快于近端策略优化(PPO)和任务模式的方法。

英文摘要:

In recent years, the robotics community has made substantial progress in robotic manipulation using deep reinforcement learning (RL). Effectively learning of long-horizon tasks remains a challenging topic. Typical RL-based methods approximate long-horizon tasks as Markov decision processes and only consider current observation (images or other sensor information) as input state. However, such approximation ignores the fact that skill-sequence also plays a crucial role in long-horizon tasks. In this paper, we take both the observation and skill sequences into account and propose a skill-sequence-dependent hierarchical policy for solving a typical long-horizon task. The proposed policy consists of a high-level skill policy (utilizing skill sequences) and a low-level parameter policy (responding to observation) with corresponding training methods, which makes the learning much more sample-efficient. Experiments in simulation demonstrate that our approach successfully solves a long-horizon task and is significantly faster than Proximal Policy Optimization (PPO) and the task schema methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑