arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2211.09382cs.ROcs.CV
收藏

基于分层强化学习的不规则物体布局规划

Planning Irregular Object Packing via Hierarchical Reinforcement Learning

Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心);Tsinghua University(清华大学)

Sichao Huang, Ziwei Wang, Jie Zhou, Jiwen Lu

摘要:

利用自主机器人进行物品包装是仓库和物流行业的一个重要挑战。大多数传统的数据驱动的包装规划方法集中在规则的长方体包装,这通常是启发式的,并限制了实际应用中的日常物品。在本文中,我们提出了一个深层的强化学习方法,同时规划包装序列和放置不规则的物体包装。具体地说,顶层管理者网络从所有对象的六个主视图高度图推断出包装顺序,然后底层工作者网络接收下一个对象的高度图以预测放置位置和方向。这两个网络在自监督Q-Learning框架中进行分层训练,其中奖励由包装结果提供,包装结果基于顶部高度,物体体积和盒子中的放置稳定性。框架迭代地重复序列和放置规划,直到所有对象都被打包到盒子中或者没有空间留给未打包的物品。我们比较我们的方法与现有的机器人包装方法在物理模拟器中的不规则物体。实验表明,该方法可以包装更多的对象,以更少的时间成本比国家的最先进的不规则对象的包装方法。我们还实现了我们的包装计划与机器人机械手,以显示在现实世界中的泛化能力。

英文摘要:

Object packing by autonomous robots is an im-portant challenge in warehouses and logistics industry. Most conventional data-driven packing planning approaches focus on regular cuboid packing, which are usually heuristic and limit the practical use in realistic applications with everyday objects. In this paper, we propose a deep hierarchical reinforcement learning approach to simultaneously plan packing sequence and placement for irregular object packing. Specifically, the top manager network infers packing sequence from six principal view heightmaps of all objects, and then the bottom worker network receives heightmaps of the next object to predict the placement position and orientation. The two networks are trained hierarchically in a self-supervised Q-Learning framework, where the rewards are provided by the packing results based on the top height , object volume and placement stability in the box. The framework repeats sequence and placement planning iteratively until all objects have been packed into the box or no space is remained for unpacked items. We compare our approach with existing robotic packing methods for irregular objects in a physics simulator. Experiments show that our approach can pack more objects with less time cost than the state-of-the-art packing methods of irregular objects. We also implement our packing plan with a robotic manipulator to show the generalization ability in the real world.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑