arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.01993cs.ROcs.AI
收藏

论时间索引作为深度强化学习在序列操作任务中的归纳偏置

On Time-Indexing as Inductive Bias in Deep RL for Sequential Manipulation Tasks

Carnegie Mellon University(卡内基梅隆大学)

M. Nomaan Qureshi, Ben Eisner, David Held

AI总结:

本文提出一种在固定持续时间内顺序执行不同动作头的策略架构,解决了标准单输出头网络在多模态技能学习中的样本效率低和性能差问题,在 Metaworld 任务上表现优于标准方法。

摘要:

在解决复杂操作任务时,操作策略通常需要学习一组多样化的技能来完成这些任务。这组技能通常具有高度多模态性——每种技能可能具有截然不同的动作和状态分布。标准的深度策略学习算法通常将策略建模为具有单一输出头(确定性或随机性)的深度神经网络。这种结构要求网络在内部学习如何在不同模式之间进行切换,这可能导致样本效率降低和性能不佳。在本文中,我们探索了一种简单的结构,该结构有助于许多操作任务所需的技能学习。具体而言,我们提出了一种策略架构,该架构在固定持续时间内顺序执行不同的动作头,从而能够学习诸如到达和抓取等基本技能。我们在 Metaworld 任务上的实证评估表明,这种简单的结构优于标准的策略学习方法,凸显了其在改进技能获取方面的潜力。

英文摘要:

While solving complex manipulation tasks, manipulation policies often need to learn a set of diverse skills to accomplish these tasks. The set of skills is often quite multimodal - each one may have a quite distinct distribution of actions and states. Standard deep policy-learning algorithms often model policies as deep neural networks with a single output head (deterministic or stochastic). This structure requires the network to learn to switch between modes internally, which can lead to lower sample efficiency and poor performance. In this paper we explore a simple structure which is conducive to skill learning required for so many of the manipulation tasks. Specifically, we propose a policy architecture that sequentially executes different action heads for fixed durations, enabling the learning of primitive skills such as reaching and grasping. Our empirical evaluation on the Metaworld tasks reveals that this simple structure outperforms standard policy learning methods, highlighting its potential for improved skill acquisition.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑