arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2205.07562cs.LGcs.AIcs.RO
收藏

非平稳相互依赖任务的自主开放学习

Autonomous Open-Ended Learning of Tasks with Non-Stationary Interdependencies

Universidade da Coruña(拉科鲁尼亚大学);Consiglio Nazionale delle Ricerche(意大利国家研究委员会);Istituto di Scienze e Tecnologie della Cognizione(认知科学与技术研究所);CITIC research center(CITIC研究中心)

Alejandro Romero, Gianluca Baldassarre, Richard J. Duro, Vieri Giuliano Santucci

摘要:

自主开放式学习是机器学习和机器人技术中的一种相关方法,允许设计能够获得目标和运动技能的人工智能体,而无需用户分配任务。这种方法的一个关键问题是制定策略,以确保代理商能够在尽可能短的时间内最大限度地提高他们在尽可能多的任务上的能力。内在动机已被证明可以产生一个与任务无关的信号,以便在目标之间正确分配训练时间。虽然大多数的作品在该领域的内在动机的开放式学习的重点是场景中的目标是相互独立的,只有少数人研究的自主获取的相互依存的任务,更少的解决方案,其中的目标涉及非静态的相互依存关系。在以前工作的基础上,我们在决策层面解决这些关键问题(即,建立战略,以适当地选择目标之间),我们提出了一个分层架构,将子任务的选择作为一个马尔可夫决策过程是能够正确地学习相互依赖的技能的基础上,内在产生的动机。特别是,我们首先深化以前的系统的分析,显示在更高层次的架构(目标选择)的任务之间的关系,将信息的重要性。然后,我们介绍H-GRAIL,一个新的系统,通过添加一个新的学习层来存储自主获取的任务序列,以便在相互依赖性不稳定的情况下修改它们,从而扩展了以前的系统。所有系统都在真实的机器人场景中进行了测试,Baxter机器人执行多个相互依赖的到达任务。

英文摘要:

Autonomous open-ended learning is a relevant approach in machine learning and robotics, allowing the design of artificial agents able to acquire goals and motor skills without the necessity of user assigned tasks. A crucial issue for this approach is to develop strategies to ensure that agents can maximise their competence on as many tasks as possible in the shortest possible time. Intrinsic motivations have proven to generate a task-agnostic signal to properly allocate the training time amongst goals. While the majority of works in the field of intrinsically motivated open-ended learning focus on scenarios where goals are independent from each other, only few of them studied the autonomous acquisition of interdependent tasks, and even fewer tackled scenarios where goals involve non-stationary interdependencies. Building on previous works, we tackle these crucial issues at the level of decision making (i.e., building strategies to properly select between goals), and we propose a hierarchical architecture that treating sub-tasks selection as a Markov Decision Process is able to properly learn interdependent skills on the basis of intrinsically generated motivations. In particular, we first deepen the analysis of a previous system, showing the importance of incorporating information about the relationships between tasks at a higher level of the architecture (that of goal selection). Then we introduce H-GRAIL, a new system that extends the previous one by adding a new learning layer to store the autonomously acquired sequences of tasks to be able to modify them in case the interdependencies are non-stationary. All systems are tested in a real robotic scenario, with a Baxter robot performing multiple interdependent reaching tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑