arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.20573cs.AIcs.HCcs.LGcs.RO
收藏

序列任务中强化学习与模仿学习的内在动机

The intrinsic motivation of reinforcement and imitation learning for sequential tasks

ETIS Lab(ETIS实验室);Idiap Research Institute(Idiap研究所);Universidade da Coruna(拉科鲁尼亚大学);Okinawa Institute of Science and Technology(冲绳科学技术大学院大学)

Sao Mai Nguyen

AI总结:

本文提出一种基于经验进展的内在动机统一框架,使学习智能体在多任务序列学习中主动选择探索或模仿、任务分解及最佳导师请求示范,从而实现更快、更鲁棒的自动课程学习。

摘要:

这项属于发展认知机器人学领域的工作旨在设计一个新的领域,在强化学习和模仿学习之间架起桥梁,并提出一种学习智能体内在动机模型,使其能够在导师指导下学习多个任务,包括序列任务。主要贡献在于提出了一种基于经验进展的内在动机统一表述,使学习智能体能够通过主动选择简单或序列任务的学习策略来自动选择其学习课程:学习哪个任务、在自主探索或模仿学习之间选择、在低层动作或任务分解之间选择、在多个导师之间选择。其创新之处在于设计的学习者不仅被动地从导师提供的数据中受益,而且能够主动选择何时请求指导、请求什么内容以及向谁请求。因此,该学习者对指导质量的鲁棒性更强,并且能够以更少的示范更快地学习。我们开发了社会引导内在动机框架,利用机器学习算法学习多个任务,通过被动方式或主动方式利用人类示范的泛化特性,主动方式即针对简单任务和组合子任务向最佳导师请求示范。后者依赖于为构建过程提出的子任务组合表示,该表示应通过用于分析人类运动和日常生活活动的观察过程表示加以改进。着眼于与导师进行类语言交流的前景,我们研究了利用内在动机使连续感觉运动空间和任务的符号表示得以涌现的问题。我们在强化学习框架内提出了一种与导师交互的奖励函数,用于多任务学习中的自动课程学习。

英文摘要:

This work in the field of developmental cognitive robotics aims to devise a new domain bridging between reinforcement learning and imitation learning, with a model of the intrinsic motivation for learning agents to learn with guidance from tutors multiple tasks, including sequential tasks. The main contribution has been to propose a common formulation of intrinsic motivation based on empirical progress for a learning agent to choose automatically its learning curriculum by actively choosing its learning strategy for simple or sequential tasks: which task to learn, between autonomous exploration or imitation learning, between low-level actions or task decomposition, between several tutors. The originality is to design a learner that benefits not only passively from data provided by tutors, but to actively choose when to request tutoring and what and whom to ask. The learner is thus more robust to the quality of the tutoring and learns faster with fewer demonstrations. We developed the framework of socially guided intrinsic motivation with machine learning algorithms to learn multiple tasks by taking advantage of the generalisability properties of human demonstrations in a passive manner or in an active manner through requests of demonstrations from the best tutor for simple and composing subtasks. The latter relies on a representation of subtask composition proposed for a construction process, which should be refined by representations used for observational processes of analysing human movements and activities of daily living. With the outlook of a language-like communication with the tutor, we investigated the emergence of a symbolic representation of the continuous sensorimotor space and of tasks using intrinsic motivation. We proposed within the reinforcement learning framework, a reward function for interacting with tutors for automatic curriculum learning in multi-task learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑