arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2011.13847cs.ROcs.LG
收藏

自主学习多种情境相关任务

Autonomous learning of multiple, context-dependent tasks

Istituto di Scienze e Tecnologie della Cognizione (ISTC), Consiglio Nazionale delle Ricerce (CNR)(国家研究委员会认知科学与技术研究所);Institute of Informatics, Federal University of Rio Grande do Sul (UFRGS)(南里奥格兰德联邦大学信息学研究所)

Vieri Giuliano Santucci, Davide Montella, Bruno Castro da Silva, Gianluca Baldassarre

摘要:

当面对使用强化学习系统自主学习多个任务的问题时,研究人员通常专注于每个任务只有一个参数化策略就足以解决它们的解决方案。然而,在呈现不同上下文的复杂环境中,相同的任务可能需要一组不同的技能来解决。这些情况提出了两个挑战:(a)识别需要不同策略的不同上下文;(b)快速学习策略以在新发现的上下文中完成相同的任务。如果在开放式学习框架中面对这两个挑战,智能体必须自主地发现它在给定环境中可能实现的目标,并学习完成这些目标的运动技能,那么这两个挑战就更加困难。我们提出了一种新的开放式学习机器人体系结构,C-GRAIL,以综合的方式解决了这两个挑战。特别是,该架构能够检测新的相关的比赛,并忽略不相关的,对一个给定的目标的预期性能的下降的基础上。此外,该架构可以通过利用从已经获得的策略导入知识的迁移学习来快速学习新上下文的策略。该架构进行了测试,在一个模拟的机器人环境中,涉及一个机器人,自主学习,以达到相关的目标对象中存在的多个障碍物产生几个不同的障碍。所提出的架构优于其他模型不使用所提出的自主上下文发现和迁移学习机制。

英文摘要:

When facing the problem of autonomously learning multiple tasks with reinforcement learning systems, researchers typically focus on solutions where just one parametrised policy per task is sufficient to solve them. However, in complex environments presenting different contexts, the same task might need a set of different skills to be solved. These situations pose two challenges: (a) to recognise the different contexts that need different policies; (b) quickly learn the policies to accomplish the same tasks in the new discovered contexts. These two challenges are even harder if faced within an open-ended learning framework where an agent has to autonomously discover the goals that it might accomplish in a given environment, and also to learn the motor skills to accomplish them. We propose a novel open-ended learning robot architecture, C-GRAIL, that solves the two challenges in an integrated fashion. In particular, the architecture is able to detect new relevant contests, and ignore irrelevant ones, on the basis of the decrease of the expected performance for a given goal. Moreover, the architecture can quickly learn the policies for the new contexts by exploiting transfer learning importing knowledge from already acquired policies. The architecture is tested in a simulated robotic environment involving a robot that autonomously learns to reach relevant target objects in the presence of multiple obstacles generating several different obstacles. The proposed architecture outperforms other models not using the proposed autonomous context-discovery and transfer-learning mechanisms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑