arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1107.0048cs.AI
收藏

可分类环境中多传感器和执行器Agent的强化学习

Reinforcement Learning for Agents with Many Sensors and Actuators Acting in Categorizable Environments

University of Amsterdam(阿姆斯特丹大学);Spanish Council of Scientific Research (CSIC)(西班牙科学研究理事会)

E. Celaya, J. M. Porta

摘要:

在本文中,我们面临的问题,应用强化学习代理感知环境,通过许多传感器,可以执行并行操作,使用许多执行器的情况下,在复杂的自主机器人。我们认为,强化学习只能成功地应用于这种情况下,如果强有力的假设进行学习的环境的特点,使相关的传感器读数和电机命令可以很容易地识别。这种假设的引入导致了强偏差的学习系统,最终可能会失去传统学习算法的通用性。在这条线上,我们观察到,在现实情况下,机器人收到的奖励只取决于所有执行的动作的减少的子集,只有减少的传感器输入的子集(可能在每个情况下和每个动作不同)与预测奖励相关。我们正式这个属性在所谓的“可分类性假设”,我们提出了一种算法,利用环境的可分类性,允许减少学习时间相对于现有的学习算法。该算法的应用程序的一对夫妇的模拟现实的机器人问题(基于地标的导航和六条腿的足式机器人步态生成)的结果报告,以验证我们的方法,并将其与现有的平面和广义为基础的学习方法。

英文摘要:

In this paper, we confront the problem of applying reinforcement learning to agents that perceive the environment through many sensors and that can perform parallel actions using many actuators as is the case in complex autonomous robots. We argue that reinforcement learning can only be successfully applied to this case if strong assumptions are made on the characteristics of the environment in which the learning is performed, so that the relevant sensor readings and motor commands can be readily identified. The introduction of such assumptions leads to strongly-biased learning systems that can eventually lose the generality of traditional reinforcement-learning algorithms. In this line, we observe that, in realistic situations, the reward received by the robot depends only on a reduced subset of all the executed actions and that only a reduced subset of the sensor inputs (possibly different in each situation and for each action) are relevant to predict the reward. We formalize this property in the so called 'categorizability assumption' and we present an algorithm that takes advantage of the categorizability of the environment, allowing a decrease in the learning time with respect to existing reinforcement-learning algorithms. Results of the application of the algorithm to a couple of simulated realistic-robotic problems (landmark-based navigation and the six-legged robot gait generation) are reported to validate our approach and to compare it to existing flat and generalization-based reinforcement-learning approaches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑