arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1309.6989cs.AI
收藏

情景策略梯度下一步预测信息与外部奖励的线性组合:批判性分析

Linear combination of one-step predictive information with an external reward in an episodic policy gradient setting: a critical analysis

Max Planck Institute for Mathematics in the Sciences(马克斯·普朗克数学科学研究所);Santa Fe Institute(圣塔菲研究所)

Keyan Zahedi, Georg Martius, Nihat Ay

摘要:

嵌入式人工智能领域的主要挑战之一是复杂行为的开放式自主学习。我们的方法是使用独立于任务的、信息驱动的内在动机来支持任务依赖性学习。这里提出的工作是一个初步的步骤,我们调查的预测信息(传感器流的过去和未来的互信息)作为一个内在的驱动器,理想地支持任何类型的任务采集。以前的实验表明,预测信息(PI)是一个很好的候选人,以支持自主的,开放式的学习复杂的行为,因为最大化的PI对应于探索形态和环境依赖的行为模式。这个想法是,这些问题可以被利用,以解决任何给定的任务。三个不同的实验和他们的结果导致的结论是,一步PI与外部奖励函数的线性组合,一般不建议在一个情节性的策略梯度设置。只有在硬任务中,才能以渐进性能损失为代价来实现极大的加速。

英文摘要:

One of the main challenges in the field of embodied artificial intelligence is the open-ended autonomous learning of complex behaviours. Our approach is to use task-independent, information-driven intrinsic motivation(s) to support task-dependent learning. The work presented here is a preliminary step in which we investigate the predictive information (the mutual information of the past and future of the sensor stream) as an intrinsic drive, ideally supporting any kind of task acquisition. Previous experiments have shown that the predictive information (PI) is a good candidate to support autonomous, open-ended learning of complex behaviours, because a maximisation of the PI corresponds to an exploration of morphology- and environment-dependent behavioural regularities. The idea is that these regularities can then be exploited in order to solve any given task. Three different experiments are presented and their results lead to the conclusion that the linear combination of the one-step PI with an external reward function is not generally recommended in an episodic policy gradient setting. Only for hard tasks a great speed-up can be achieved at the cost of an asymptotic performance lost.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑