arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1610.02847cs.AI
收藏

通过风险意识技能提高情境意识

Situational Awareness by Risk-Conscious Skills

The Technion - Israel Institute of Technology(以色列理工学院);UC Berkeley(加州大学伯克利分校)

Daniel J. Mankowitz, Aviv Tamar, Shie Mannor

摘要:

分层强化学习先前已被证明可以加快RL规划算法的收敛速度,并减轻基于特征的模型错误指定(Mankowitz et. 2016a,b,Bacon 2015)。为了做到这一点,它利用了层次抽象,也被称为技能-一种时间上扩展的动作(Sutton et. 1999年),以计划在一个更高的水平,抽象远离较低层次的细节。我们将风险敏感性,也被称为态势感知(SA),到分层RL的第一次定义和学习风险意识的技能,在概率目标半马尔可夫决策过程(PG-SMDP)。这是使用我们的新的情境意识的风险意识技能(SARiCoS)算法来实现的,该算法具有理论上的收敛保证。我们表明,在RoboCup足球领域的学习风险意识的技能表现出复杂的人类行为,如“浪费时间”在足球比赛中。此外,学习到的风险意识技能能够减轻基于奖励的模型错误。

英文摘要:

Hierarchical Reinforcement Learning has been previously shown to speed up the convergence rate of RL planning algorithms as well as mitigate feature-based model misspecification (Mankowitz et. al. 2016a,b, Bacon 2015). To do so, it utilizes hierarchical abstractions, also known as skills -- a type of temporally extended action (Sutton et. al. 1999) to plan at a higher level, abstracting away from the lower-level details. We incorporate risk sensitivity, also referred to as Situational Awareness (SA), into hierarchical RL for the first time by defining and learning risk aware skills in a Probabilistic Goal Semi-Markov Decision Process (PG-SMDP). This is achieved using our novel Situational Awareness by Risk-Conscious Skills (SARiCoS) algorithm which comes with a theoretical convergence guarantee. We show in a RoboCup soccer domain that the learned risk aware skills exhibit complex human behaviors such as `time-wasting' in a soccer game. In addition, the learned risk aware skills are able to mitigate reward-based model misspecification.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑