arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2102.02594math.OC
收藏

风险敏感的无限地平线问题的路径积分控制

Risk Sensitive Path Integral Control for Infinite Horizon Problem Formulations

Tom Lefebvre, Guillaume Crevecoeur

摘要:

研究了随机最优控制的路径积分控制方法,该方法涵盖了一类具有控制仿射非线性动力学的有限时域公式。该类的特征在于HJB方程是线性的,因此价值函数可以表示为在具有不受控系统动态特性的轨迹上评估的指数加权成本的条件期望,因此得名。随后,它表明,在相同的假设下,路径积分控制推广到有限时间风险敏感的随机最优控制问题。在这里,我们研究是否HJB的无限地平线配方可以线性以及。我们对无限视界公式的兴趣是由相关值函数的平稳性及其内在的动态稳定性所激发的。从技术上讲,平稳值函数可以简化相关线性HJB的求解。其次,我们认为这可能为非线性强化学习应用提供了一个有趣的起点。我们正式表明,折扣和平均成本配方分别是棘手的和易处理的。

英文摘要:

Path Integral Control methods were developed for stochastic optimal control covering a wide class of finite horizon formulations with control affine nonlinear dynamics. Characteristic for this class is that the HJB equation is linear and consequently the value function can be expressed as a conditional expectation of the exponentially weighted cost-to-go evaluated over trajectories with uncontrolled system dynamics, hence the name. Subsequently it was shown that under the same assumptions Path Integral Control generalises to finite horizon risk sensitive stochastic optimal control problems. Here we study whether the HJB of infinite horizon formulations can be made linear as well. Our interest in infinite horizon formulations is motivated by the stationarity of the associated value function and their inherent dynamic stability seeking nature. Technically a stationary value function may ease the solution of the associated linear HJB. Second we argue this may offer an interesting starting point for off-linear Reinforcement Learning applications. We show formally that the discounted and average cost formulations are respectively intractable and tractable.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑