非参数无穷时间Kullback-Leibler随机控制
Nonparametric Infinite Horizon Kullback-Leibler Stochastic Control
AI总结:
本文提出基于高斯过程和Nyström近似的非参数方法,解决Kullback-Leibler控制问题,通过准确的函数逼近和高效的在线操作提升性能,理论和算法均验证了方法的有效性。
摘要:
我们提出两种非参数方法用于Kullback-Leibler(KL)控制,或线性可解马尔可夫决策问题(LMDP),基于高斯过程(GP)和Nyström近似。与最近发展的参数方法相比,所提出的数据驱动框架具有准确的函数逼近和高效的在线操作。理论上,我们推导了基于动态规划的KL控制与控制理论中依赖信息论对偶性的早期工作之间的数学联系,用于无限时间 horizon 情况。算法上,我们给出了非参数形式的最优控制策略,并提出了具有预算计算成本的在线更新方案。数值结果展示了所提框架的有效性和实用性。
英文摘要:
We present two nonparametric approaches to Kullback-Leibler (KL) control, or linearly-solvable Markov decision problem (LMDP) based on Gaussian processes (GP) and Nyström approximation. Compared to recently developed parametric methods, the proposed data-driven frameworks feature accurate function approximation and efficient on-line operations. Theoretically, we derive the mathematical connection of KL control based on dynamic programming with earlier work in control theory which relies on information theoretic dualities for the infinite time horizon case. Algorithmically, we give explicit optimal control policies in nonparametric forms, and propose on-line update schemes with budgeted computational costs. Numerical results demonstrate the effectiveness and usefulness of the proposed frameworks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。