arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.07437cs.LGcs.SYeess.SYmath.OCstat.ML
收藏

基于成本驱动的线性二次高斯控制表示学习:第二部分

Cost-Driven Representation Learning for Linear Quadratic Gaussian Control: Part II

Massachusetts Institute of Technology(麻省理工学院);University of Maryland, College Park(大学公园大学);Technical University Munich(慕尼黑技术大学)

Yi Tian, Kaiqing Zhang, Russ Tedrake, Suvrit Sra

AI总结:

本文提出了一种基于成本驱动的线性二次高斯控制表示学习方法,通过隐式学习潜在动态来改进控制器性能。

摘要:

我们研究了从部分和可能高维观测中学习状态表示以用于控制的问题。我们通过成本驱动的状态表示学习方法来解决这个问题,其中通过预测累积成本来在潜在状态空间中学习动态模型。特别是,我们建立了在有限样本下找到近最优表示函数和近最优控制器的保证,使用所学的潜在模型进行无限时间不变线性二次高斯(LQG)控制。我们研究了两种成本驱动的表示学习方法,区别在于潜在状态的转移函数是显式还是隐式学习。第一种方法在本部分的第一部分中已被研究,用于有限时间变化的LQG控制。第二种方法与最近在经验强化学习中的突破MuZero相似,因为它通过预测累积成本隐式地学习潜在动态。本部分的第二部分的关键技术贡献是证明了一个新的随机过程的持续激励性,该过程来源于我们方法中二次回归的分析,可能具有独立的兴趣。

英文摘要:

We study the problem of state representation learning for control from partial and potentially high-dimensional observations. We approach this problem via cost-driven state representation learning, in which we learn a dynamical model in a latent state space by predicting cumulative costs. In particular, we establish finite-sample guarantees on finding a near-optimal representation function and a near-optimal controller using the learned latent model for infinite-horizon time-invariant Linear Quadratic Gaussian (LQG) control. We study two approaches to cost-driven representation learning, which differ in whether the transition function of the latent state is learned explicitly or implicitly. The first approach has also been investigated in Part I of this work, for finite-horizon time-varying LQG control. The second approach closely resembles MuZero, a recent breakthrough in empirical reinforcement learning, in that it learns latent dynamics implicitly by predicting cumulative costs. A key technical contribution of this Part II is to prove persistency of excitation for a new stochastic process that arises from the analysis of quadratic regression in our approach, and may be of independent interest.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑