基于强化学习的未知线性系统控制
Learning-based Control of Unknown Linear Systems with Thompson Sampling
AI总结：
本文提出一种基于Thompson采样的学习算法，用于未知参数的线性二次控制问题。算法通过动态周期控制策略，保证预期遗憾随时间T增长为O(√T)，并具备对模型参数时变漂移的鲁棒性。
摘要：
我们提出了一种基于Thompson采样的学习算法，用于具有未知系统参数的线性二次（LQ）控制问题。该算法称为带有动态周期的Thompson采样（TSDE），其中两个停止准则决定Thompson采样中动态周期的长度。第一个停止准则控制周期长度的增长率。第二个停止准则在样本协方差矩阵的行列式小于前一次值的一半时触发。我们证明在某些先验分布条件下，TSDE算法在时间T内累积的预期（贝叶斯）遗憾被界于O(√T)。这里O(.)隐藏了常数和对数因子。这是首次在LQ控制中获得O(√T)预期遗憾界。通过引入重新初始化计划，我们还证明该算法对模型参数的时间变化漂移具有鲁棒性。数值模拟用于展示TSDE的性能。
英文摘要：
We propose a Thompson sampling-based learning algorithm for the Linear Quadratic (LQ) control problem with unknown system parameters. The algorithm is called Thompson sampling with dynamic episodes (TSDE) where two stopping criteria determine the lengths of the dynamic episodes in Thompson sampling. The first stopping criterion controls the growth rate of episode length. The second stopping criterion is triggered when the determinant of the sample covariance matrix is less than half of the previous value. We show under some conditions on the prior distribution that the expected (Bayesian) regret of TSDE accumulated up to time T is bounded by O(\sqrt{T}). Here O(.) hides constants and logarithmic factors. This is the first O(\sqrt{T} ) bound on expected regret of learning in LQ control. By introducing a reinitialization schedule, we also show that the algorithm is robust to time-varying drift in model parameters. Numerical simulations are provided to illustrate the performance of TSDE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
