arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2202.03620math.OCcs.SYeess.SY
收藏

具有动态学习的最优控制:漂移未知系统

Optimal Control with Learning on the Fly: System with Unknown Drift

Daniel Gurevich, Debdipta Goswami, Charles L. Fefferman, Clarence W. Rowley

摘要:

本文导出了具有常值漂移和附加控制输入的简单随机动力系统的最优控制策略。受一个物理系统的动力学发生意外变化的例子的启发,我们将漂移参数设为未知,因此必须在控制系统时学习。系统的状态通过具有高斯噪声的线性观测模型来观测。与以往的大多数工作,它侧重于控制器的渐近性能在无限的时间范围内,我们最小化的二次费用函数在有限的时间范围内。我们的控制策略的性能进行量化,通过比较其成本与最优控制器,具有充分的知识的参数所产生的成本。这种方法产生了几种“遗憾”的概念。“我们推导出一组控制策略,可证明最小化最坏情况下的遗憾;这些来自贝叶斯策略,假设漂移参数的特定固定先验。这项工作表明,检查贝叶斯策略可能会导致最佳或接近最优控制策略的一个更大的类的现实动态模型与未知参数。

英文摘要:

This paper derives an optimal control strategy for a simple stochastic dynamical system with constant drift and an additive control input. Motivated by the example of a physical system with an unexpected change in its dynamics, we take the drift parameter to be unknown, so that it must be learned while controlling the system. The state of the system is observed through a linear observation model with Gaussian noise. In contrast to most previous work, which focuses on a controller's asymptotic performance over an infinite time horizon, we minimize a quadratic cost function over a finite time horizon. The performance of our control strategy is quantified by comparing its cost with the cost incurred by an optimal controller that has full knowledge of the parameters. This approach gives rise to several notions of "regret." We derive a set of control strategies that provably minimize the worst-case regret; these arise from Bayesian strategies that assume a specific fixed prior on the drift parameter. This work suggests that examining Bayesian strategies may lead to optimal or near-optimal control strategies for a much larger class of realistic dynamical models with unknown parameters.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑