线性控制策略在卡尔曼约束下的拟合
Fitting a Linear Control Policy to Demonstrations with a Kalman Constraint
摘要：
我们认为学习线性动力系统的线性控制策略的问题，从专家调节系统的演示。解决这个问题的标准方法是策略拟合，它通过最小化演示和策略输出之间的损失函数加上编码先验知识的正则化函数来拟合线性策略。尽管它的简单性，这种方法无法学习低成本甚至有限的政策时，有几个示范。我们建议增加一个额外的约束政策拟合，政策是解决一些LQR问题，即，在随机控制意义下，对于二次成本的某些选择是最优的。我们将此约束称为卡尔曼约束。具有卡尔曼约束的策略拟合需要求解具有凸成本和双线性约束的最优化问题。我们提出了一种启发式方法，基于交替方向法的乘数（ADMM），近似解决这个问题。数值实验表明，添加卡尔曼约束允许我们学习好，即，低成本，即使在数据很少的情况下也有策略。
英文摘要：
We consider the problem of learning a linear control policy for a linear dynamical system, from demonstrations of an expert regulating the system. The standard approach to this problem is policy fitting, which fits a linear policy by minimizing a loss function between the demonstrations and the policy's outputs plus a regularization function that encodes prior knowledge. Despite its simplicity, this method fails to learn policies with low or even finite cost when there are few demonstrations. We propose to add an additional constraint to policy fitting, that the policy is the solution to some LQR problem, i.e., optimal in the stochastic control sense for some choice of quadratic cost. We refer to this constraint as a Kalman constraint. Policy fitting with a Kalman constraint requires solving an optimization problem with convex cost and bilinear constraints. We propose a heuristic method, based on the alternating direction method of multipliers (ADMM), to approximately solve this problem. Numerical experiments demonstrate that adding the Kalman constraint allows us to learn good, i.e., low cost, policies even when very few data are available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
