arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2209.12644cs.ROcs.LGcs.SYeess.SYmath.OC
收藏

FORESEE:面向在线策略优化的基于扩张-压缩无迹变换的预测方法

FORESEE: Prediction with Expansion-Compression Unscented Transform for Online Policy Optimization

University of Michigan(密歇根大学)

Hardik Parwana, Dimitra Panagou

AI总结:

该研究提出扩张-压缩无迹变换的状态预测方法,结合约束梯度下降变体以滚动时域方式优化策略,在性能比肩Monte Carlo的同时大幅降低计算成本,已在四旋翼稳定等任务中验证有效性。

摘要:

众所周知,通过通用的不确定非线性动力学模型传播状态分布是难以精确求解的,通常需要借助数值或解析近似方法。我们提出了一种名为扩张-压缩无迹变换(Expansion-Compression Unscented Transform)的状态预测方法,并将其用于求解一类在线策略优化问题。\n我们提出的算法通过状态依赖分布传播有限数量的sigma点,该分布要求每个时间步增加sigma点的数量以表征得到的分布,这一过程被我们称为扩张操作。为保证算法的可扩展性,我们在扩张操作的基础上加入了基于矩匹配的压缩操作,从而在多时间步的预测过程中保持sigma点数量恒定。实验表明,该方法的性能与Monte Carlo方法相当,但计算成本低得多。\n在状态和控制输入约束下,该状态预测方法随后与我们提出的一种约束梯度下降变体结合,以滚动时域的方式在线更新策略参数。该框架被实现为可微计算图用于策略训练。我们在safe-control-gym的基准对比中,针对四旋翼稳定任务以及领导者-跟随者问题中基于控制屏障函数(Control Barrier Function)的控制器参数优化任务,验证了所提框架的有效性。

英文摘要:

Propagating state distributions through a generic, uncertain nonlinear dynamical model is known to be intractable and usually begets numerical or analytical approximations. We introduce a method for state prediction, called the Expansion-Compression Unscented Transform, and use it to solve a class of online policy optimization problems. Our proposed algorithm propagates a finite number of sigma points through a state-dependent distribution, which dictates an increase in the number of sigma points at each time step to represent the resulting distribution; this is what we call the expansion operation. To keep the algorithm scalable, we augment the expansion operation with a compression operation based on moment matching, thereby keeping the number of sigma points constant across predictions over multiple time steps. Its performance is empirically shown to be comparable to Monte Carlo but at a much lower computational cost. Under state and control input constraints, the state prediction is subsequently used in tandem with a proposed variant of constrained gradient-descent for online update of policy parameters in a receding horizon fashion. The framework is implemented as a differentiable computational graph for policy training. We showcase our framework for a quadrotor stabilization task as part of a benchmark comparison in safe-control-gym and for optimizing the parameters of a Control Barrier Function based controller in a leader-follower problem.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑