基于微分动态规划的可微最优控制
Differentiable Optimal Control via Differential Dynamic Programming
摘要:
机器人设计优化、模仿学习和系统辨识有一个共同的问题,即在优化机器人运动的同时需要对机器人或任务参数进行优化。为了解决这些问题,我们可以使用可微的最优控制,其中需要机器人的运动相对于参数的梯度。我们提出了一种方法来有效地计算这些梯度分析通过微分动态规划(DDP)算法,使用灵敏度分析(SA)。我们表明,我们必须包括二阶动力学项计算梯度时。然而,我们在计算运动时不需要包括它们。我们验证我们的方法上的单摆和双摆系统。此外,我们比较使用的衍生物的迭代线性二次调节器(iLQR),忽略这些二阶项无处不在,在一个共同设计任务的Kinova臂,在那里我们优化的链接长度的机器人的目标达到任务。我们表明,优化使用iLQR梯度发散忽略二阶动力学影响计算的衍生物。相反,使用DDP梯度优化收敛到一系列初始设计的相同最佳值,使我们的配方能够扩展到复杂的系统。
英文摘要:
Robot design optimization, imitation learning and system identification share a common problem which requires optimization over robot or task parameters at the same time as optimizing the robot motion. To solve these problems, we can use differentiable optimal control for which the gradients of the robot's motion with respect to the parameters are required. We propose a method to efficiently compute these gradients analytically via the differential dynamic programming (DDP) algorithm using sensitivity analysis (SA). We show that we must include second-order dynamics terms when computing the gradients. However, we do not need to include them when computing the motion. We validate our approach on the pendulum and double pendulum systems. Furthermore, we compare against using the derivatives of the iterative linear quadratic regulator (iLQR), which ignores these second-order terms everywhere, on a co-design task for the Kinova arm, where we optimize the link lengths of the robot for a target reaching task. We show that optimizing using iLQR gradients diverges as ignoring the second-order dynamics affects the computation of the derivatives. Instead, optimizing using DDP gradients converges to the same optimum for a range of initial designs allowing our formulation to scale to complex systems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。