发表机构
Jilin University; Kyoto University; Johns Hopkins University（吉林大学; 京都大学; 约翰斯·霍普金斯大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对非凸约束优化层可微求解的挑战，提出无矩阵求解器lapanda，通过增广拉格朗日子问题和近端拟牛顿内层求解实现高效前向与后向，大幅降低计算和内存开销，并保持性能。
AI 中文摘要
可微优化将数学优化的结构保证引入网络管道，使其能够进行端到端训练。然而，对于非凸约束问题，其应用仍然具有挑战性，因为现有的可微求解器往往因依赖专门的问题结构而受到有限的建模表达能力限制，同时在前向和后向传播中也会产生大量的计算时间和内存开销。为了解决这些挑战，我们提出了lapanda，一种用于具有一般约束的非凸优化的无矩阵可微求解器。它将问题重新表述为一系列增广拉格朗日子问题，每个子问题通过一个带有自适应线搜索的近端平均拟牛顿算法的一阶内层求解器来处理，从而实现高效的前向优化。我们建立了解映射的局部适定性和外迭代的收敛性，并进一步推导了原始问题与后向传播中最终子问题之间的敏感性对齐，证明了子问题敏感性（可以以无矩阵方式高效计算）提供了对精确优化器敏感性的原则性近似。我们在非凸约束Rosenbrock基准、具有几个代表性约束最优控制问题的模仿学习以及嵌入式机器人避障任务上评估了lapanda。与最先进的可微求解器相比，lapanda在保持可靠的约束满足和学习性能的同时，大幅减少了计算时间和内存占用。
英文摘要
Differentiable optimization brings the structural guarantees of mathematical optimization to network pipelines, allowing them to be trained end-to-end. However, its application remains challenging for nonconvex constrained problems, as existing differentiable solvers often suffer from limited modeling expressiveness due to their reliance on specialized problem structures, while also incurring substantial computation time and memory overhead in both the forward and backward passes. To address these challenges, we propose lapanda, a matrix-free differentiable solver for nonconvex optimization with general constraints. It reformulates the problem to a sequence of augmented Lagrangian subproblems, each handled by a first-order inner solver through a proximal averaged quasi-Newton algorithm with adaptive linesearch, thus enabling efficient forward optimization. We establish local well-posedness of the solution map and convergence of the outer iterations, and further derive a sensitivity alignment between the original problem and the final subproblem in the backward pass, demonstrating that the subproblem sensitivity, which can be computed efficiently in a matrix-free manner, provides a principled approximation to the exact optimizer sensitivity. We evaluate lapanda on nonconvex constrained Rosenbrock benchmarks, imitation learning with several representative constrained optimal control problems, and embedded robotic obstacle-avoidance tasks. Compared with state-of-the-art differentiable solvers, lapanda delivers substantial reductions in computation time and memory footprint while maintaining reliable constraint satisfaction and learning performance.
