参数优化问题的最小范数解的微分

中文摘要 AI 辅助

通过参数优化问题进行微分是双层规划和元学习的核心，通常采用近似隐式微分来实现。隐函数定理要求对最优性条件的偏雅可比矩阵求逆，当存在多个解时该方法失效。然而在这类情况下，可将可逆性放宽为严格更弱的一致值域条件，在此条件下，通过极限Tikhonov正则化论证和保守集值场理论可证明，最小范数解映射存在广义导数。在对广义海森矩阵的特征值施加额外控制的情况下，可证明伪逆公式成立。该结论针对一类光滑凸目标函数建立，并扩展到非光滑复合问题。对于最小二乘、Huber回归和LASSO，验证了这些假设。在数据投毒和数据超清洁问题上，通过实验检验了非光滑隐式微分到不适定设置的扩展效果。

英文摘要

Differentiating through parametric optimization problems is central to bilevel programming and meta-learning, often accomplished using approximate implicit differentiation. The implicit function theorem requires inverting a partial Jacobian of the optimality condition, which fails when there are many solutions. Nonetheless, in such cases it is possible to relax invertibility to a strictly weaker uniform range condition, under which it is shown that the minimal-norm solution mapping admits generalized derivatives by using a limiting Tikhonov regularization argument and conservative set-valued field theory. With additional control on the eigenvalues of the generalized Hessians, a pseudoinverse formula is justified. This is established for a class of smooth convex objectives and extended to nonsmooth composite problems. These assumptions are verified for Least-Squares, Huber regression and LASSO. The resulting extension of nonsmooth implicit differentiation to ill-posed settings is examined experimentally on data poisoning and data hypercleaning problems.

发表机构

• CentraleSupélec（中央苏佩莱克高等学院）
• GREYC CNRS（法国国家科学研究中心格雷亚克实验室）
• ENSICAEN（恩西辛高等师范学院）

机构由 AI 辅助整理，请以论文原文为准。