CDOpt:面向一类黎曼优化的Python包
CDOpt: A Python Package for a Class of Riemannian Optimization
AI总结:
CDOpt是一个用户友好的Python包,通过约束溶解方法将黎曼优化问题转化为无约束优化问题,支持自定义流形并扩展了PyTorch和Flax的神经层,使无约束优化求解器能直接用于流形约束神经网络训练,实验证明其高效且鲁棒。
摘要:
在过去几十年中,由于在各个领域的广泛应用,由约束 $c(x) = 0$ 定义的嵌入子流形上的优化问题引起了极大关注。许多相关的优化包都是基于黎曼优化方法开发的,这些方法依赖于黎曼流形的一些基本几何材料,包括收缩、向量传输等。这些几何材料在一般情况下很难确定。现有的包只能容纳少数几何材料容易获取的著名流形。对于这些包中未包含的其他流形,用户必须自行开发几何材料。此外,将各种最先进的无约束优化求解器的高级特性引入黎曼优化方法并不总是可行的。我们介绍了 CDOpt(可在 https://cdopt.github.io/ 获取),这是一个用于一类黎曼优化的用户友好型 Python 包。基于约束溶解方法,CDOpt 将黎曼优化问题转化为其等价的无约束对应问题。因此,通过 CDOpt 解决黎曼优化问题可以直接受益于各种现有的求解器以及几十年来在无约束优化方面积累的丰富经验。此外,CDOpt 中与任何相关流形有关的所有计算都在其约束表达式上进行,因此用户可以轻松地在 CDOpt 中定义新流形,而无需具备微分几何背景。进一步地,CDOpt 扩展了 PyTorch 和 Flax 的神经层,从而允许用户直接通过无约束优化求解器训练受流形约束的神经网络。大量数值实验表明,CDOpt 在解决各类黎曼优化问题方面具有高效性和鲁棒性。
英文摘要:
Optimization over the embedded submanifold defined by constraints $c(x) = 0$ has attracted much interest over the past few decades due to its wide applications in various areas. Plenty of related optimization packages have been developed based on Riemannian optimization approaches, which rely on some basic geometrical materials of Riemannian manifolds, including retractions, vector transports, etc. These geometrical materials can be challenging to determine in general. Existing packages only accommodate a few well-known manifolds whose geometrical materials are easily accessible. For other manifolds which are not contained in these packages, the users have to develop the geometric materials by themselves. In addition, it is not always tractable to adopt advanced features from various state-of-the-art unconstrained optimization solvers to Riemannian optimization approaches. We introduce CDOpt (available at https://cdopt.github.io/), a user-friendly Python package for a class Riemannian optimization. Based on constraint dissolving approaches, Riemannian optimization problems are transformed into their equivalent unconstrained counterparts in CDOpt. Therefore, solving Riemannian optimization problems through CDOpt directly benefits from various existing solvers and the rich expertise gained over decades for unconstrained optimization. Moreover, all the computations in CDOpt related to any manifold in question are conducted on its constraints expression, hence users can easily define new manifolds in CDOpt without any background on differential geometry. Furthermore, CDOpt extends the neural layers from PyTorch and Flax, thus allows users to train manifold constrained neural networks directly by the solvers for unconstrained optimization. Extensive numerical experiments demonstrate that CDOpt is highly efficient and robust in solving various classes of Riemannian optimization problems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。