从流形识别到交点处的牛顿加速：稀疏Stiefel优化
From Manifold Identification to Newton Acceleration on Intersections: Sparse Stiefel Optimization
浏览论文内容
中文总结 AI 辅助
研究Stiefel流形上稀疏复合优化的牛顿加速问题，针对活动流形与Stiefel流形相交的几何困难，提出非对角扰动Stiefel族及MIX方法，证明相关保证，数值实验表明该方法在保持解质量时显著提升效率。
中文摘要 AI 辅助
我们研究了Stiefel流形上稀疏复合优化的牛顿加速。主要困难在于几何方面：由非光滑正则化器识别的活动流形可能无法与Stiefel流形横向相交，这阻碍了在识别出的流形上进行黎曼牛顿步。在横向情况下，我们证明了ManPG切近端映射的局部识别。对于非横向情况，我们引入了一个非对角扰动的Stiefel族，它通常能恢复识别几何，同时为原始问题提供\(O(\|\Delta\|_F)\)-KKT保证。我们还推导了清晰相交的可验证支持水平条件，涵盖非横向稀疏模式并产生牛顿校正所使用的光滑移动局部模型。基于这些结果，我们提出了MIX，一种在移动识别交点上的安全ManPG/牛顿 - CG方法。在一般清晰相交设置中，我们证明了MIX的全局下降和KKT残差保证。在横向或一般扰动情况下，我们进一步表明，在序列收敛和二阶充分条件下，MIX在有限时间内识别活动流形，然后局部超线性收敛。关于压缩模式和稀疏PCA的数值实验表明，MIX在保持解质量的同时显著提高了效率。
英文摘要
We study a Newton acceleration for sparse composite optimization on the Stiefel manifold. The main difficulty is geometric: the active manifold identified by the nonsmooth regularizer may fail to intersect the Stiefel manifold transversely, which obstructs a Riemannian Newton step on the identified manifold. In the transverse case, we prove local identification of the ManPG tangent proximal mapping. For nontransverse cases, we introduce an off-diagonally perturbed Stiefel family that generically restores the identification geometry while yielding an \(O(\|Δ\|_F)\)-KKT guarantee for the original problem. We also derive verifiable support-level conditions for clean intersection, which cover nontransverse sparse patterns and yield the smooth moving local models used by the Newton correction. Based on these results, we propose MIX, a safeguarded ManPG/Newton-CG method on moving identified intersections. In the general clean-intersection setting, we prove global descent and KKT-residual guarantees for MIX. In the transverse or generically perturbed cases, if the sequence has an accumulation point satisfying certain regularity assumptions and the second-order sufficient condition (SOSC), then the full sequence converges to that point, with finite active-manifold identification and a local Q-superlinear rate. Numerical experiments on compressed modes and sparse PCA show that MIX substantially improves efficiency while preserving solution quality. Beyond the Stiefel manifold, we also outline how the safeguarded global-convergence mechanism of MIX extends to general smooth equality-constrained manifolds.
