IFNSO：无迭代牛顿-施卢茨正交化
IFNSO: Iteration-Free Newton-Schulz Orthogonalization
AI总结：
本文提出IFNSO方法，通过无迭代框架提升正交化效率，优化系数确保计算效率与收敛稳定性，实验验证其优越性。
摘要：
牛顿-施卢茨迭代已成为Muon等优化器和斯蒂费尔流形优化中的关键技术。尽管其有效，传统NS迭代因重复高维矩阵乘法导致计算开销大。为克服这些限制，我们提出无迭代牛顿-施卢茨正交化（IFNSO）新框架，将传统迭代结构整合为统一的无迭代形式。通过分析单个矩阵幂的贡献，我们通过去除不显著项并引入可学习系数的多项式来简化流程。这些系数被优化以确保优越的计算效率和稳定的收敛性。广泛实验表明，IFNSO在现有方法上表现更优。我们的代码可在：https://github.com/greekinRoma/Unified_Newton_Schulz_Orthogonalization。
英文摘要：
The Newton-Schulz (NS) iteration has become a key technique for orthogonalization in optimizers such as Muon and for optimization on the Stiefel manifold. Despite its effectiveness, the conventional NS iteration incurs significant computational overhead due to repeated high-dimensional matrix multiplications. To overcome these limitations, we propose Iteration-Free Newton-Schulz Orthogonalization (IFNSO), a novel framework that consolidates the traditional iterative structure into a unified and Iteration-Free formulation. By analyzing the contribution of individual matrix powers, we streamline the process by removing insignificant terms and introducing a polynomial with learnable coefficients. These coefficients are optimized to ensure both superior computational efficiency and stable convergence. Extensive experiments demonstrate that IFNSO achieves superior performance compared to existing methods. Our code is available at: https://github.com/greekinRoma/Unified_Newton_Schulz_Orthogonalization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
