发表机构
University of Toronto; Emory University（多伦多大学; 埃默里大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究提出ManifoldFlow，通过\(W = Q S^{1/2}\)在斯蒂费尔流形上学习有界正谱，放松固定谱斯蒂费尔层约束，在部分实验中可学习的对称正定谱表现更好。
AI 中文摘要
正交层和斯蒂费尔层能精确控制神经权重谱，但施加了强建模约束。我们引入ManifoldFlow，它是固定谱斯蒂费尔层的最小松弛，通过\(W = Q S^{1/2}\)在斯蒂费尔流形上学习有界正谱，\(W^T W = S\)使特征值裁剪成为直接奇异值控制机制，在部分实验中可学习的对称正定谱表现更好。
英文摘要
Orthogonal and Stiefel layers give neural weights exact spectral control, but they also impose a strong modeling constraint: all represented singular values are fixed at one. Many settings that benefit from an orthonormal basis still need direction-dependent attenuation or amplification. We introduce ManifoldFlow, a minimal relaxation of a fixed-spectrum Stiefel layer that keeps the basis on the Stiefel manifold while learning a bounded positive spectrum through W = Q S^{1/2}, with Q^T Q = I and S positive definite. Since W^T W = S, the eigenvalues of S are exactly the squared singular values of the realized weight, making eigenvalue clipping a direct singular-value control mechanism. Across paired sequence, tabular, and image experiments, the learnable SPD spectrum improves the fixed-spectrum Stiefel counterpart in the reported settings where the Stiefel prior is useful, with the largest gains in recurrent language-model projections. Boundary cases in convolutional classifier heads clarify the intended scope: ManifoldFlow is not a universal dense-layer replacement, but a spectrum-learnable Stiefel relaxation for settings where an orthonormal basis is a useful prior. When the basis should be orthonormal, its spectrum need not be frozen. Code available at https://github.com/Hik289/manifold_flow
Comments35 pages
