发表机构
School of Mathematics, Jilin University（吉林大学数学学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究正二次网络的商结构对训练动力学等的影响，通过推导有效黑塞矩阵等方法，在高斯测量下计算曲率等，建立收敛性，还研究了欠定情况及初始化影响，数值实验验证相关结论。
AI 中文摘要
正二次网络具有低秩表示\(f_U(x)=x^T UU^T x\)，其中\(U\in\mathbb{R}^{d\times r}\)仅在右正交乘法下可识别，代表秩为\(r\)的PSD矩阵\(Q = UU^T\)。本文研究这种商结构如何控制训练动力学、曲率、恢复和插值偏差。在满列秩层，将\(\mathbb{R}^{d\times r}_*/O(r)\)与秩为\(r\)的PSD流形等同。对于光滑目标\(L(U)=\ell(UU^T)\)，欧几里得因子梯度是水平的。因子梯度流精确投影到商黎曼梯度流，有限步梯度下降为预测器诱导精确的同余递归。对于二次回归，推导插值器处的有效黑塞矩阵为相对于商度量限制在切空间的经验测量Gram形式。在高斯秩一测量下，计算总体曲率，证明经验正规算子的一致偏差界，构造谱初始化器，并建立梯度流的局部指数收敛和小步下降的线性收敛。恢复保证明确但保守，因为依赖全空间二阶矩控制。在欠定交换 regime中，因子梯度流在联合谱坐标中成为精确的熵镜像流。严格正初始化收敛到插值集上的Bregman投影。对于各向同性初始化\(q(0)=\varepsilon^2\mathbf{1}\)，预测器随着\(\varepsilon\downarrow0\)接近最小迹解集，通过不变联合谱代数内的加权熵解决非唯一性。有限步下降选择与连续时间Bregman投影相差\(O(\eta)\)的插值器。数值实验验证了这些商恒等式、曲率预测、恢复行为和选择定律。
英文摘要
Positive quadratic networks admit the low-rank representation f_U(x)=x^top UU^top x, where Uinmathbb{R}^{dtimes r} is identifiable only up to right orthogonal multiplication, representing a rank-r PSD matrix Q=UU^top. We study how this quotient structure governs training dynamics, curvature, recovery, and interpolation bias. On the full-column-rank stratum, we identify mathbb{R}^{dtimes r}_*/O(r) with the rank-r PSD manifold. For smooth objectives L(U)=ell(UU^top), the Euclidean factor gradient is horizontal. Thus, factor gradient flow projects exactly to quotient Riemannian gradient flow, while finite-step gradient descent induces an exact congruence recursion for the predictor. For quadratic regression, we derive the effective Hessian at interpolators as the empirical measurement Gram form restricted to the tangent space relative to the quotient metric. Under Gaussian rank-one measurements, we compute population curvature, prove uniform deviation bounds for the empirical normal operator, construct a spectral initializer, and establish local exponential convergence for gradient flow and linear convergence for small-step descent. Recovery guarantees are explicit but conservative due to reliance on full-space second-moment control. In underdetermined commuting regimes, factor gradient flow becomes an exact entropy mirror flow in joint spectral coordinates. Strictly positive initializations converge to Bregman projections onto the interpolation set. With isotropic initialization q(0)=varepsilon^2mathbf{1}, predictors approach the minimum-trace solution set as varepsilondownarrow0, resolving nonuniqueness via weighted entropy within the invariant joint spectral algebra. Finite-step descent selects interpolants differing from continuous-time Bregman projections by O(eta). Numerical experiments verify these quotient identities, curvature predictions, recovery behaviors, and selection laws.
Comments75 pages, 19 figures
