对称商平坦性与泛化
Symmetry-quotient Flatness and Generalization
AI总结：
本文提出对称商空间下的平坦性理论，证明商线性稳定性、商平坦性、输入光滑性与泛化之间的定理级联系，并给出基于批大小和学习率的显式界及人群泛化界。
AI中文摘要：
本文在对称商设置下建立了一个定理级流水线：商线性稳定性蕴含商平坦性，商平坦性蕴含输入光滑性，且在局部覆盖假设下输入光滑性可导出泛化。平坦性常与泛化相关联，随机梯度下降（SGD）也常被视为隐式偏向于平坦解。然而，标准平坦性度量通常在原始参数空间中定义，因此在保函数对称性（如正缩放）下不具有不变性。我们发展了一套对称感知的商平坦性、商线性稳定性、输入光滑性以及神经网络参数商空间上泛化的理论。对于平方损失和具有保函数群作用的模型，我们将商平坦性定义为正则商流形上经验损失Hessian的迹。我们证明商平坦性通过平坦性到光滑性论证的商空间类比来控制输入光滑性。我们还证明线性化SGD动力学的一步均方商线性稳定性蕴含一个以批大小和学习率表示的显式商平坦性界，并将此分析扩展到高阶张量矩。最后，在局部覆盖和有界性假设下，我们推导出以商平坦性及由此以商线性稳定性表示的人群泛化界。
英文摘要：
This paper develops a theorem-level pipeline in symmetry-quotient settings: quotient linear stability implies quotient flatness, quotient flatness implies input smoothness, and input smoothness yields generalization under local covering assumptions. Flatness is often associated with generalization, and Stochastic Gradient Descent (SGD) is frequently viewed as implicitly biased toward flat solutions. However, standard flatness measures are typically defined in the raw parameter space and are therefore not invariant under function-preserving symmetries such as positive rescaling. We develop a symmetry-aware theory of quotient flatness, quotient linear stability, input smoothness, and generalization on quotient spaces of neural-network parameters. For square loss and models equipped with function-preserving group actions, we define quotient flatness as the trace of the Hessian of the empirical loss on the regular quotient manifold. We show that quotient flatness controls input smoothness through a quotient-space analogue of the flatness-to-smoothness argument. We also prove that one-step mean-square quotient linear stability of the linearized SGD dynamics implies an explicit quotient-flatness bound in terms of the batch size and learning rate, and extend this analysis to higher-order tensor moments. Finally, under local covering and boundedness assumptions, we derive population generalization bounds in terms of quotient flatness and, consequently, in terms of quotient linear stability.
