正齐次神经网络中的尺度冗余与软规范固定
Scale redundancy and soft gauge fixing in positively homogeneous neural networks
AI总结：
该研究通过引入规范适应坐标和软轨道选择功能，揭示了正齐次神经网络中尺度冗余与优化条件之间的联系，提升了模型的稳定性与学习效率。
摘要：
具有正齐次激活函数的神经网络表现出精确的连续重新参数化对称性：神经元层面的缩放操作生成参数空间中的轨道，在这些轨道上输入-输出函数保持不变。我们将这种对称性解释为规范冗余，并引入了规范适应的坐标，将不变方向和尺度不平衡方向分开。受场论中规范固定的启发，我们引入了一个仅作用于冗余尺度坐标上的软轨道选择（范数平衡）功能。我们证明了该功能通过诱导不平衡模式的耗散松弛来保持实现的功能。在受控实验中，这种轨道选择惩罚扩展了稳定学习率范围，并抑制了尺度漂移，而不会改变表达能力。这些结果建立了规范轨道几何与优化条件之间的结构联系，为规范理论概念与机器学习之间提供了具体的连接。
英文摘要：
Neural networks with positively homogeneous activations exhibit an exact continuous reparametrization symmetry: neuron-wise rescalings generate parameter-space orbits along which the input--output function is invariant. We interpret this symmetry as a gauge redundancy and introduce gauge-adapted coordinates that separate invariant and scale-imbalance directions. Inspired by gauge fixing in field theory, we introduce a soft orbit-selection (norm-balancing) functional acting only on redundant scale coordinates. We show analytically that it induces dissipative relaxation of imbalance modes to preserve the realized function. In controlled experiments, this orbit-selection penalty expands the stable learning-rate regime and suppresses scale drift without changing expressivity. These results establish a structural link between gauge-orbit geometry and optimization conditioning, providing a concrete connection between gauge-theoretic concepts and machine learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
