LLN：用于参数高效长时程动力学预测的可学习透镜网络

中文摘要 AI 辅助

形如(x+f(x))的显式残差连接，通常与归一化层结合使用，已成为训练极深度神经网络的标准策略。然而，残差加法主要提供了梯度传播的代数捷径，而跨层特征几何的演化在很大程度上不受约束。我们引入了可学习透镜网络（LLN），这是一种受物理启发的架构，在增广的位置-角度相空间中用可学习的光学传输替代直接的特征空间残差累积。每一层在自由传播（提供隐式传输路径）与可学习透镜场（执行非线性轨迹变换和聚焦）之间交替进行。理论上，我们证明了对于任何可微的透镜场，LLN传输是全局可逆且体积保持的，所实现的逐坐标高斯传输进一步满足辛性。重要的是，这些结构约束并不限制表达能力：通过无限制的嵌入和读出，LLN保留了对连续端到端映射的通用逼近能力。跨多种动力学系统的实验表明，LLN在显著减少参数数量的同时（与同深度比较器相比），提升了长时程预测性能。进一步的分析揭示了在耦合的传播与折射设计下，稳定的逐层梯度传输和可解释的学习动力学。

英文摘要

Explicit residual connections of the form (x+f(x)), often combined with normalization layers, have become a standard strategy for training very deep neural networks. However, residual addition primarily provides an algebraic shortcut for gradient propagation, while leaving the evolution of feature geometry across layers largely unconstrained. We introduce Learnable Lens Networks (LLN), a physics-inspired architecture that replaces direct feature-space residual accumulation with learnable optical transport in an augmented position-angle phase space. Each layer alternates between free propagation, which provides an implicit transport path, and a learnable lens field that performs nonlinear trajectory transformation and focusing. Theoretically, we establish that LLN transport is globally invertible and volume-preserving for any differentiable lens field, with the implemented coordinate-wise Gaussian transport further satisfying symplecticity. Importantly, these structural constraints do not limit expressivity: with unrestricted embeddings and readouts, LLN retain universal approximation of continuous end-to-end maps. Experiments across diverse dynamical systems demonstrate that LLN improves long-horizon prediction while using substantially fewer parameters than same-depth comparators. Further analysis reveals stable depth-wise gradient transport and interpretable learned dynamics under the coupled propagation and refraction design.

发表机构

• Lanzhou University（兰州大学）
• Monash University（莫纳什大学）
• University of Wollongong（伍伦贡大学）
• Nanjing University of Aeronautics and Astronautics（南京航空航天大学）

机构由 AI 辅助整理，请以论文原文为准。