LipSSM：通过连续SSM层之间的度量传递实现结构上Lipschitz有界的级联状态空间模型
LipSSM: Structurally Lipschitz-Bounded Cascaded State-Space Model via Metric Transfer between Consecutive SSM Layers
浏览论文内容
中文总结 AI 辅助
本文提出LipSSM架构，通过度量传递将LipKernel思想扩展到级联状态空间模型，以构建Lipschitz连续且能建模长期依赖的深度神经网络，获得更紧的Lipschitz界并提升鲁棒性。
中文摘要 AI 辅助
Lipschitz连续性是为可认证鲁棒的深度神经网络（DNNs）设计的基本原则，其中调整Lipschitz常数（用于量化网络鲁棒性）具有核心理论重要性。强制Lipschitz连续性的标准方法要求DNN的每一层都是Lipschitz连续的，从而保证整体的Lipschitz连续性。然而，这种逐层方法通常会产生对整体Lipschitz常数的松散估计，从而施加过于保守的限制，这限制了DNN的表达能力，并在规定的鲁棒性水平下降级了经验性能。为了克服这种松散估计，最近提出的LipKernel通过跨层传递信息，比传统的逐层构造产生更紧的整体Lipschitz界。在本文中，我们将这一概念扩展到级联状态空间模型（SSMs），以构建能够建模长期依赖关系的Lipschitz连续DNN。所提出的架构名为LipSSM，在理论上得到了验证，并在经验上进行了评估。
英文摘要
Lipschitz continuity is a fundamental principle in the design of certifiably robust deep neural networks (DNNs), wherein adjusting the Lipschitz constant, which quantifies network robustness, is of central theoretical importance. A standard approach to enforcing Lipschitz continuity requires each layer of a DNN to be Lipschitz continuous, thereby guaranteeing overall Lipschitz continuity. However, this layer-wise approach typically imposes overly conservative restrictions by producing a loose estimate of the overall Lipschitz constant, which limits the expressive capacity of the DNN and degrades empirical performance at a prescribed level of robustness. To overcome this loose estimation, the recently proposed LipKernel transfers information across layers to yield a much tighter overall Lipschitz bound than conventional layer-wise construction. In this paper, we extend this concept to cascaded state-space models (SSMs) to construct Lipschitz-continuous DNNs capable of modeling longer-term dependencies. The proposed architecture, named LipSSM, is theoretically justified and empirically evaluated.
发表机构
- Tokyo University of Agriculture and Technology (TUAT)（东京农工大学）
机构由 AI 辅助整理，请以论文原文为准。
