平均激活算子驱动的分层网络结构的Lipschitz证明
Lipschitz Certificates for Layered Network Structures Driven by Averaged Activation Operators
摘要:
获得尖锐的Lipschitz常数的前馈神经网络是必不可少的,以评估其鲁棒性,面对其输入的扰动。我们推导出这样的常数的上下文中的一般分层网络模型,涉及非扩张平均运营商和仿射运营商的组合物。通过利用这种架构,我们的分析精细地捕捉层之间的相互作用,产生更紧密的Lipschitz常数比那些从产品的各个层组的界限。所提出的框架,特别是前馈神经网络中遇到的许多实际情况。我们的Lipschitz常数估计在采用标量非线性函数的结构的情况下得到进一步改进,其中包括标准卷积网络作为特殊情况。
英文摘要:
Obtaining sharp Lipschitz constants for feed-forward neural networks is essential to assess their robustness in the face of perturbations of their inputs. We derive such constants in the context of a general layered network model involving compositions of nonexpansive averaged operators and affine operators. By exploiting this architecture, our analysis finely captures the interactions between the layers, yielding tighter Lipschitz constants than those resulting from the product of individual bounds for groups of layers. The proposed framework is shown to cover in particular many practical instances encountered in feed-forward neural networks. Our Lipschitz constant estimates are further improved in the case of structures employing scalar nonlinear functions, which include standard convolutional networks as special cases.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。