arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2207.07232cs.LGcs.AIcs.CV
收藏

神经网络的Lipschitz界分析

Lipschitz Bound Analysis of Neural Networks

University of Calcutta(加尔各答大学)

Sarosij Bose

摘要:

Lipschitz界估计是一种有效的正则化深度神经网络的方法,使其对对抗攻击具有鲁棒性。这在从强化学习到自治系统的各种应用中都很有用。在本文中,我们强调了在获得卷积神经网络(CNN)的非平凡Lipschitz边界证书方面的重大差距,并通过大量的图形分析来实证支持它。我们还表明,展开卷积层或Toeplitz矩阵可以用于将卷积神经网络(CNN)转换为全连接网络。此外,我们提出了一个简单的算法,以显示现有的20倍-50倍的差距,在一个特定的数据分布之间的实际lipschitz常数和所获得的紧界。我们还在各种网络架构上进行了一系列彻底的实验,并在MNIST和CIFAR-10等数据集上进行了基准测试。所有这些建议都得到了广泛的测试、图表、直方图和比较分析的支持。

英文摘要:

Lipschitz Bound Estimation is an effective method of regularizing deep neural networks to make them robust against adversarial attacks. This is useful in a variety of applications ranging from reinforcement learning to autonomous systems. In this paper, we highlight the significant gap in obtaining a non-trivial Lipschitz bound certificate for Convolutional Neural Networks (CNNs) and empirically support it with extensive graphical analysis. We also show that unrolling Convolutional layers or Toeplitz matrices can be employed to convert Convolutional Neural Networks (CNNs) to a Fully Connected Network. Further, we propose a simple algorithm to show the existing 20x-50x gap in a particular data distribution between the actual lipschitz constant and the obtained tight bound. We also ran sets of thorough experiments on various network architectures and benchmark them on datasets like MNIST and CIFAR-10. All these proposals are supported by extensive testing, graphs, histograms and comparative analysis.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑