arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2004.10700cs.LGcs.CRcs.ITmath.ITstat.ML
收藏

CodNN --编码分类的鲁棒神经网络

CodNN -- Robust Neural Networks From Coded Classification

Washington University in St. Louis(圣路易斯华盛顿大学);California Institute of Technology(加州理工学院);Texas A&M University(德州农工大学)

Netanel Raviv, Siddharth Jain, Pulakesh Upadhyaya, Jehoshua Bruck, Anxiao Jiang

摘要:

深度神经网络(DNN)是正在进行的信息革命中的革命性力量,但其内在属性仍然是一个谜。特别是,众所周知,DNN对噪声高度敏感,无论是对抗性的还是随机的。这对DNN的硬件实现及其在自动驾驶等关键应用中的部署提出了根本性挑战。在本文中,我们通过纠错码构造鲁棒DNN。通过我们的方法,DNN的数据或内部层都用纠错码编码,并且保证了在噪声下的成功计算。由于DNN可以被视为分类任务的分层级联,因此我们的研究从对噪声编码输入进行分类的核心任务开始,并朝着鲁棒DNN的方向发展。我们专注于二进制数据和线性代码。我们的主要结果是,流行的奇偶校验码可以保证一个大家庭的DNN,其中包括最近流行的二进制神经网络的鲁棒性。此外,我们表明,编码分类问题有着深刻的联系,布尔函数的傅立叶分析。与文献中现有的解决方案相比,我们的结果不依赖于改变DNN的训练过程,并提供数学上严格的保证,而不是实验证据。

英文摘要:

Deep Neural Networks (DNNs) are a revolutionary force in the ongoing information revolution, and yet their intrinsic properties remain a mystery. In particular, it is widely known that DNNs are highly sensitive to noise, whether adversarial or random. This poses a fundamental challenge for hardware implementations of DNNs, and for their deployment in critical applications such as autonomous driving. In this paper we construct robust DNNs via error correcting codes. By our approach, either the data or internal layers of the DNN are coded with error correcting codes, and successful computation under noise is guaranteed. Since DNNs can be seen as a layered concatenation of classification tasks, our research begins with the core task of classifying noisy coded inputs, and progresses towards robust DNNs. We focus on binary data and linear codes. Our main result is that the prevalent parity code can guarantee robustness for a large family of DNNs, which includes the recently popularized binarized neural networks. Further, we show that the coded classification problem has a deep connection to Fourier analysis of Boolean functions. In contrast to existing solutions in the literature, our results do not rely on altering the training process of the DNN, and provide mathematically rigorous guarantees rather than experimental evidence.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑