arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2103.14731cs.LGeess.IVeess.SPstat.AP
收藏

现代神经网络的非光滑性建模

Modeling the Nonsmoothness of Modern Neural Networks

NC State University(北卡罗来纳州立大学)

Runze Liu, Chau-Wai Wong, Huaiyu Dai

摘要:

现代神经网络在许多基于回归的任务中取得了成功,例如人脸识别,面部标志检测和图像生成。在这项工作中,我们研究了一个直观的,但研究不足的现代神经网络的特点,即非光滑性。使用合成数据的实验证实,现代神经网络中的ReLU和max pooling等操作会导致非平滑性。我们量化的非平滑性使用的功能命名为峰值的幅度之和(SMP)和现代神经网络的构建模块的输入输出关系模型。实验结果证实,我们的模型可以准确地预测非平滑性的统计行为,因为它通过卷积层,ReLU激活和最大池化层等构建块传播。我们设想,非平滑功能可以潜在地用作神经网络的基于回归的应用程序的取证工具。

英文摘要:

Modern neural networks have been successful in many regression-based tasks such as face recognition, facial landmark detection, and image generation. In this work, we investigate an intuitive but understudied characteristic of modern neural networks, namely, the nonsmoothness. The experiments using synthetic data confirm that such operations as ReLU and max pooling in modern neural networks lead to nonsmoothness. We quantify the nonsmoothness using a feature named the sum of the magnitude of peaks (SMP) and model the input-output relationships for building blocks of modern neural networks. Experimental results confirm that our model can accurately predict the statistical behaviors of the nonsmoothness as it propagates through such building blocks as the convolutional layer, the ReLU activation, and the max pooling layer. We envision that the nonsmoothness feature can potentially be used as a forensic tool for regression-based applications of neural networks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑