基于松弛变量分裂法的$\ell_0$和T $\ell_1$稀疏神经网络及其在多尺度曲线分类中的应用
Learning Sparse Neural Networks via $\ell_0$ and T$\ell_1$ by a Relaxed Variable Splitting Method with Application to Multi-scale Curve Classification
摘要:
我们研究稀疏化的卷积神经网络(CNN)的松弛变量分裂方法的$\ell_0$和转换- $\ell_1$(T $\ell_1$)的处罚,与应用程序复杂的曲线,如文字写在不同的字体,和文字写的颤抖的手模拟帕金森氏病患者。CNN包含3个卷积层,每个卷积层后面都有一个最大池,最后是一个包含最大数量网络权重的全连接层。在$\ell_0$惩罚的情况下,我们在区分不稳定字体与常规字体或手写体方面实现了超过99 \%的测试准确率,其中全连接层中超过86 \%的权重为零。通过适当选择T $\ell_1$惩罚,也达到了相当的稀疏性和测试精度。
英文摘要:
We study sparsification of convolutional neural networks (CNN) by a relaxed variable splitting method of $\ell_0$ and transformed-$\ell_1$ (T$\ell_1$) penalties, with application to complex curves such as texts written in different fonts, and words written with trembling hands simulating those of Parkinson's disease patients. The CNN contains 3 convolutional layers, each followed by a maximum pooling, and finally a fully connected layer which contains the largest number of network weights. With $\ell_0$ penalty, we achieved over 99 \% test accuracy in distinguishing shaky vs. regular fonts or hand writings with above 86 \% of the weights in the fully connected layer being zero. Comparable sparsity and test accuracy are also reached with a proper choice of T$\ell_1$ penalty.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。