稀疏超正则网络
Sparse Super-Regular Networks
摘要：
Thom和Palm认为稀疏连接的神经网络（SCN）比全连接网络（FCN）表现出更好的性能。超正则网络（Super-regular networks，SRN）是由一组稀疏的（S，delta）-超正则对堆叠而成的神经网络，节点顺序随机排列。使用爆破引理，我们证明，由于每对层的个人超正则性，SRN保证了一些属性，使它们适合替代许多任务的FCN。这些保证包括所有足够大的子集的边缘一致性，最小节点入度和出度，输入输出灵敏度以及嵌入预训练结构的能力。事实上，SRN有能力像FCN一样工作，并消除了对Dropout等昂贵的正则化方案的需求。我们通过易于重现的实验表明，SRN的表现与X-Nets相似，并且对网络结构提供了更大的保证和控制。
英文摘要：
It has been argued by Thom and Palm that sparsely-connected neural networks (SCNs) show improved performance over fully-connected networks (FCNs). Super-regular networks (SRNs) are neural networks composed of a set of stacked sparse layers of (epsilon, delta)-super-regular pairs, and randomly permuted node order. Using the Blow-up Lemma, we prove that as a result of the individual super-regularity of each pair of layers, SRNs guarantee a number of properties that make them suitable replacements for FCNs for many tasks. These guarantees include edge uniformity across all large-enough subsets, minimum node in- and out-degree, input-output sensitivity, and the ability to embed pre-trained constructs. Indeed, SRNs have the capacity to act like FCNs, and eliminate the need for costly regularization schemes like Dropout. We show that SRNs perform similarly to X-Nets via readily reproducible experiments, and offer far greater guarantees and control over network structure.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
