受网络科学启发的自适应稀疏连接人工神经网络的可扩展训练
Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Science
摘要：
通过深度学习在各个领域的成功，人工神经网络目前是最常用的人工智能方法之一。从生物神经网络的网络特性（例如稀疏性、无标度性）中获得灵感，我们认为（与一般实践相反）人工神经网络也不应该具有完全连接的层。在这里，我们提出了人工神经网络的稀疏进化训练，在学习过程中，将两个连续神经元层的初始稀疏拓扑（Erdens-Rényi随机图）进化为无标度拓扑的算法。我们的方法取代人工神经网络全连接层与稀疏的训练前，减少了二次参数的数量，而不降低精度。我们在15个数据集上证明了我们对限制玻尔兹曼机，多层感知器和卷积神经网络的无监督和监督学习的要求。我们的方法有可能使人工神经网络的规模超过目前的可能。
英文摘要：
Through the success of deep learning in various domains, artificial neural networks are currently among the most used artificial intelligence methods. Taking inspiration from the network properties of biological neural networks (e.g. sparsity, scale-freeness), we argue that (contrary to general practice) artificial neural networks, too, should not have fully-connected layers. Here we propose sparse evolutionary training of artificial neural networks, an algorithm which evolves an initial sparse topology (Erdős-Rényi random graph) of two consecutive layers of neurons into a scale-free topology, during learning. Our method replaces artificial neural networks fully-connected layers with sparse ones before training, reducing quadratically the number of parameters, with no decrease in accuracy. We demonstrate our claims on restricted Boltzmann machines, multi-layer perceptrons, and convolutional neural networks for unsupervised and supervised learning on 15 datasets. Our approach has the potential to enable artificial neural networks to scale up beyond what is currently possible.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
