由低偏差序列生成的Artificial Neural Networks
Artificial Neural Networks generated by Low Discrepancy Sequences
AI总结：
提出利用低偏差序列在稠密图上生成稀疏神经网络，支持确定性初始化与从头稀疏训练，避免存储体冲突，在低计算复杂度下达到接近稠密网络的准确率。
摘要：
人工神经网络可以通过路径来表示。在稠密网络图上作为随机游走生成时，我们发现得到的稀疏网络允许确定性初始化，甚至允许具有固定符号的权重。这种网络可以从头开始稀疏训练，避免了先训练稠密网络再压缩的昂贵过程。尽管是稀疏的，权重仍作为连续内存块被访问。此外，使用确定性低偏差序列（例如Sobol'序列）枚举路径，相当于通过渐进排列连接神经单元层，这自然避免了并行计算机硬件中的存储体冲突。我们证明，由低偏差序列生成的人工神经网络能够达到接近其稠密对应物的准确率，且计算复杂度低得多。
英文摘要：
Artificial neural networks can be represented by paths. Generated as random walks on a dense network graph, we find that the resulting sparse networks allow for deterministic initialization and even weights with fixed sign. Such networks can be trained sparse from scratch, avoiding the expensive procedure of training a dense network and compressing it afterwards. Although sparse, weights are accessed as contiguous blocks of memory. In addition, enumerating the paths using deterministic low discrepancy sequences, for example the Sobol' sequence, amounts to connecting the layers of neural units by progressive permutations, which naturally avoids bank conflicts in parallel computer hardware. We demonstrate that the artificial neural networks generated by low discrepancy sequences can achieve an accuracy within reach of their dense counterparts at a much lower computational complexity.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
