发表机构
The Harker School; University of California, Davis（哈克学校; 加利福尼亚大学戴维斯分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究评估了生物学合理的无监督局部剪枝规则noise-prune，发现其在任务训练后的循环神经网络中保持任务性能的效果优于仅用连接幅度的策略，确定了其最优参数设置。
AI 中文摘要
网络连接的剪枝是大脑功能的关键，但尽管其重要性突出，目前仍缺乏经过验证且性能良好的生物学合理剪枝规则。在本研究中，我们评估了noise-prune，这是一种最近提出的用于循环网络的无监督局部剪枝规则，它利用噪声波动来确定连接的重要性。此前，noise-prune仅在无特定计算功能的随机网络上进行过实证测试。我们表明，noise-prune在任务训练后的循环神经网络中能够保持任务性能，其表现远超仅使用连接幅度的策略，且与使用二阶信息的非局部策略表现相当或更优。noise-prune并非确定性地移除重要性低于某一阈值的连接，而是根据连接的重要性采样要保留的连接，并强化保留的连接以维持平均突触强度。我们发现，这种采样和重缩放对于良好性能至关重要，但最优的经验重缩放程度低于原始理论论证所预测的程度。因此，本研究验证了noise-prune是一种适用于功能循环网络架构的生物学合理剪枝规则，并确定了其最优参数设置。
英文摘要
The pruning of network connections is key to brain function but, despite its importance, there exist few biologically-plausible pruning rules with demonstrated good performance. In this work we evaluate noise-prune, a recently introduced unsupervised local pruning rule for recurrent networks that uses noisy fluctuations to determine the importance of connections. Noise-prune has previously only been empirically tested on random networks without a specific computational function. We show that noise-prune preserves task-performance in task-trained recurrent neural networks, greatly outperforming a strategy that only uses the magnitude of connections and performing on par with or exceeding a non-local strategy that uses second-order information. Rather than deterministically removing connections that fall below a certain threshold importance, noise-prune samples connections to preserve based on their importance and strengthens retained connections to preserve average synaptic strength. We show that this sampling and rescaling is essential to good performance, but that the optimal empirical degree of rescaling is lower than that predicted by the original theoretical argument. Our work thus validates noise-prune as a biologically-plausible pruning rule for functional recurrent network architectures and characterizes its optimal parameter settings.
