弱相关突触促进深度神经网络降维
Weakly-correlated synapses promote dimension reduction in deep neural networks
摘要：
通过控制突触和神经相关性，深度学习在提高分类性能方面取得了经验性成功。突触相关性如何影响神经相关性以产生解开的隐藏表征仍然是难以捉摸的。在这里，我们提出了一个简化的降维模型，考虑到突触之间的成对相关性，揭示突触相关性如何影响降维的机制。我们的理论确定的突触相关缩放形式只需要数学的自洽性，二进制和连续突触。该理论还预测，弱相关的突触与正交的突触相比，会促进维度的降低。此外，这些突触减缓了沿网络深度的去相关过程。这两个计算的角色解释所提出的平均场方程。理论预测与数值模拟非常一致，并且通过使用Hebbian规则的深度学习也捕获了关键特征。
英文摘要：
By controlling synaptic and neural correlations, deep learning has achieved empirical successes in improving classification performances. How synaptic correlations affect neural correlations to produce disentangled hidden representations remains elusive. Here we propose a simplified model of dimension reduction, taking into account pairwise correlations among synapses, to reveal the mechanism underlying how the synaptic correlations affect dimension reduction. Our theory determines the synaptic-correlation scaling form requiring only mathematical self-consistency, for both binary and continuous synapses. The theory also predicts that weakly-correlated synapses encourage dimension reduction compared to their orthogonal counterparts. In addition, these synapses slow down the decorrelation process along the network depth. These two computational roles are explained by the proposed mean-field equation. The theoretical predictions are in excellent agreement with numerical simulations, and the key features are also captured by a deep learning with Hebbian rules.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
