arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2209.11883cs.NEcs.LGq-bio.NC
收藏

无需反馈的赫布深度学习

Hebbian Deep Learning Without Feedback

Huawei Zurich Research Center(华为苏黎世研究中心);Huawei ACS Lab(华为ACS实验室)

Adrien Journé, Hector Garcia Rodriguez, Qinghai Guo, Timoleon Moraitis

AI总结:

基于软胜者全得网络的赫布学习理论,提出无需反馈信号的多层SoftHebb算法,在保持高生物兼容性与效率的同时提升了深度学习的准确率。

摘要:

反向传播(BP)的近期近似方法缓解了BP的诸多计算低效和生物学不兼容问题,但重要局限依然存在。此外,这些近似方法在基准测试中显著降低了准确率,表明一种完全不同的方法可能更为有效。在此,基于软胜者全得网络中赫布学习的最新理论,我们提出了多层SoftHebb,即一种无需任何反馈、目标或误差信号即可训练深度神经网络的算法。因此,它通过避免权重传输、非局部可塑性、层更新的时间锁定、迭代平衡以及(自)监督或其他反馈信号来实现高效性——而这些在其他方法中是必需的。与最先进的生物似真学习相比,其提升的效率和生物兼容性并未以牺牲准确率为代价,反而有所改善。在多达五个隐藏层和附加线性分类器的情况下,在MNIST、CIFAR-10、STL-10和ImageNet上的准确率分别达到99.4%、80.3%、76.2%和27.3%。结论表明,SoftHebb以与BP截然不同的方法证明了少数层上的深度学习在脑中可能是似真的,并提高了生物似真机器学习的准确率。代码可在https://github.com/NeuromorphicComputing/SoftHebb获取。

英文摘要:

Recent approximations to backpropagation (BP) have mitigated many of BP's computational inefficiencies and incompatibilities with biology, but important limitations still remain. Moreover, the approximations significantly decrease accuracy in benchmarks, suggesting that an entirely different approach may be more fruitful. Here, grounded on recent theory for Hebbian learning in soft winner-take-all networks, we present multilayer SoftHebb, i.e. an algorithm that trains deep neural networks, without any feedback, target, or error signals. As a result, it achieves efficiency by avoiding weight transport, non-local plasticity, time-locking of layer updates, iterative equilibria, and (self-) supervisory or other feedback signals -- which were necessary in other approaches. Its increased efficiency and biological compatibility do not trade off accuracy compared to state-of-the-art bio-plausible learning, but rather improve it. With up to five hidden layers and an added linear classifier, accuracies on MNIST, CIFAR-10, STL-10, and ImageNet, respectively reach 99.4%, 80.3%, 76.2%, and 27.3%. In conclusion, SoftHebb shows with a radically different approach from BP that Deep Learning over few layers may be plausible in the brain and increases the accuracy of bio-plausible machine learning. Code is available at https://github.com/NeuromorphicComputing/SoftHebb.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑