arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2006.11671cs.LGcs.MAcs.NEstat.ML
收藏

利他多样化神经网络集成的集体学习

Collective Learning by Ensembles of Altruistic Diversifying Neural Networks

Weizmann Institute of Science(魏茨曼科学研究所)

Benjamin Brazowski, Elad Schneidman

摘要:

组合神经网络集合的预测通常优于最好的单个网络。这种集合通常是独立训练的,其优越的“群体智慧”源于网络之间的差异。在社会互动的动物群体中,集体觅食和决策往往会得到改善,甚至是最佳的,这要归功于同种动物之间的局部信息共享。因此,我们提出了一个共同学习的相互作用的神经网络,旨在最大限度地提高自己的性能,但也与其他网络的功能关系的合奏模型。我们发现,合奏的相互作用的网络优于独立的,并达到最佳的合奏性能时,网络之间的耦合增加多样性和降低个别网络的性能。因此,即使没有一个整体目标,最佳的集体行为也会从网络之间的局部相互作用中产生。我们展示了最佳耦合强度与合奏大小的缩放,以及这些合奏中的网络在功能上的专业化,并在他们的评估变得更加“自信”。此外,与独立训练的网络相比,最佳协同学习网络在结构上有所不同,依赖于更稀疏的活动,更广泛的突触权重和更高的发射率。最后,我们探索基于互动的共同学习作为一个框架,扩大和提高合奏。

英文摘要:

Combining the predictions of collections of neural networks often outperforms the best single network. Such ensembles are typically trained independently, and their superior `wisdom of the crowd' originates from the differences between networks. Collective foraging and decision making in socially interacting animal groups is often improved or even optimal thanks to local information sharing between conspecifics. We therefore present a model for co-learning by ensembles of interacting neural networks that aim to maximize their own performance but also their functional relations to other networks. We show that ensembles of interacting networks outperform independent ones, and that optimal ensemble performance is reached when the coupling between networks increases diversity and degrades the performance of individual networks. Thus, even without a global goal for the ensemble, optimal collective behavior emerges from local interactions between networks. We show the scaling of optimal coupling strength with ensemble size, and that networks in these ensembles specialize functionally and become more `confident' in their assessments. Moreover, optimal co-learning networks differ structurally, relying on sparser activity, a wider range of synaptic weights, and higher firing rates - compared to independently trained networks. Finally, we explore interactions-based co-learning as a framework for expanding and boosting ensembles.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑