arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.02482cs.ITcs.LGcs.NEmath.IT

神经元应以什么为目标?基于信息论设计局部目标函数

What should a neuron aim for? Designing local objective functions based on information theory

Faculty of Physics, Institute for the Dynamics of Complex Systems, University of Göttingen(物理学院、复杂系统动力学研究所、哥廷根大学);Max Planck Institute for Dynamics and Self-Organization, Göttingen(动态与自我组织马克斯·普朗克研究所,哥廷根);Göttingen Campus Institute for Dynamics of Biological Networks, University of Göttingen(哥廷根校区生物网络动力学研究所、哥廷根大学);Institute of Computer Science and Campus Institute Data Science, University of Göttingen(计算机科学研究所和校园数据科学研究所、哥廷根大学)

Andreas C. Schneider, Valentin Neuhaus, David A. Ehrlich, Abdullah Makkeh, Alexander S. Ecker, Viola Priesemann, Michael Wibral

AI总结:

本文提出基于偏信息分解(PID)的神经元局部目标框架,通过加权组合独特、冗余与协同信息项来调控前馈、反馈和侧向输入,在保持强性能的同时实现神经元级可解释性。

摘要:

在现代深度神经网络中,由于网络通过全局优化进行训练,单个神经元的学习动态往往晦涩不明。相反,生物系统依赖自组织的局部学习,在有限全局信息下实现鲁棒性和效率。我们在此展示,如何通过设计受生物学启发的抽象局部学习目标,实现单个人工神经元之间的自组织。这些目标利用信息论的近期扩展——偏信息分解(Partial Information Decomposition,PID)进行参数化;PID 将一组信息源关于某个结果所携带的信息分解为独特、冗余和协同贡献。我们的框架使神经元能够局部塑造来自不同输入类别(即前馈、反馈和侧向)的信息整合,方式是选择三种输入中的哪一种应以独特、冗余或协同方式对输出作出贡献。这种选择表示为 PID 项的加权和;对于给定问题,该加权和可以直接由直觉推理导出,或通过数值优化得到,从而为理解任务相关的局部信息处理提供窗口。通过实现神经元级可解释性,同时利用局部学习获得强劲性能,我们的工作为局部学习策略推进了一项有原则的信息论基础。

英文摘要:

In modern deep neural networks, the learning dynamics of the individual neurons is often obscure, as the networks are trained via global optimization. Conversely, biological systems build on self-organized, local learning, achieving robustness and efficiency with limited global information. We here show how self-organization between individual artificial neurons can be achieved by designing abstract bio-inspired local learning goals. These goals are parameterized using a recent extension of information theory, Partial Information Decomposition (PID), which decomposes the information that a set of information sources holds about an outcome into unique, redundant and synergistic contributions. Our framework enables neurons to locally shape the integration of information from various input classes, i.e. feedforward, feedback, and lateral, by selecting which of the three inputs should contribute uniquely, redundantly or synergistically to the output. This selection is expressed as a weighted sum of PID terms, which, for a given problem, can be directly derived from intuitive reasoning or via numerical optimization, offering a window into understanding task-relevant local information processing. Achieving neuron-level interpretability while enabling strong performance using local learning, our work advances a principled information-theoretic foundation for local learning strategies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑