Dale约束下的深度网络学习

中文摘要 AI 辅助

生物可解释学习模型旨在解释神经回路如何在真实神经元的约束下实现有效学习。尽管已取得显著进展，但仍存在一个主要挑战：现有模型通常允许神经元或突触表示正负混合的数值，这违背了皮层回路的一个基本方面——Dale约束：生物神经元要么是兴奋性的，要么是抑制性的，不能同时兼具两种属性，且突触不能改变符号。在本研究中，我们通过引入一种受生物启发的神经架构来解决这一差异，该架构中神经激活和学习信号均由非负活动表示，突触具有固定符号，同时仍支持类反向传播学习。我们的方法使用两个互补交互的非负通道来表示正负贡献，这一设计受大脑中开-关表征的证据启发。这些通道通过简单的神经回路基序实现，该基序在网络的自底向上和自顶向下通路中重复出现。结合局部Hebbian学习规则，所得模型仅利用神经元间的局部交互即可传播学习信号并更新权重。我们从理论上证明，尽管仅依赖非负误差信号，我们的学习方案仍能精确恢复反向传播更新。从经验上看，除了满足更强的生物约束外，该开-关架构还能学习高效的表征，在Tiny ImageNet基准测试中，相比可比的普通网络取得了显著提升。这些结果表明，有效的学习可以从生物可解释机制中产生，无需混合符号信号，为更现实的神经计算模型迈出了一步。

英文摘要

Biologically plausible learning models aim to explain how neural circuits can implement effective learning under the constraints of real neurons. Although significant progress has been made, a major remaining challenge is that existing models often allow neurons or synapses to represent mixed-sign values, both positive and negative, in violation of a basic aspect of cortical circuitry -- Dale's constraint: biological neurons are either excitatory or inhibitory, but not both, and synapses cannot change sign. In this work, we address this discrepancy by introducing a biologically motivated neural architecture in which both neural activations and learning signals are represented by non-negative activity, and synapses have fixed sign, while still supporting backpropagation-like learning. Our approach uses two complementary interacting non-negative channels to represent positive and negative contributions, inspired by evidence of on-off representations in the brain. These channels are implemented through a simple neural circuit motif, which is repeated throughout the network in both bottom-up and top-down pathways. Combined with a local Hebbian learning rule, the resulting model propagates learning signals and updates weights using only local interactions between neurons. We show theoretically that our learning scheme can exactly recover the backpropagation update despite relying solely on non-negative error signals. Empirically, beyond satisfying stronger biological constraints, the on-off architecture learns efficient representations, yielding substantial gains over comparable vanilla networks on the Tiny ImageNet benchmark. These results demonstrate that effective learning can emerge from biologically plausible mechanisms without requiring mixed-sign signals, providing a step toward more realistic models of neural computation.

发表机构

• Weizmann Institute of Science（魏茨曼科学研究所）

机构由 AI 辅助整理，请以论文原文为准。