发表机构
Laboratoire des Systèmes Perceptifs, Ecole Normale Supérieure, PSL University, CNRS; Institut de Biologie de l’Ecole Normale Supérieure (IBENS), Ecole Normale Supérieure, PSL University, CNRS, INSERM（感知系统实验室，巴黎高等师范学院，PSL大学，法国国家科学研究中心; 巴黎高等师范学院生物学研究所（IBENS），巴黎高等师范学院，PSL大学，法国国家科学研究中心，法国健康与医学研究院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究提出解析框架，揭示分段线性神经网络在梯度对齐学习的平稳点处分解为局部线性回归，形成决策树结构，并阐明神经选择性组织，预测与模拟及经验数据匹配。
AI 中文摘要
非线性神经网络会发展出结构化的内部表征，然而其几何结构如何由所学任务决定，目前仍知之甚少。在此，我们为分段线性的前馈和循环网络开发了一个解析框架，该框架将学习、激活区域结构和神经选择性联系起来。我们证明，在梯度对齐学习的任何平稳点处，非线性网络都会在其激活区域上分解为局部线性回归。与相应最小二乘解的偏差受到跨区域共享的网络权重的共同约束，并在低误差区域内消失，从而产生任务的分段最小二乘近似分解。这种结构允许决策树解释，我们通过拟合树来从输入预测网络激活模式，从而在数值上恢复该解释。我们进一步推导了神经元活跃的区域如何决定它们捕获的任务统计量，从而将神经选择性组织为不同的亚群。由此产生的预测与模拟网络和两个经验神经数据集中观察到的选择性几何结构密切匹配。最后，我们表明神经基线调节激活模式的多样性，将网络置于从粗略、泛化的表征到精细、表达性表征的连续谱上。总之，这些结果确立了激活区域作为一个统一框架，用于描述非线性网络如何将任务结构分解为局部计算，以及这些计算如何通过神经活动被揭示。
英文摘要
Nonlinear neural networks develop structured internal representations, yet how their geometry is determined by the tasks being learned remains poorly understood. Here, we develop an analytical framework for piecewise-linear feedforward and recurrent networks that links learning, activation-region structure, and neural selectivity. We show that, at any stationary point of gradient-aligned learning, a nonlinear network decomposes into local linear regressions over its activation regions. Deviations from the corresponding least-squares solutions are jointly constrained by network weights shared across regions and vanish in the low-error regime, yielding an approximate piecewise least-squares decomposition of the task. This structure admits a decision-tree interpretation, which we recover numerically by fitting trees to predict network activation patterns from the input. We further derive how the regions in which neurons are active determine the task statistics they capture, thereby organizing neural selectivity into distinct subpopulations. The resulting predictions closely match the selectivity geometry observed in simulated networks and two empirical neural datasets. Finally, we show that neural baseline regulates activation pattern diversity, placing networks along a continuum between coarse, generalizing representations and fine-grained, expressive representations. Together, these results establish activation regions as a unifying framework for describing how nonlinear networks decompose task structure into local computations and how those computations can be revealed through neural activity.
Comments37 pages, 9 Figures, 4 Supplementary Figures
