发表机构
UC San Diego; University of Pennsylvania; The Wharton School; Halıcıoğlu Data Science Institute（加州大学圣地亚哥分校; 宾夕法尼亚大学; 沃顿商学院; 哈利西奥格鲁数据科学学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究指出现有特征学习理论仅聚焦全局低维预测几何的不足，发现MLPs在聚类数据回归中会产生单义性专用神经元，形成局部低维表示集合，从而获得数据效率优势。
AI 中文摘要
理解神经网络如何学习并组织特征，是理解其行为的核心。现有大量特征学习理论聚焦于全局低维预测几何的涌现，我们证明该图景并不完整。在具有聚类数据的回归问题中，我们展示多层感知机（MLPs）会自然发展出单义性专用神经元：单个神经元会与输入空间特定区域相关的某一具体预测特征强对齐。MLPs并非学习单一全局低维表示，而是学习一组局部低维表示，可共同张成高维空间。这种专业化可证明使MLPs相较基于全局低维表示的特征学习方法具备数据效率优势。
英文摘要
Understanding how neural networks learn and organize features is central to understanding their behavior. Much existing theory of feature learning has focused on the emergence of a global low-dimensional representation. We show that this picture is incomplete. In regression problems with clustered data, we demonstrate that multilayer perceptrons (MLPs) naturally develop monosemantic specialized neurons: individual neurons become strongly aligned with a specific predictive feature relevant to a particular region of the input space. Rather than learning a single global low-dimensional representation, MLPs learn a collection of local low-dimensional representations. We show that this ability to specialize gives MLPs a provable data-efficiency advantage over feature-learning methods based on a global low-dimensional representation.
