发表机构
University of Houston（休斯顿大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究探讨大语言模型是否存在特定领域参数壳，通过多模型、多领域实验发现训练数据的标记级模块化是参数壳形成的关键，为大语言模型的参数模块化研究提供了核心结论。
AI 中文摘要
大语言模型是否包含特定领域的参数壳：即集中且具有因果必要性的神经元群体，移除这些神经元会选择性降低目标领域的性能，同时不影响其他领域？我们在两种领域粒度、三类模型家族（参数规模15亿至70亿）及八个领域上应用统一的因果方法。在学术学科层面，939008个合并的前馈神经网络（FFN）神经元中，没有任何神经元的领域选择性超过60%，尽管领域身份的线性解码准确率超过85%，但因果损伤矩阵呈平坦状。在语言与模态层面，0.65%至1.14%的神经元选择性超过60%，损伤矩阵近乎完全对角化（比率最高达595:1），且壳神经元集合基本不相交（交并比IoU < 0.003）。掩码代码选择性神经元会使所有模型的数学推理准确率降低16至24个百分点；掩码西班牙语或汉语神经元则使数学推理准确率降至或低于随机水平。壳强度随模型规模单调递增，且壳以空间交错的模式存在，无法进行组级选择性量化。参数壳仅在训练数据为标记级模块化的地方形成。
英文摘要
Do large language models contain domain-specific parametric shells: concentrated, causally necessary neuron populations whose removal selectively degrades a target domain while sparing others? We apply a uniform causal methodology across two domain granularities, three model families (1.5B to 7B parameters), and eight domains. At the academic subject level, zero neurons exceed 60\% domain selectivity across 939,008 combined FFN neurons and causal damage matrices are flat, despite domain identity being linearly decodable above 85\% accuracy. At the language and modality level, 0.65--1.14\% of neurons exceed 60\% selectivity, damage matrices are near-perfectly diagonal (ratios up to 595:1), and shell neuron sets are essentially disjoint (IoU $< 0.003$). Masking code-selective neurons reduces mathematical reasoning accuracy by 16--24 percentage points across all models; masking Spanish or Chinese neurons leaves it at or below random. Shell strength increases monotonically with scale and shells are spatially interleaved in a pattern that precludes group-level selective quantization. Parametric shells form where and only where training data was modular at the token level.
