arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.09972cs.LGcs.AIcs.CV
收藏

从数据统计到特征几何:相关性如何塑造叠加

From Data Statistics to Feature Geometry: How Correlations Shape Superposition

Imperial College London(帝国理工学院伦敦分校)

Lucas Prieto, Edward Stevinson, Melih Barsbey, Tolga Birdal, Pedro A. M. Mediano

AI总结:

通过Bag-of-Words Superposition研究特征相关性对叠加的影响,揭示现实数据中叠加可产生建设性干扰和语义聚类。

摘要:

机械可解释性的一个核心思想是神经网络表示的特征数量多于其维度,将它们叠加以形成一个过完备基。这一框架具有影响力,推动了字典学习方法,如稀疏自动编码器。然而,叠加大多在理想化设置中研究,其中特征稀疏且不相关。在这些设置中,叠加通常被理解为引入干扰,必须通过几何方法最小化并被非线性如ReLU过滤掉,产生局部结构如正则多面体。我们通过引入Bag-of-Words Superposition(BOWS),一个受控环境来编码互联网文本的二进制词袋表示,展示这种账户在现实数据中是不完整的。使用BOWS,我们发现当特征相关时,干扰可以是建设性的而非仅仅是需要过滤的噪声。这通过根据共同激活模式排列特征实现,使活跃特征之间的干扰建设性,同时仍使用ReLU避免假阳性。我们证明这种排列在使用权重衰减训练的模型中更普遍,并自然产生语义聚类和循环结构,这些在真实语言模型中被观察到,但未被标准叠加图景解释。本文代码可在https://github.com/LucasPrietoAl/correlations-feature-geometry找到。

英文摘要:

A central idea in mechanistic interpretability is that neural networks represent more features than they have dimensions, arranging them in superposition to form an over-complete basis. This framing has been influential, motivating dictionary learning approaches such as sparse autoencoders. However, superposition has mostly been studied in idealized settings where features are sparse and uncorrelated. In these settings, superposition is typically understood as introducing interference that must be minimized geometrically and filtered out by non-linearities such as ReLUs, yielding local structures like regular polytopes. We show that this account is incomplete for realistic data by introducing Bag-of-Words Superposition (BOWS), a controlled setting to encode binary bag-of-words representations of internet text in superposition. Using BOWS, we find that when features are correlated, interference can be constructive rather than just noise to be filtered out. This is achieved by arranging features according to their co-activation patterns, making interference between active features constructive, while still using ReLUs to avoid false positives. We show that this kind of arrangement is more prevalent in models trained with weight decay and naturally gives rise to semantic clusters and cyclical structures which have been observed in real language models yet were not explained by the standard picture of superposition. Code for this paper can be found at https://github.com/LucasPrietoAl/correlations-feature-geometry.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑