发表机构
VinUniversity; Nanyang Technological University; University of Science and Technology of China; Monash University（VinUniversity; 南洋理工大学; 中国科学技术大学; 莫纳什大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究针对Gemma 2和Gemma 3，复现并扩展SAE特征发现方法，经因果验证发现多数跨场景循环SAE特征影响有限，仅特定特征可稳定影响多语言翻译的COMET分数，识别出与语言无关的翻译启动方向。
AI 中文摘要
稀疏自编码器（Sparse Autoencoder, SAE）特征越来越多地用于解释和引导语言模型的行为，但目前仍不清楚在一种语言语境中发现的特征，在处理另一种语言的提示时是否发挥相同的因果作用。我们利用翻译启动特征（Wu等人，2026）研究这一问题。我们在Gemma 2中复现了Wu等人的SAE特征发现方法，并将其扩展到提示语言、源语言和目标语言各不相同的多语言场景。随后，我们通过在推理过程中放大或消融这些特征的激活值，测试在不同场景中循环出现的特征是否会影响翻译行为。我们还检验了该方法是否可应用于Gemma 3。在两个模型中，我们都观察到相同的结果：尽管我们可以找到20多个在所有发现场景中频繁激活的特征，但因果验证显示，几乎所有特征的影响都很小或不一致。相比之下，有一个特征——Gemma 2的（L10, 5717）和Gemma 3的（L20, 2456）——在23种语言场景中，放大时能持续提高COMET分数，消融时则会降低该分数。这些结果表明，特征循环可能夸大了跨语言迁移，同时在Gemma 2和Gemma 3中识别出了一种与语言无关的翻译启动方向。
英文摘要
Sparse autoencoder (SAE) features are increasingly used to explain and steer language-model behavior, but it remains unclear whether a feature found in one language context plays the same causal role when processing prompts in another language. We study this question using translation-initiation features (Wu et al., 2026). We reproduce the SAE feature discovery method from Wu et al. in Gemma 2 and extend it to multilingual settings that vary prompt language, source language, and target language. We then test whether features that recur across settings affect translation behavior by amplifying or ablating their activations during inference. We also examine whether the method can be applied to Gemma 3. In both models, we observe an identical finding: although we can find more than 20 features that activate frequently across all discovery settings, causal validation shows that nearly all have small or inconsistent effects. In contrast, one feature -- Gemma 2's (L10, 5717) and Gemma 3's (L20, 2456) -- consistently improves COMET scores when amplified and degrades them when ablated across 23 language settings. These results show that feature recurrence can overstate cross-lingual transfer, while identifying a language-agnostic translation-initiation direction in Gemma 2 and Gemma 3.
CommentsAccepted to BlackboxNLP 2026 Special Track
