探究字符级Transformer在西班牙语L形词素上的表现
Probing Character-level Transformers for the Spanish L-shaped Morphome
浏览论文内容
中文总结 AI 辅助
本研究探究字符级Transformer对西班牙语L形词素的表征，发现模型编码了该词素本身，以特定于项的词汇抽象存储，可复现模式但无法如人类般泛化。
中文摘要 AI 辅助
当Transformer学习不规则形态模式时，它学到了什么？我们的测试对象是西班牙语L形词素，这是一种复杂的不规则模式，动词词干仅在第一人称单数直陈式和所有虚拟式形式中发生交替，且其成员无法通过任何音系、语义或句法特征预测。先前研究表明，字符级Transformer可复现该模式，但这些证据描述的是模型的输出，而非其表征。我们在 lemma 不相交的交叉验证设置下，对五种架构、每种架构训练的十二个模型，结合控制项和表层基线进行探究，结果显示模型编码了L形类别本身，而非仅其可见的交替模式。该编码在所有表层基线之上均可被解码，在所有形式呈现相同词干的实例中仍存在，且在交替实例上训练的探针仍可对非交替实例进行分类。该编码定位于词干选择的位置，即中间解码器的词干末尾辅音位置，且在交替形式被读取之前。此外，该编码具有特定于项的特性：模型学习的动词项远比其架构类型更重要。这些模型将该词素存储为特定于项的词汇抽象，足以复现该模式，但无法像人类那样进行泛化。
英文摘要
When a transformer learns an irregular morphological pattern, what has it learned? Our test case is the Spanish \emph{L-shaped morphome}, a complex irregular pattern in which the verb's stem alternates in exactly the first-person singular indicative and all subjunctive forms, and whose membership no phonological, semantic, or syntactic feature predicts. Prior studies have shown that character-level transformers can reproduce this pattern, but that evidence describes what models produce, not what they represent. Probing five architectures, twelve trained models each, under lemma-disjoint cross-validation with controls and surface baselines, we show that the models encode the L-shaped class itself, not just its visible alternations. It is decodable above every surface baseline, survives instances in which every form shows the same stem, and probes trained on alternating instances still classify non-alternating ones. The encoding is localized where the stem choice is made, at the stem-final consonant position of the middle decoder, before the alternant is read. And it is item-specific: which verbs a model learned matters far more than which architecture it is. The models store the morphome as an item-specific lexical abstraction, sufficient to reproduce the pattern but not to generalize it as humans do.
