发表机构
Université Paris 8 Vincennes–Saint-Denis; University of Geneva; Athena Research Center（巴黎第八大学（万塞纳-圣但尼）; 日内瓦大学; 雅典娜研究中心）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究通过对希腊语微调的大语言模型开展跨层泛化分析，发现Transformer后期层会定向重编码句法信息，为人类脑电解码研究提供了可验证预测。
AI 中文摘要
探测研究已证实，Transformer的早期和中间层可解码句法信息，但后期层中该信息的变化却鲜为人知。我们对三个经希腊语微调的大型语言模型开展跨层泛化分析，采用严格控制的最小对句进行评估：现代希腊语的宾语关系结构中，规范语序（主-谓-宾，SVO）与非规范语序（谓-主-宾，VSO）仅在从句内词序上存在差异，却保留命题意义。当在后期层（20-31）训练的探测模型分别在每个早期层上测试时，其迁移效果低于随机水平（经聚类校正，p<0.01），将99.3%的非规范句子分类为规范句子。探测模型系数在第22层附近符号反转，表明是向规范形式的定向重编码，而非简单的信息丢失。这些发现揭示了Transformer后期层中超出已确立的句法可解码性下降的表征格式变化，并为使用相同刺激的人类EEG和MEG解码研究提供了可直接验证的预测。代码和刺激已在OSF上公开。
英文摘要
Probing studies have established that syntactic information is decodable in early and middle transformer layers, but what happens to that information in later layers remains poorly understood. We apply a cross-layer generalisation analysis to three Greek-tuned large language models evaluated on tightly controlled minimal pairs: object-relative constructions in Modern Greek, where canonical (Subject-Verb-Object; SVO) and non-canonical (Verb-Subject-Object; VSO) orders differ only in within-clause word order, while preserving propositional meaning. When a probe trained on late layers (20-31) is tested on each early layer individually, it produces below-chance transfer (cluster-corrected, p<0.01), classifying 99.3% of non-canonical sentences as canonical. Probe coefficients reverse sign around layer 22, indicating a directional recoding toward the canonical form rather than simple information loss. These findings characterise a representational format change in late transformer layers that goes beyond the well-established decline in syntactic decodability, and they generate a directly testable prediction for human EEG and MEG decoding studies using the same stimuli. Code and stimuli are publicly available on OSF.
Comments10 pages, 3 main figures, 2 appendices. Code and stimuli: https://osf.io/5d3w8/
