arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.00628cs.LGcs.AIcs.CL
收藏

从关联到激活:在大语言模型中比较行为与隐藏状态语义几何

From Associations to Activations: Comparing Behavioral and Hidden-State Semantic Geometry in LLMs

Computational Modelling Lab, Humboldt-Universität zu Berlin, Berlin, Germany(洪堡-柏林大学计算建模实验室);Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany(柏林自由大学教育与心理学系);Department for AI in Society, Science, and Technology, Zuse Institute Berlin, Germany(柏林祖克研究所人工智能与社会、科学与技术部门);Institute of Mathematics, Technische Universitat Berlin, Germany(柏林技术大学数学研究所)

Louis Schiekiera, Max Zimmer, Christophe Roux, Sebastian Pokutta, Fritz Günther

AI总结:

研究比较了大语言模型中行为与隐藏状态的语义几何,发现强制选择行为能更准确地反映隐藏状态几何,且行为数据能预测未见隐藏状态相似性。

摘要:

我们研究了大语言模型的隐藏状态几何结构能否从心理语言学实验中的行为中恢复。在八个经过指令微调的Transformer模型上,我们运行了两种实验范式——基于相似性的强制选择和自由联想——在共享的5000词词汇上进行,收集了1750万+次试验以构建基于行为的相似性矩阵。使用表征相似性分析,我们将行为几何与逐层隐藏状态相似性进行比较,并与FastText、BERT和跨模型共识进行基准测试。我们发现,强制选择行为在很大程度上与隐藏状态几何对齐,而自由联想则不如此。在留出词回归中,行为相似性(尤其是强制选择)预测了未见隐藏状态相似性,超越了词汇基线和跨模型共识,表明仅基于行为的测量保留了关于内部语义几何的可恢复信息。最后,我们讨论了行为任务揭示隐藏认知状态的能力。

英文摘要:

We investigate the extent to which an LLM's hidden-state geometry can be recovered from its behavior in psycholinguistic experiments. Across eight instruction-tuned transformer models, we run two experimental paradigms -- similarity-based forced choice and free association -- over a shared 5,000-word vocabulary, collecting 17.5M+ trials to build behavior-based similarity matrices. Using representational similarity analysis, we compare behavioral geometries to layerwise hidden-state similarity and benchmark against FastText, BERT, and cross-model consensus. We find that forced-choice behavior aligns substantially more with hidden-state geometry than free association. In a held-out-words regression, behavioral similarity (especially forced choice) predicts unseen hidden-state similarities beyond lexical baselines and cross-model consensus, indicating that behavior-only measurements retain recoverable information about internal semantic geometry. Finally, we discuss implications for the ability of behavioral tasks to uncover hidden cognitive states.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑