arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.07539q-bio.NCcs.CL
收藏

训练驱动的表征几何模块化预测语言模型中的脑对齐

Training-Driven Representational Geometry Modularization Predicts Brain Alignment in Language Models

School of Biomedical Engineering, Tsinghua University, Beijing, China(生物医学工程学院,清华大学,北京,中国);Tsinghua Laboratory of Brain and Intelligence, Tsinghua University, Beijing, China(脑与智能实验室,清华大学,北京,中国);School of Basic Medical Sciences, Tsinghua University, Beijing, China(基础医学学院,清华大学,北京,中国);Department of Psychological and Cognitive Sciences, Tsinghua University, Beijing, China(心理学与认知科学系,清华大学,北京,中国)

Yixuan Liu, Zhiyuan Ma, Likai Tang, Runmin Gan, Xinche Zhang, Jinhao Li, Chao Xie, Sen Song

AI总结:

本研究通过训练驱动的表征几何模块化,发现语言模型中的低复杂度模块能更准确预测人类大脑语言网络活动,揭示了表征平滑对神经样语言处理的作用。

摘要:

大型语言模型(LLMs)如何与人类语言的神经表征和计算对齐,是认知科学中的核心问题。通过将表征几何作为机制视角,我们通过跟踪Pythia(70M-1B)训练过程中的熵、曲率和fMRI编码分数来解决这一问题。我们识别出一种几何模块化,其中层自我组织成稳定的低复杂度和高复杂度集群。低复杂度模块,其特征为减少的熵和曲率,始终更好地预测人类语言网络活动。这种对齐遵循异质的空间-时间轨迹:在时间区域(AntTemp,PostTemp)中快速且稳定,但在额叶区域(IFG,IFGorb)中延迟且动态。关键的是,减少曲率在控制训练进度后仍是一个稳健的模型-脑对齐预测因素,这种效果随着模型规模的增加而增强。这些结果将训练驱动的几何重组与时间-额叶功能专业化联系起来,表明表征平滑促进了神经样的语言处理。

英文摘要:

How large language models (LLMs) align with the neural representation and computation of human language is a central question in cognitive science. Using representational geometry as a mechanistic lens, we addressed this by tracking entropy, curvature, and fMRI encoding scores throughout Pythia (70M-1B) training. We identified a geometric modularization where layers self-organize into stable low- and high-complexity clusters. The low-complexity module, characterized by reduced entropy and curvature, consistently better predicted human language network activity. This alignment followed heterogeneous spatial-temporal trajectories: rapid and stable in temporal regions (AntTemp, PostTemp), but delayed and dynamic in frontal areas (IFG, IFGorb). Crucially, reduced curvature remained a robust predictor of model-brain alignment even after controlling for training progress, an effect that strengthened with model scale. These results links training-driven geometric reorganization to temporal-frontal functional specialization, suggesting that representational smoothing facilitates neural-like linguistic processing.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑