arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.01030cs.AI
收藏

揭示大型语言模型(LLM)中类人表征的计算要素

Uncovering the Computational Ingredients of Human-Like Representations in LLMs

University of Wisconsin–Madison(威斯康星大学麦迪逊分校);Stanford University(斯坦福大学)

Zach Studdiford, Timothy T. Rogers, Kushin Mukherjee, Siddharth Suresh

AI总结:

该研究通过对75余个模型开展 triplet 相似度任务,发现指令微调与大注意力头维度是LLM类人概念表征对齐的关键要素,明确了现有基准无法完全衡量人与模型的对齐程度。

摘要:

人类将多样感知与语言输入转化为结构化行为的能力,被认为依赖于对概念的鲁棒表征学习。基于Transformer的大型语言模型(LLM)的快速发展,已涌现出多种与模型构建相关的计算要素,包括架构、微调方法、训练数据集等,但目前尚不清楚哪些要素对开发类人概念表征最为关键。此外,多数现有基准不适用于测量表征对齐,使得LLM在这些基准上的得分无法可靠评估其作为认知模型的进展。我们通过在 triplet 相似度任务上评估超过75个模型,解决了这些局限;该任务是认知科学中测量概念表征的成熟方法,使用来自 THINGS 数据库的概念。我们发现,指令微调与更大的注意力头维度是人类对齐的最强预测因素,而激活函数选择、多模态预训练及参数规模对对齐的影响有限。对齐得分与现有基准得分的相关性显示,虽然部分基准(如 BigBenchHard)比其他基准(如 MUSR)更能捕捉表征对齐,但没有一个基准能完全解释人与模型对齐的方差,证明了它们的不足。综上,我们的发现突出了推进LLM作为人类概念表征模型的关键计算要素,并填补了LLM评估中的一个关键空白。

英文摘要:

The human ability to translate diverse perceptual and linguistic inputs into structured behavior has been thought to rest on learning robust representations of concepts. The rapid advancement of transformer-based large language models (LLMs) has surfaced a diversity of computational ingredients relevant for model building - architectures, fine-tuning methods, and training datasets among others - yet it remains unclear which are most crucial for developing human-like conceptual representations. Further, most current benchmarks are ill-suited to measuring representational alignment, making LLMs' scores on them unreliable for assessing whether they are progressing as cognitive models. We address these limitations by evaluating over 75 models on a triplet similarity task, a method well established in cognitive science for measuring conceptual representations, using concepts from the THINGS database. We find that instruction fine-tuning and larger attention head dimensionality are among the strongest predictors of human alignment, while activation function choice, multimodal pretraining, and parameter size have limited influence on alignment. Correlations between alignment scores and existing benchmark scores reveal that while some benchmarks (e.g., BigBenchHard) better capture representational alignment than others (e.g., MUSR), none fully accounts for the variance in human-model alignment, demonstrating their insufficiency. Taken together, our findings highlight key computational ingredients for advancing LLMs as models of human conceptual representation and address a key gap in LLM evaluation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑