arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2407.01047cs.CL
收藏

预训练语言模型中认知智能的发展

Development of Cognitive Intelligence in Pre-trained Language Models

Georgia Institute of Technology(佐治亚理工学院)

Raj Sanjay Shah, Khushi Bhardwaj, Sashank Varma

AI总结:

该研究借鉴人类智力心理测量测试,通过四类任务探究10类预训练语言模型训练各阶段与人类认知发展的对齐情况,发现存在固定的最高对齐窗口期,为利用PLMs构建人类认知模型提供了发展轨迹层面的依据。

摘要:

近期研究表明,大型预训练语言模型(PLMs)已显现出涌现的认知能力证据。这类模型与人类认知的对齐程度不断提升,使其成为认知科学理论的潜在研究载体。此前针对PLMs涌现认知能力的研究大多与模型训练路径无关,即仅关注最终模型权重,而非训练的中间阶段。然而,若要利用PLMs构建符合人类认知规律的合理模型,需考量其训练过程中的性能发展轨迹与儿童思维发展轨迹的对齐情况。\n受人类智力心理测量测试的启发,我们选取四组任务(数值能力、语言能力、概念理解、流体推理),针对10个主流PLMs家族展开研究,评估其可获取的中间训练阶段与最终训练阶段的表现。我们发现了一项显著规律:无论模型规模大小,PLMs的发展轨迹始终存在一个与人类认知发展对齐程度最高的窗口期。在该窗口期之前,训练似乎为“白板”模型赋予了必要的结构,使其做好从经验中快速学习的准备;在该窗口期之后,训练更多服务于降低损失的工程目标,而非提升与人类认知对齐度的科学目标。

英文摘要:

Recent studies show evidence for emergent cognitive abilities in Large Pre-trained Language Models (PLMs). The increasing cognitive alignment of these models has made them candidates for cognitive science theories. Prior research into the emergent cognitive abilities of PLMs has largely been path-independent to model training, i.e., has focused on the final model weights and not the intermediate steps. However, building plausible models of human cognition using PLMs would benefit from considering the developmental alignment of their performance during training to the trajectories of children's thinking. Guided by psychometric tests of human intelligence, we choose four sets of tasks to investigate the alignment of ten popular families of PLMs and evaluate their available intermediate and final training steps. These tasks are Numerical ability, Linguistic abilities, Conceptual understanding, and Fluid reasoning. We find a striking regularity: regardless of model size, the developmental trajectories of PLMs consistently exhibit a window of maximal alignment to human cognitive development. Before that window, training appears to endow "blank slate" models with the requisite structure to be poised to rapidly learn from experience. After that window, training appears to serve the engineering goal of reducing loss but not the scientific goal of increasing alignment with human cognition.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑