Recent studies show evidence for emergent cognitive abilities in Large Pre-trained Language Models (PLMs). The increasing cognitive alignment of these models has made them candidates for cognitive science theories. Prior research into the emergent cognitive abilities of PLMs has largely been path-independent to model training, i.e., has focused on the final model weights and not the intermediate steps. However, building plausible models of human cognition using PLMs would benefit from considering the developmental alignment of their performance during training to the trajectories of children's thinking. Guided by psychometric tests of human intelligence, we choose four sets of tasks to investigate the alignment of ten popular families of PLMs and evaluate their available intermediate and final training steps. These tasks are Numerical ability, Linguistic abilities, Conceptual understanding, and Fluid reasoning. We find a striking regularity: regardless of model size, the developmental trajectories of PLMs consistently exhibit a window of maximal alignment to human cognitive development. Before that window, training appears to endow "blank slate" models with the requisite structure to be poised to rapidly learn from experience. After that window, training appears to serve the engineering goal of reducing loss but not the scientific goal of increasing alignment with human cognition.
Large Language Models and Cognitive Science: A Comprehensive Review of Similarities, Differences, and Challenges
Kyoto University(京都大学);Indiana University(印第安纳大学);National Taiwan Normal University(台湾师范大学);Georgia Institute of Technology(佐治亚理工学院);Hong Kong University of Science(香港科学大学);The University of Texas at Dallas(德克萨斯大学达拉斯分校);University of Wisconsin-Madison(威斯康星大学麦迪逊分校);Cornell University(康奈尔大学);University of Liverpool, UK(利物浦大学);University of Edinburgh, UK(爱丁堡大学);Zhejiang University(浙江大学);Purdue University(普渡大学);Emory University(埃默里大学);West China Biomedical Big Data Center, West China Hospital, Sichuan University(四川大学西昌生物大数据中心、西昌医院)
Text Production and Comprehension by Human and Artificial Intelligence: Interdisciplinary Workshop Report
Nottingham Trent University(诺丁汉特伦特大学);Cornell University(康奈尔大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of the Saarland(萨尔大学);University of Texas at Austin(德克萨斯大学奥斯汀分校);University of Edinburgh(爱丁堡大学);Tuebingen University(图宾根大学);Montclair State University(蒙特克莱尔州立大学);Carnegie Mellon University(卡内基梅隆大学);Georgia Institute of Technology(佐治亚理工学院);Iowa State University(爱荷华州立大学);Embry-Riddle Aeronautical University(安柏瑞德航空航天大学);Midland University(米德兰大学);University of Chicago(芝加哥大学);University of Iowa(爱荷华大学)
PerSo: Investigating Facets of LLMs' Social Intelligence in Real-World Persuasion Scenarios
School of Electrical and Computer Engineering, College of Engineering, University of Tehran(德黑兰大学工程学院电气与计算机工程学院);Cardiff University(卡迪夫大学);Institute for Research in Fundamental Sciences (IPM)(基础科学研究所);Tehran Institute for Advanced Studies, Khatam University(哈塔姆大学德黑兰高级研究院)
Integration of cognitive tasks into artificial general intelligence test for large models
Southern University of Science and Technology(南方科技大学);Shanghai AI Laboratory(上海人工智能实验室);Tencent AI lab(腾讯AI实验室);iFLYTEK AI Research(科大讯飞人工智能研究院);Beijing Institute of Technology(北京理工大学);Peking University(北京大学);University of Macau(澳门大学);Tsinghua University(清华大学)
M3GIA: A Cognition Inspired Multilingual and Multimodal General Intelligence Ability Benchmark
AutoLab, Westlake University(西湖大学AutoLab);Alibaba Group(阿里巴巴集团);Zhejiang University(浙江大学);Key Laboratory of Behavioral Science, Institute of Psychology, CAS(中国科学院心理研究所行为科学重点实验室);Key Laboratory of AI Safety, Institute of Computing Technology, CAS(中国科学院计算技术研究所人工智能安全重点实验室)
Human-AI Coevolution Dynamics: A Formal Theory of Social Intelligence Emergence Through Long-Term Interaction
School of Information and Electronics, Beijing Institute of Technology(信息与电子学院,北京理工大学);Institute of Scientific and Technical Research on Archives, Beijing(档案科学与技术研究所,北京);China Electronics Engineering Design Institute Co., Ltd.(中国电子工程设计院有限公司)