Piaget's Theory of Cognitive Development (PTC) posits that the development of cognitive levels forms the foundation for human learning across various abilities. As Large Language Models (LLMs) have recently shown remarkable abilities across a wide variety of tasks, we are curious about the cognitive levels of current LLMs: to what extent they have developed and how this development has been achieved. To this end, we construct a benchmark CogLM (Cognitive Ability Evaluation for Language Model) based on PTC to assess the cognitive levels of LLMs. CogLM comprises 1,220 questions spanning 10 cognitive abilities crafted by more than 20 human experts, providing a comprehensive testbed for the cognitive levels of LLMs. Through extensive experiments across multiple mainstream LLMs with CogLM, we find that: (1) In our testing framework, advanced LLMs (such as GPT-4) have demonstrated human-like cognitive abilities, comparable to those of a 20-year-old human. (2) The parameter size and optimization objective are two key factors affecting the cognitive levels of LLMs. (3) The performance on downstream tasks is positively correlated with the level of cognitive abilities. These findings fill the gap in research on the cognitive abilities of LLMs, tracing the development of LLMs from a cognitive perspective and guiding the future direction of their evolution.
Integrating LLM, EEG, and Eye-Tracking Biomarker Analysis for Word-Level Neural State Classification in Semantic Inference Reading Comprehension
Brown University（布朗大学）；University of California San Diego（加利福尼亚大学圣迭戈分校）；Yale University（耶鲁大学）；University of California Los Angeles（加利福尼亚大学洛杉矶分校）；National Central University（国立中央大学）
Spontaneous Speech Variables for Evaluating LLMs Cognitive Plausibility
Academia Sinica Institute of Linguistics（学术院语言研究所）；CNRS & MEAE CEFC（国家科学研究中心与外交与国际合作部CEFC）；Graduate Institute of Linguistics National Taiwan University（国立台湾大学语言研究所）；Department of CSIE National Taiwan University（国立台湾大学计算机科学系）
Detecting Alzheimer's Disease by estimating attention and elicitation path through the alignment of spoken picture descriptions with the picture prompt
University of Sheffield（谢菲尔德大学）；Royal Hallamshire Hospital（皇家哈勒姆医院）；Sheffield Institute for Translational Neuroscience（谢菲尔德转化神经科学研究所）
