Large Language Models (LLMs) represent one of the most significant advances in AI and natural language processing in recent years. Still, many pressing questions about their mechanisms, capabilities, and relationship to human cognition remain highly debated. This chapter aims to outline our current understanding of LLMs by discussing recent evidence on emerging capabilities and their mechanistic implementation within processing layers. We begin with a concise overview of the Transformer architecture, emphasizing how the attention mechanism enables training on massive datasets, allowing LLMs to function as generalist rather than specialized models. Next, we examine emergent LLM capabilities that appear to resemble aspects of human cognition, including symbolic reasoning, theory of mind, and deception strategies. Several studies provide evidence that LLMs can solve tasks previously thought to require human-like cognition. Other studies reveal insightful failure cases that shed light on the differences between human and LLM cognition. Alongside these findings, we review explainable AI approaches ranging from neuron activation analysis to circuit tracing. In the final section, we address current debates concerning what LLMs genuinely understand versus what they merely appear to understand. Prominent arguments against AI anthropomorphism point to the simplicity of LLM training objectives, claiming that LLM behavior is better explained by pattern memorization of training data than by genuine cognition. We argue that this standpoint is guided by misconceptions about optimization processes and cognitive capacity, and advocate for a more nuanced discussion of LLM cognition that neither dismisses the differences between humans and LLMs nor precludes the possibility of AI cognition through overly simplistic reductionist arguments.
Memorization $\neq$ Understanding: Do Large Language Models Have the Ability of Scenario Cognition?
School of Computer and Information Technology, Shanxi University(山西大学计算机与信息学院);Information Systems Technology and Design, Singapore University of Technology and Design(新加坡科技设计大学信息系统技术与设计)
Large Language Models Have Intrinsic Meta-Cognition, but Need a Good Lens
School of Computer Science and Engineering, Key Laboratory of Computer Network and Information Integration, Ministry of Education, Southeast University, China(计算机科学与工程学院、计算机网络与信息集成重点实验室、教育部、东南大学,中国);Department of Neurosurgery, Shanghai Tenth People’s Hospital, School of Clinical Medicine of Nanjing Medical University, China(神经外科科、上海第十人民医院、南京医科大学临床医学学院,中国)
ThinkNote: Enhancing Knowledge Integration and Utilization of Large Language Models via Constructivist Cognition Modeling
School of Computer Science and Engineering, Northeastern University(东北大学计算机科学与工程学院);Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系);Language Technologies Institute, Carnegie Mellon University(卡内基梅隆大学语言技术研究所)
How Deep is Love in LLMs' Hearts? Exploring Semantic Size in Human-like Cognition
Shanghai Jiao Tong University(上海交通大学);Shanghai Key Laboratory of Trusted Data Circulation and Governance in Web3(上海Web3可信数据流通与治理重点实验室);Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering(上海市教委智能交互与认知工程重点实验室)
CDT: A Comprehensive Capability Framework for Large Language Models Across Cognition, Domain, and Task
Institute of Computing and Intelligence, Harbin Institute of Technology(计算与智能学院,哈尔滨工业大学);NLP 2 CT Lab, Department of Computer and Information Science, University of Macau(计算机与信息科学系,澳门大学);School of Integrated Circuits, Zhejiang University(集成电路学院,浙江大学);State Key Lab of Smart Farm Technologies and Systems, Harbin Institute of Technology(智能农业技术与系统国家重点实验室,哈尔滨工业大学)
Artificial entrepreneurial cognition: Locating and causally steering an opportunity recognition dial inside large language models (LLMs)
Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg(卢森堡大学跨学科安全、可靠性与信任中心(SnT));Amsterdam Business School, University of Amsterdam(阿姆斯特丹大学阿姆斯特丹商学院);Stanford Graduate School of Business, Stanford University(斯坦福大学斯坦福商学院)
Identifying Quantum Structure in AI Language: Evidence for Evolutionary Convergence of Human and Artificial Cognition
Center Leo Apostel for Interdisciplinary Studies, Vrije Universiteit Brussel (VUB)(利奥·阿波斯泰尔跨学科研究中心,布鲁塞尔自由大学);Department of Economics, University of Bergamo(博洛尼亚大学经济系);Department of Humanities and Cultural Heritage (DIUM) and Centre CQSCS, University of Udine(乌迪内大学人文与文化遗产系及CQSCS中心)
CogSteer: Cognition-Inspired Selective Layer Intervention for Efficiently Steering Large Language Models
Department of Informatics, Universität Hamburg(汉堡大学信息学院);The University of Sydney(悉尼大学);Alibaba International Digital Commerce(阿里巴巴国际数字商业);Institute of Psychology, Chinese Academy of Sciences(中国科学院心理研究所)
Psychomatics -- A Multidisciplinary Framework for Understanding Artificial Minds
Catholic University of Sacred Heart(圣心天主教大学);Istituto Auxologico Italiano IRCCS(意大利奥索洛吉科研究所);University of Milano - Bicocca(米兰比可卡大学);Virtual Reality Medical Center(虚拟现实医疗中心);Interactive Media Institute(交互媒体研究所)
Metareasoning constraints couple narratives, affect and cognition
Applied Computational Psychiatry Lab(应用计算精神病学实验室);Max Planck UCL Centre for Computational Psychiatry and Ageing Research(马克斯·普朗克UCL计算精神病学与衰老研究中心);Queen Square Institute of Neurology and Mental Health(圣夸克广场神经病学与心理健康研究所);Neuroscience Department(神经科学系);Division of Psychiatry(精神病学系)
CityGPT: Empowering Urban Spatial Cognition of Large Language Models
Department of Electronic Engineering, BNRist, Tsinghua University(电子工程系、BNRist、清华大学);School of Electronic and Information Engineering, Beijing Jiaotong University(电子信息工程学院、北京交通大学);Department of Electronic Engineering, Tsinghua University(电子工程系、清华大学);Department of Urban Planning, Tsinghua University(城市规划系、清华大学)
Language Cognition and Language Computation -- Human and Machine Language Understanding
Institute of Automation, CAS(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Zhejiang University(浙江大学);Zhejiang Lab(之江实验室);Institute of Psychology(中国科学院心理研究所)