Analogical reasoning is at the core of human cognition, serving as an important foundation for a variety of intellectual activities. While prior work has shown that LLMs can represent task patterns and surface-level concepts, it remains unclear whether these models can encode high-level relational concepts and apply them to novel situations through structured comparisons. In this work, we explore this fundamental aspect using proportional and story analogies, and identify three key findings. First, LLMs effectively encode the underlying relationships between analogous entities; both attributive and relational information propagate through mid-upper layers in correct cases, whereas reasoning failures reflect missing relational information within these layers. Second, unlike humans, LLMs often struggle not only when relational information is missing, but also when attempting to apply it to new entities. In such cases, strategically patching hidden representations at critical token positions can facilitate information transfer to a certain extent. Lastly, successful analogical reasoning in LLMs is marked by strong structural alignment between analogous situations, whereas failures often reflect degraded or misplaced alignment. Overall, our findings reveal that LLMs exhibit emerging but limited capabilities in encoding and applying high-level relational concepts, highlighting both parallels and gaps with human cognition.
Information Sciences Institute, University of Southern California（南加州大学信息科学研究所）；Department of Computer Science, University of Southern California（南加州大学计算机科学系）；Department of Computer Science, Vrije Universiteit Amsterdam（阿姆斯特丹自由大学计算机科学系）；Computational Linguistics & Text Mining Lab, Vrije Universiteit Amsterdam（阿姆斯特丹自由大学计算语言学与文本挖掘实验室）
ANALOGICAL -- A Novel Benchmark for Long Text Analogy Evaluation in Large Language Models
AI Institute, University of South Carolina（南卡罗来纳大学人工智能研究所）；Nirma University（尼玛大学）；BITS Pilani（博拉理工学院皮拉尼校区）；IIIT Delhi（德里印度信息技术学院）；Amazon AI（亚马逊人工智能）；Stanford（斯坦福大学）；Amazon（亚马逊）
ANALOGYKB: Unlocking Analogical Reasoning of Language Models with A Million-scale Knowledge Base
School of Data Science, Fudan University（复旦大学数据科学学院）；Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University（复旦大学计算机科学学院上海市数据科学重点实验室）；ByteDance AI Lab（字节跳动人工智能实验室）；East China Normal University（华东师范大学）
Modelling Analogies and Analogical Reasoning: Connecting Cognitive Science Theory and NLP Research
Computation, Cognition & Language Group, Idiap Research Institute（计算、认知与语言组，Idiap研究所）；NLP Lab, EPFL（NLP实验室，EPFL）；Psychological Methods, University of Amsterdam（心理方法，阿姆斯特丹大学）；Faculty of Communication, Culture and Society and Faculty of Informatics Università della Svizzera italiana（传播、文化与社会学院和信息学院，瑞士大学）
E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoning
Fudan University（复旦大学）；Brain Technologies, Inc.（脑技术公司）；South China University of Technology（华南理工大学）；ByteDance AI Lab（字节跳动人工智能实验室）；University of California Santa Barbara（加州大学圣巴巴拉分校）；Fudan-Aishu Cognitive Intelligence Joint Research Center（复旦-爱数认知智能联合研究中心）
KiVA: Kid-inspired Visual Analogies for Testing Large Multimodal Models
University of California, Berkeley（加州大学伯克利分校）；Boston University（波士顿大学）；Google DeepMind（谷歌DeepMind）；Toyota Technological Institute at Chicago（芝加哥丰田技术研究所）
