Numerous benchmarks aim to evaluate the capabilities of Large Language Models (LLMs) for causal inference and reasoning. However, many of them can likely be solved through the retrieval of domain knowledge, questioning whether they achieve their purpose. In this review, we present a comprehensive overview of LLM benchmarks for causality. We highlight how recent benchmarks move towards a more thorough definition of causal reasoning by incorporating interventional or counterfactual reasoning. We derive a set of criteria that a useful benchmark or set of benchmarks should aim to satisfy. We hope this work will pave the way towards a general framework for the assessment of causal understanding in LLMs and the design of novel benchmarks.
Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?
Intelligent Game and Decision Lab（智能游戏与决策实验室）；National University of Defense Technology（国防科技大学）；University of Melbourne（墨尔本大学）；University of Sydney（悉尼大学）；Hong Kong Baptist University（香港 Baptist 大学）
Causal Reasoning Favors Encoders: On The Limits of Decoder-Only Models
SIRE, IIT Delhi（IIT德里SIRE）；Robert Bosch GmbH, India（印度罗伯特博世集团）；IIIT Hyderabad（海得拉巴IIIT）；IISER Kolkata（科钦IISER）；Microsoft and the University of York（微软和约克大学）
Synergizing Large Language Models and Theory for Human-like Causal Reasoning
Center for Data Science, AAIS, Peking University（数据科学中心，AAIS，北京大学）；Wangxuan Institute of Computer Technology, Peking University（王轩计算机技术研究所，北京大学）；Tsinghua University（清华大学）；State Key Laboratory of General Artificial Intelligence（通用人工智能国家重点实验室）
CLadder: Assessing Causal Reasoning in Language Models
MPI for Intelligent Systems, Tübingen（马克斯·普朗克智能系统研究所，蒂宾根）；ETH Zürich（苏黎世联邦理工学院）；IIT Kharagpur（印度理工学院卡拉格普尔分校）；University of Hong Kong（香港大学）；University of Washington（华盛顿大学）
METER: Evaluating Multi-Level Contextual Causal Reasoning in Large Language Models
College of Computer Science, Sichuan University（四川大学计算机学院）；Institute of Data Science, National University of Singapore（新加坡国立大学数据科学研究所）；Engineering Research Center of Machine Learning and Industry Intelligence, Ministry of Education, China（中国教育部机器学习与产业智能工程研究中心）
Eliciting Causal Abilities in Large Language Models for Reasoning Tasks
BNU-HKBU United International College（北师港浸大）；Hong Kong Baptist University（香港浸会大学）；Beijing Normal University at Zhuhai（北京师范大学珠海校区）；The Chinese University of Hong Kong（香港中文大学）
Inducing Causal World Models in LLMs for Zero-Shot Physical Reasoning
Department of Electrical Engineering（电气工程系）；Indian Institute of Technology Bombay（印度理工学院孟买分校）；Department of Computer Science and Automation（计算机科学与自动化系）；Indian Institute of Science（印度科学研究所）；Department of Computer Science（计算机科学系）；San Francisco State University（旧金山州立大学）；University of Lagos（拉各斯大学）；Faculty of Mathematics, Informatics, and Mechanics（数学、信息学与力学学院）；University of Warsaw（华沙大学）
SemEval-2026 Task 12: Abductive Event Reasoning: Towards Real-World Event Causal Inference for Large Language Models
The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China（认知与决策智能复杂系统重点实验室，自动化研究所，中国科学院，北京，中国）；School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China（人工智能学院，中国科学院大学，北京，中国）
Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoning
Tianjin University（天津大学）；City University of Hong Kong（香港城市大学）；University College London（伦敦大学学院）；Peking University（北京大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；University of Bristol（布里斯托大学）
Towards Generalizable Reasoning: Group Causal Counterfactual Policy Optimization for LLM Reasoning
Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Wangxuan Institute of Computer Technology, Peking University（北京大学王宣计算机技术研究所）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
ChainReaction: Causal Chain-Guided Reasoning for Modular and Explainable Causal-Why Video Question Answering
Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore（新加坡高性能计算研究所，科技研究局）；Centre for Frontier AI Research, Agency for Science, Technology and Research, Singapore（科技研究局前沿人工智能研究中心，新加坡）；College of Computing and Data Science, Nanyang Technological University, Singapore（新加坡南洋理工大学计算与数据科学学院）
