自面向公众推出以来,ChatGPT已产生了前所未有的影响。部分专家称赞人工智能的进步并强调其潜在风险,也有学者对大语言模型(Large Language Models, LLMs)的准确性和实用性提出批评。本文聚焦大语言模型的因果关系识别能力。研究以成熟的GPT-4 (Turbo)为对象,在最严格的条件下评估其性能:仅向模型提供变量标签、不附加任何人类给出的其他上下文,隔离其因果关系推断能力,以此明确仅给定标签信息时该模型可达到的最低效果水平。研究结果显示,问卷参与者认为在所有评估类别中,GPT-4生成的因果图准确率最高,领域专家构建的知识图谱紧随其后,因果机器学习(Machine Learning, ML)的表现则远远落后。基于上述结果,本文指出因果机器学习的一项重要局限:其生成的因果图往往违背常识,会影响用户对这类模型的信任度。但研究同时证明,将GPT-4与因果机器学习结合可突破这一局限:与仅由因果机器学习从真实数据中学习得到的图结构相比,二者结合后从真实数据中学习得到的图结构,与领域专家识别的结构契合度更高。总体而言,研究结果表明,尽管GPT-4并非专为因果推理设计,但它仍可作为因果表征的有效工具,能够提升专门用于因果发现的因果机器学习算法的因果发现过程效果。
英文摘要:
Since its introduction to the public, ChatGPT has had an unprecedented impact. While some experts praised AI advancements and highlighted their potential risks, others have been critical about the accuracy and usefulness of Large Language Models (LLMs). In this paper, we are interested in the ability of LLMs to identify causal relationships. We focus on the well-established GPT-4 (Turbo) and evaluate its performance under the most restrictive conditions, by isolating its ability to infer causal relationships based solely on the variable labels without being given any other context by humans, demonstrating the minimum level of effectiveness one can expect when it is provided with label-only information. We show that questionnaire participants judge the GPT-4 graphs as the most accurate in the evaluated categories, closely followed by knowledge graphs constructed by domain experts, with causal Machine Learning (ML) far behind. We use these results to highlight the important limitation of causal ML, which often produces causal graphs that violate common sense, affecting trust in them. However, we show that pairing GPT-4 with causal ML overcomes this limitation, resulting in graphical structures learnt from real data that align more closely with those identified by domain experts, compared to structures learnt by causal ML alone. Overall, our findings suggest that despite GPT-4 not being explicitly designed to reason causally, it can still be a valuable tool for causal representation, as it improves the causal discovery process of causal ML algorithms that are designed to do just that.
ChainReaction: Causal Chain-Guided Reasoning for Modular and Explainable Causal-Why Video Question Answering
Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore(新加坡高性能计算研究所,科技研究局);Centre for Frontier AI Research, Agency for Science, Technology and Research, Singapore(科技研究局前沿人工智能研究中心,新加坡);College of Computing and Data Science, Nanyang Technological University, Singapore(新加坡南洋理工大学计算与数据科学学院)
Compiler-Guided Inference-Time Adaptation: Improving GPT-5 Programming Performance in Idris
Ming Hsieh Department of Electrical and Computer Engineering(明希斯部门电子与计算机工程系);Viterbi School of Engineering, University of Southern California(维特比工程学院,南加州大学)
Causal Ensemble Agent: Hierarchical Causal Discovery with LLM-guided Expert Reweighting
The University of Melbourne(墨尔本大学);MBZUAI(穆罕默德·本·扎耶德人工智能大学);Peking University(北京大学);Adelaide University(阿德莱德大学);Hong Kong Baptist University(香港浸会大学);The University of Sydney(悉尼大学);Carnegie Mellon University(卡内基梅隆大学)
Causal Graph Fuzzy LLMs: A First Introduction and Applications in Time Series Forecasting
Federal Institute of Northern Minas Gerais(巴西北矿联邦理工学院);Graduate Program in Electrical Engineering(电气工程研究生项目);Universidade Federal de Minas Gerais(伯南巴西联邦大学);Graduate Program in Computer Science(计算机科学研究生项目);Federal University of Minas Gerais(伯南巴西联邦大学);Future Lab(未来实验室)
Development of the ChatGPT, Generative Artificial Intelligence and Natural Large Language Models for Accountable Reporting and Use (CANGARU) Guidelines
University of Southern California(南加州大学);University of Oxford(牛津大学)
Information Gain-Guided Causal Intervention for Autonomous Debiasing Large Language Models
Research Center for Social Computing and Interactive Robotics(社会计算与交互机器人研究中心);Harbin Institute of Technology(哈尔滨工业大学);Beijing Academy of Artificial Intelligence(北京人工智能研究院)
On the physics of nested Markov models: a generalized probabilistic theory perspective
QICI Quantum Information and Computation Initiative, School of Computing and Data Science, The University of Hong Kong, Hong Kong SAR, China(量子信息与计算倡议,计算与数据科学学院,香港大学,香港特别行政区,中国);Institute for Interdisciplinary Information Sciences, Tsinghua University, Beijing, China(交叉信息科学研究所,清华大学,北京,中国);Shanghai Qi Zhi Institute, Shanghai, China(上海启智研究所,上海,中国);Perimeter Institute for Theoretical Physics, Waterloo, Ontario, Canada(理论物理研究所,滑铁卢,安大略,加拿大)
Com$^2$: A Causal-Guided Benchmark for Exploring Complex Commonsense Reasoning in Large Language Models
Research Center for Social Computing and Interactive Robotics(社会计算与交互机器人研究中心);Harbin Institute of Technology(哈尔滨工业大学);Institute of Trustworthy Embodied AI(可信具身人工智能研究院);Fudan University(复旦大学);Beijing Academy of Artificial Intelligence(北京人工智能研究院);Nanjing University(南京大学)
Annotation Guidelines-Based Knowledge Augmentation: Towards Enhancing Large Language Models for Educational Text Classification
National Engineering Research Center of Educational Big Data, Faculty of Artificial Intelligence in Education, Central China Normal University(国家教育大数据工程研究中心,教育人工智能学院,中央财经大学)
CausaLab: A Scalable Environment for Interactive Causal Discovery Toward AI Scientists
Tsinghua University(清华大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Carnegie Mellon University(卡内基梅隆大学);University of Chicago(芝加哥大学);Adobe