arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.18607cs.AIcs.HCcs.LG
收藏

利用GPT-4引导因果机器学习

Using GPT-4 to guide causal machine learning

Queen Mary University of London(伦敦玛丽女王大学)

Anthony C. Constantinou, Neville K. Kitson, Alessio Zanga

AI总结:

本研究评估仅提供变量标签时GPT-4的因果识别能力,发现其生成的因果图准确率优于领域专家知识图谱和因果机器学习,且二者结合可提升因果机器学习的因果发现效果,弥补其生成的因果图违背常识的缺陷。

摘要:

自面向公众推出以来,ChatGPT已产生了前所未有的影响。部分专家称赞人工智能的进步并强调其潜在风险,也有学者对大语言模型(Large Language Models, LLMs)的准确性和实用性提出批评。本文聚焦大语言模型的因果关系识别能力。研究以成熟的GPT-4 (Turbo)为对象,在最严格的条件下评估其性能:仅向模型提供变量标签、不附加任何人类给出的其他上下文,隔离其因果关系推断能力,以此明确仅给定标签信息时该模型可达到的最低效果水平。研究结果显示,问卷参与者认为在所有评估类别中,GPT-4生成的因果图准确率最高,领域专家构建的知识图谱紧随其后,因果机器学习(Machine Learning, ML)的表现则远远落后。基于上述结果,本文指出因果机器学习的一项重要局限:其生成的因果图往往违背常识,会影响用户对这类模型的信任度。但研究同时证明,将GPT-4与因果机器学习结合可突破这一局限:与仅由因果机器学习从真实数据中学习得到的图结构相比,二者结合后从真实数据中学习得到的图结构,与领域专家识别的结构契合度更高。总体而言,研究结果表明,尽管GPT-4并非专为因果推理设计,但它仍可作为因果表征的有效工具,能够提升专门用于因果发现的因果机器学习算法的因果发现过程效果。

英文摘要:

Since its introduction to the public, ChatGPT has had an unprecedented impact. While some experts praised AI advancements and highlighted their potential risks, others have been critical about the accuracy and usefulness of Large Language Models (LLMs). In this paper, we are interested in the ability of LLMs to identify causal relationships. We focus on the well-established GPT-4 (Turbo) and evaluate its performance under the most restrictive conditions, by isolating its ability to infer causal relationships based solely on the variable labels without being given any other context by humans, demonstrating the minimum level of effectiveness one can expect when it is provided with label-only information. We show that questionnaire participants judge the GPT-4 graphs as the most accurate in the evaluated categories, closely followed by knowledge graphs constructed by domain experts, with causal Machine Learning (ML) far behind. We use these results to highlight the important limitation of causal ML, which often produces causal graphs that violate common sense, affecting trust in them. However, we show that pairing GPT-4 with causal ML overcomes this limitation, resulting in graphical structures learnt from real data that align more closely with those identified by domain experts, compared to structures learnt by causal ML alone. Overall, our findings suggest that despite GPT-4 not being explicitly designed to reason causally, it can still be a valuable tool for causal representation, as it improves the causal discovery process of causal ML algorithms that are designed to do just that.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑