arXiv 2401.001392026-05-21cs.AIcs.CLcs.LGstat.ME

增强大语言模型中的因果推理：一种用于精确微调的因果归因模型

Enhancing Causal Reasoning in Large Language Models: A Causal Attribution Model for Precision Fine-Tuning

University of California, Irvine（加州大学尔湾分校）；North Carolina State University（北卡罗来纳州立大学）；Amazon（亚马逊公司）

Hengrui Cai, Shengjie Liu, Rui Song

备注：: A Python implementation of our proposed method is available at https://github.com/ncsulsj/Causal_LLM

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出一种因果归因模型，通过精确微调提升大语言模型的可解释性和因果推理能力，展示了模型在不同领域中的因果发现任务中的有效性。

摘要：

本文介绍了一种因果归因模型，旨在通过精确微调增强大语言模型（LLMs）的可解释性并提高其因果推理能力。尽管LLMs在多种任务中表现出色，但其推理过程往往仍是一个黑箱，限制了有针对性的增强。我们提出了一种新的因果归因模型，利用“do-运算符”构建干预场景，使我们能够系统地量化LLMs因果推理过程中不同组件的贡献。通过在各种领域中进行因果发现任务来评估所提出的归因分数，我们证明了LLMs在因果发现中的有效性严重依赖于提供的上下文和领域特定知识，但也可以利用数值数据进行有限的相关性推理，而非因果性。这促使了所提出的微调LLM用于成对因果发现，有效且正确地利用了知识和数值信息。

英文摘要：

This paper introduces a causal attribution model to enhance the interpretability of large language models (LLMs) and improve their causal reasoning abilities via precise fine-tuning. Despite LLMs' proficiency in diverse tasks, their reasoning processes often remain black box, and thus restrict targeted enhancement. We propose a novel causal attribution model that utilizes "do-operators" for constructing interventional scenarios, allowing us to quantify the contribution of different components in LLMs's causal reasoning process systematically. By assessing the proposed attribution scores through causal discovery tasks across various domains, we demonstrate that LLMs' effectiveness in causal discovery heavily relies on provided context and domain-specific knowledge but can also utilize numerical data with limited calculations in correlation, not causation. This motivates the proposed fine-tuned LLM for pairwise causal discovery, effectively and correctly leveraging both knowledge and numerical information.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。