arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.10282cs.LG
收藏

线性-LLM-SCM：用于线性高斯因果模型系数提取的LLM基准测试

Linear-LLM-SCM: Benchmarking LLMs for Coefficient Elicitation in Linear-Gaussian Causal Models

Data Science and its Applications, German Research Centre for Artificial Intelligence (DFKI)（德国人工智能研究中心数据科学与应用部门）；Dept. of Computer Science, University of Kaiserslautern–Landau (RPTU)（科隆-兰道大学计算机科学系）；Digital Health - Machine Learning Research Group, Hasso Plattner Institute for Digital Engineering（哈索·普朗纳研究所数字工程学院数字健康-机器学习研究组）；Institute of Informatics, University of Munich (LMU)（慕尼黑大学信息学院）；Hasso Plattner Institute for Digital Health at Mount Sinai, Icahn School of Medicine at Mount Sinai（西奈山医学院哈索·普朗纳研究所数字健康中心）；Munich Center for Machine Learning (MCML), Germany（慕尼黑机器学习中心）

Kanta Yamaoka, Sumantrak Mukherjee, Thomas Gärtner, David Antony Selby, Stefan Konigorski, Eyke Hüllermeier, Viktor Bengs, Sebastian Josef Vollmer

备注：
[v2] Accepted at Workshop on Structured Data for Health@ICML 2026 Seoul,South Korea. 19 pages, 8 figures, preprint
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出线性-LLM-SCM框架，用于评估LLM在连续域中对线性高斯因果模型参数化的表现，揭示了LLM在定量因果推理中的局限性。

摘要：

大型语言模型（LLM）在识别定性因果关系方面显示出潜力，但其在连续域中进行定量因果推理的能力——即估计参数化函数关系的效果大小——仍缺乏深入探索。我们引入了线性-LLM-SCM，这是一个可插拔的基准测试框架，用于评估LLM在给定DAG（有向无环图）时对线性高斯结构因果模型（SCM）参数化的表现。该框架将DAG分解为局部父-子集，并提示LLM为每个节点生成回归风格的结构方程，这些方程被汇总并与其他可用的真实参数进行比较。我们的实验显示，在此类基准测试任务中存在几个挑战，例如某些模型结果中的强随机性以及通过连续域中的虚假边导致的DAG规格错误的易感性。在不同模型中，我们观察到某些设置下的系数估计存在显著差异，并且对结构和语义扰动敏感，这突显了当前LLM作为定量因果参数化工具的局限性。我们还开源了该基准测试框架，使研究人员能够轻松利用他们的DAGs和任何现成的LLM进行评估。

英文摘要：

Large language models (LLMs) have shown potential in identifying qualitative causal relations, but their ability to perform quantitative causal reasoning---estimating effect sizes that parametrize functional relationships---remains underexplored in continuous domains. We introduce Linear-LLM-SCM, a plug-and-play framework for evaluating LLMs on Linear Gaussian structural causal model parametrization when a directed acyclic graph (DAG) is given. The framework decomposes a DAG into local parent-child sets and prompts an LLM to produce a regression-style structural equation per node, which is aggregated and compared against available ground-truth parameters. Our experiments with seven real-world DAGs effect ground truth illustrate limitations of LLMs as quantitative causal parameterizers. Across most models, we observe variability in coefficient estimates and sensitivity to structural perturbations. We open-sourced the framework to further encourage the community to work on studies toward the use of LLM for causal effect elicitation in safety-critical domain, e.g., healthcare.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑