关于大型语言模型在因果发现中的可靠性研究
On the Reliability of Large Language Models for Causal Discovery
AI总结：
本研究利用OLMo和BLOOM等开源LLM，通过三个研究问题考察其在因果发现中的可靠性，发现模型对频繁出现的因果关系识别有效，但泛化能力有限，且错误因果关系和情境信息显著影响其判断。
摘要：
本研究探讨了大型语言模型（LLMs）在因果发现中的效能。利用新近可用的开源LLMs，即OLMo和BLOOM，这些模型提供了对其预训练语料库的访问权限，我们通过三个研究问题来考察LLMs如何处理因果发现。我们检验了：（i）记忆对准确因果关系预测的影响，（ii）预训练数据中错误因果关系的影响，以及（iii）影响LLMs理解因果关系的情境细微差别。我们的研究结果表明，虽然LLMs能够有效识别预训练数据中频繁出现的因果关系，但它们对新的或稀有因果关系的泛化能力有限。此外，错误因果关系的存在显著削弱了LLMs对相应正确因果关系的信心，而情境信息对LLMs辨别随机变量间因果联系的结果至关重要。
英文摘要：
This study investigates the efficacy of Large Language Models (LLMs) in causal discovery. Using newly available open-source LLMs, OLMo and BLOOM, which provide access to their pre-training corpora, we investigate how LLMs address causal discovery through three research questions. We examine: (i) the impact of memorization for accurate causal relation prediction, (ii) the influence of incorrect causal relations in pre-training data, and (iii) the contextual nuances that influence LLMs' understanding of causal relations. Our findings indicate that while LLMs are effective in recognizing causal relations that occur frequently in pre-training data, their ability to generalize to new or rare causal relations is limited. Moreover, the presence of incorrect causal relations significantly undermines the confidence of LLMs in corresponding correct causal relations, and the contextual information critically affects the outcomes of LLMs to discern causal connections between random variables.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
