提示学习还是微调？探索大型语言模型在因果图验证中的应用
Prompting or Fine-tuning? Exploring Large Language Models for Causal Graph Validation
AI总结：
本研究针对传统统计因果发现方法生成的因果图需人工评估的问题，比较基于提示学习（零样本、少样本）和微调语言模型两种方法，发现微调模型在生物医学和通用领域数据集上F1分数最多提升20.5分，为因果图验证提供新方案。
摘要：
本研究探索大型语言模型（LLMs）评估传统统计因果发现方法生成的因果图中因果关系的能力——该任务传统上依赖人类领域专家的手动评估。为填补因果关系评估的这一空白，研究使用LLMs通过判断能否从文本语境中推断变量对之间的因果联系来评估因果关系。研究比较了两种方法：（1）基于提示学习的零样本和少样本因果推理方法，（2）针对因果关系预测任务微调语言模型。尽管基于提示的LLMs在多种自然语言处理（NLP）任务中展现出通用性，但在生物医学和通用领域数据集上的实验表明，微调模型持续优于前者，F1分数最多提升20.5分——即使使用更小参数的语言模型也是如此。这些发现为两种因果图评估方法的优势和局限性提供了有价值的见解。
英文摘要：
This study explores the capability of Large Language Models (LLMs) to evaluate causality in causal graphs generated by conventional statistical causal discovery methods-a task traditionally reliant on manual assessment by human subject matter experts. To bridge this gap in causality assessment, LLMs are employed to evaluate the causal relationships by determining whether a causal connection between variable pairs can be inferred from textual context. Our study compares two approaches: (1) prompting-based method for zero-shot and few-shot causal inference and, (2) fine-tuning language models for the causal relation prediction task. While prompt-based LLMs have demonstrated versatility across various NLP tasks, our experiments on biomedical and general-domain datasets show that fine-tuned models consistently outperform them, achieving up to a 20.5-point improvement in F1 score-even when using smaller-parameter language models. These findings provide valuable insights into the strengths and limitations of both approaches for causal graph evaluation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
