OCDB：通过综合基准与评估框架重新审视因果发现
OCDB: Revisiting Causal Discovery with a Comprehensive Benchmark and Evaluation Framework
AI总结：
本文提出基于真实数据的开放因果发现基准OCDB及灵活评估框架，通过新指标公平比较DAG与CPDAG，揭示现有算法泛化不足并推动因果发现技术优化。
摘要：
大型语言模型（LLMs）已在多种自然语言处理任务中表现出色，但在可解释性和可信度方面仍面临挑战，限制了其在高风险领域的应用。因果发现为提升透明度和可靠性提供了一种有前景的途径。然而，当前的评估往往具有片面性，缺乏针对可解释性表现的评估。此外，这些评估依赖合成数据，缺少对真实世界数据集的全面评估。这些问题导致一些有潜力的方法可能被忽视。为解决这些问题，我们提出了一个灵活的评估框架，其中包含用于评估因果结构和因果效应差异的指标，这些关键属性有助于提高LLMs的可解释性。我们引入了基于真实数据的开放因果发现基准（Open Causal Discovery Benchmark，OCDB），以促进公平比较并推动算法优化。此外，我们的新指标考虑了无向边，从而能够在有向无环图（DAGs）与完全部分有向无环图（CPDAGs）之间进行公平比较。实验结果表明，现有算法在真实数据上的泛化能力存在显著不足，凸显了性能提升的潜力以及我们的框架在推进因果发现技术方面的重要性。
英文摘要：
Large language models (LLMs) have excelled in various natural language processing tasks, but challenges in interpretability and trustworthiness persist, limiting their use in high-stakes fields. Causal discovery offers a promising approach to improve transparency and reliability. However, current evaluations are often one-sided and lack assessments focused on interpretability performance. Additionally, these evaluations rely on synthetic data and lack comprehensive assessments of real-world datasets. These lead to promising methods potentially being overlooked. To address these issues, we propose a flexible evaluation framework with metrics for evaluating differences in causal structures and causal effects, which are crucial attributes that help improve the interpretability of LLMs. We introduce the Open Causal Discovery Benchmark (OCDB), based on real data, to promote fair comparisons and drive optimization of algorithms. Additionally, our new metrics account for undirected edges, enabling fair comparisons between Directed Acyclic Graphs (DAGs) and Completed Partially Directed Acyclic Graphs (CPDAGs). Experimental results show significant shortcomings in existing algorithms' generalization capabilities on real data, highlighting the potential for performance improvement and the importance of our framework in advancing causal discovery techniques.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
