通过气候 discourse 中的隐含因果链发现评估 LLM 推理
Assessing LLM Reasoning Through Implicit Causal Chain Discovery in Climate Discourse
AI总结：
本文通过隐含因果链发现任务评估 LLM 的因果推理能力，发现其生成的因果步骤数量和粒度存在差异，但主要依赖关联模式匹配而非真实因果推理。
摘要：
如何导致一个效果，以及哪些中间因果步骤解释它们的联系？这项工作通过隐含因果链发现任务审视大语言模型（LLMs）的机理因果推理能力。在诊断评估框架下，我们指导九个 LLM 生成所有可能的中间因果步骤，将给定的因果对连接到因果链结构中。这些对来自最近的论证研究资源，包含关于气候变化的极化讨论。我们的分析揭示，LLMs 在生成的因果步骤数量和粒度上存在差异。尽管它们通常在生成的链中自洽且自信，但其判断主要由关联模式匹配驱动，而非真正的因果推理。不过，人类评估确认了生成链的逻辑一致性和完整性。我们的基线因果链发现方法、诊断评估的见解以及包含因果链的基准数据集为未来在论证设置中推进隐含、机理因果推理奠定了坚实基础。
英文摘要：
How does a cause lead to an effect, and which intermediate causal steps explain their connection? This work scrutinizes the mechanistic causal reasoning capabilities of large language models (LLMs) to answer these questions through the task of implicit causal chain discovery. In a diagnostic evaluation framework, we instruct nine LLMs to generate all possible intermediate causal steps linking given cause-effect pairs in causal chain structures. These pairs are drawn from recent resources in argumentation studies featuring polarized discussion on climate change. Our analysis reveals that LLMs vary in the number and granularity of causal steps they produce. Although they are generally self-consistent and confident about the intermediate causal connections in the generated chains, their judgments are mainly driven by associative pattern matching rather than genuine causal reasoning. Nonetheless, human evaluations confirmed the logical coherence and integrity of the generated chains. Our baseline causal chain discovery approach, insights from our diagnostic evaluation, and benchmark dataset with causal chains lay a solid foundation for advancing future work in implicit, mechanistic causal reasoning in argumentation settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
