生成有效的CoT轨迹以缓解因果幻觉
Generating Effective CoT Traces for Mitigating Causal Hallucination
AI总结：
本文研究了生成有效CoT轨迹以缓解小模型因果幻觉的问题，设计了生成管道并提出新的量化指标CHR，实验表明生成的CoT轨迹能有效减少幻觉并提升准确性。
摘要：
尽管大型语言模型（LLMs）在复杂推理任务中表现出色，但在事件因果性识别（ECI）中存在严重的因果幻觉问题，尤其在小模型（≤1.5B参数）中更为明显。为解决此问题，一种有前景的方法是通过链式思维（CoT）轨迹进行微调。然而，目前缺乏可用的CoT轨迹数据集。本文首先研究了有效CoT轨迹应具备的必要标准，以缓解小模型中的因果幻觉。接着设计了生成满足这些标准的CoT轨迹的管道。此外，由于目前尚无量化因果幻觉的指标，我们还引入了新的指标，即因果幻觉率（CHR），用于量化因果幻觉，指导有效CoT轨迹的制定，并验证所提出管道的有效性。实验表明，使用本文生成的CoT轨迹微调不仅显著减少了小LLM中的因果幻觉，还提高了平均准确性。此外，微调后的模型在跨数据集和跨难度测试中表现出强大的泛化能力和对误导性干预提示的鲁棒性。
英文摘要：
Although large language models (LLMs) excel in complex reasoning tasks, they suffer from severe causal hallucination in event causality identification (ECI), particularly in smaller models ($\leq$1.5B parameters). A promising approach to address this issue is to fine-tune them with Chain-of-Thought (CoT) traces. However, there is currently a lack of CoT trace dataset available for ECI. In this paper, we first investigate the essential criteria that effective CoT traces should possess to mitigate causal hallucination in smaller models. We then design a pipeline to generate CoT traces that meet these criteria. Moreover, since there is currently no metric for quantifying causal hallucination, we also introduce a new metric, the Causal Hallucination Rate (CHR), to quantify causal hallucination, guide the formulation of effective CoT trace criteria, and validate the effectiveness of our pipeline. Our experiments show that fine-tuning with the CoT traces generated by our pipeline not only substantially reduces causal hallucination in smaller LLMs but also improves mean accuracy. Moreover, the fine-tuned models exhibit strong cross-dataset and cross-difficulty generalization, as well as robustness under misleading intervention prompts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
