利用蕴含图消除 LLM 推理中的偏差
Neutralizing Bias in LLM Reasoning using Entailment Graphs
AI总结：
针对 LLM 在 NLI 中因佐证偏差产生幻觉的问题，提出基于蕴含图的无监督反事实数据构建与微调框架，并通过对抗及偏差中和数据集验证其可减偏且提升推理性能。
摘要：
人们常称 LLM 具备自然语言推理（NLI）能力，而 NLI 被广泛视为更复杂推理形式的基石。然而，近期研究表明，LLM 在 NLI 中仍会因佐证偏差而产生幻觉：它们过度依赖命题记忆来构建捷径。为解决该问题，我们设计了一种无监督框架，用于构建反事实推理数据并微调 LLM，以减少佐证偏差。为衡量偏差降低程度，我们构建了 NLI 数据集的偏差对抗变体，在前提中随机替换谓词，同时保持假设不变。大量评估表明，我们的框架能够显著减少由佐证偏差导致的幻觉。随后，我们在原始 NLI 数据集及其偏差中和版本上进一步评估了经该框架微调的 LLM；在偏差中和版本中，原始实体会被替换为随机采样的实体。大量结果表明，我们的框架在原始和偏差中和的 NLI 数据集上都能持续提升推理性能。
英文摘要：
LLMs are often claimed to be capable of Natural Language Inference (NLI), which is widely regarded as a cornerstone of more complex forms of reasoning. However, recent works show that LLMs still suffer from hallucinations in NLI due to attestation bias, where LLMs overly rely on propositional memory to build shortcuts. To solve the issue, we design an unsupervised framework to construct counterfactual reasoning data and fine-tune LLMs to reduce attestation bias. To measure bias reduction, we build bias-adversarial variants of NLI datasets with randomly replaced predicates in premises while keeping hypotheses unchanged. Extensive evaluations show that our framework can significantly reduce hallucinations from attestation bias. Then, we further evaluate LLMs fine-tuned with our framework on original NLI datasets and their bias-neutralized versions, where original entities are replaced with randomly sampled ones. Extensive results show that our framework consistently improves inferential performance on both original and bias-neutralized NLI datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
