arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.14467cs.CL
收藏

使用大型语言模型进行显式归纳推理

Explicit Inductive Inference using Large Language Models

University of Edinburgh(爱丁堡大学)

Tianyang Liu, Tianyi Li, Liang Cheng, Mark Steedman

AI总结:

本文提出一种利用LLMs证明偏差进行显式归纳推理的流水线,通过将前提转换为已证明的替代项并聚合新蕴含查询的答案,显著提升了模型推理性能并减轻了偏差影响。

摘要:

据报道,大型语言模型(LLMs)在推理任务中存在不良的证明偏差:当被要求预测前提P是否蕴含假设H时,LLMs倾向于将H的上下文外真实性标签作为脆弱的代理,而不是考虑由P蕴含的H的条件真实性。在本文中,我们提出了一种利用这种偏差来进行显式归纳推理的流水线。我们的流水线使用LLM将前提转换为一组已证明的替代项,然后聚合派生的新蕴含查询的答案以支持原始推理预测。在定向谓词蕴含基准上,我们证明通过应用这种简单的流水线,我们可以提高LLMs在推理上的整体性能,并实质性地减轻其证明偏差的影响。

英文摘要:

Large Language Models (LLMs) are reported to hold undesirable attestation bias on inference tasks: when asked to predict if a premise P entails a hypothesis H, instead of considering H's conditional truthfulness entailed by P, LLMs tend to use the out-of-context truth label of H as a fragile proxy. In this paper, we propose a pipeline that exploits this bias to do explicit inductive inference. Our pipeline uses an LLM to transform a premise into a set of attested alternatives, and then aggregate answers of the derived new entailment inquiries to support the original inference prediction. On a directional predicate entailment benchmark, we demonstrate that by applying this simple pipeline, we can improve the overall performance of LLMs on inference and substantially alleviate the impact of their attestation bias.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑