arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.19094cs.CL
收藏

从正确性中学习无需提示使大语言模型成为高效推理器

Learning From Correctness Without Prompting Makes LLM Efficient Reasoner

City University of Hong Kong(香港城市大学);Dongguan University of Technology(东莞理工学院)

Yuxuan Yao, Han Wu, Zhijiang Guo, Biyan Zhou, Jiahui Gao, Sichun Luo, Hanxu Hou, Xiaojin Fu, Linqi Song

AI总结:

本文提出LeCo框架,通过从正确推理步骤中学习并基于生成逻辑概率度量步骤置信度,无需人类反馈或外部工具即可提升大语言模型在多步推理任务上的性能并减少令牌消耗。

摘要:

大语言模型(LLMs)在各种任务中展现出卓越的性能,但仍存在幻觉、不忠实推理和有毒内容等局限性。解决这些问题的一种潜在方法是学习人类或外部反馈(如工具)。在本文中,我们提出了一种针对LLMs的内在自我纠正推理框架,该框架无需人类反馈、外部工具和手工设计的提示。所提出的框架基于多步推理范式“从正确性中学习”(LeCo),无需从错误中学习即可提高推理性能。该范式优先从正确的推理步骤中学习,并提出了一种基于生成逻辑概率来度量每个推理步骤置信度的独特方法。在多种多步推理任务上的实验结果表明,该框架在降低令牌消耗的同时有效提高了推理性能。

英文摘要:

Large language models (LLMs) have demonstrated outstanding performance across various tasks, yet they still exhibit limitations such as hallucination, unfaithful reasoning, and toxic content. One potential approach to mitigate these issues is learning from human or external feedback (e.g. tools). In this paper, we introduce an intrinsic self-correct reasoning framework for LLMs that eliminates the need for human feedback, external tools, and handcraft prompts. The proposed framework, based on a multi-step reasoning paradigm \textbf{Le}arning from \textbf{Co}rrectness (\textsc{LeCo}), improves reasoning performance without needing to learn from errors. This paradigm prioritizes learning from correct reasoning steps, and a unique method to measure confidence for each reasoning step based on generation logits. Experimental results across various multi-step reasoning tasks demonstrate the effectiveness of the framework in improving reasoning performance with reduced token consumption.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑