arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.20441cs.CL
收藏

无训练再生范式:对比反思记忆引导的自验证与自改进

A Training-Free Regeneration Paradigm: Contrastive Reflection Memory Guided Self-Verification and Self-Improvement

Department of Electrical and Computer Engineering, McGill University(麦吉尔大学电气与计算机工程系);Department of Mechanical, Industrial and Aerospace Engineering, Concordia University(Concordia大学机械、工业与航空航天工程系)

Yuran Li, Di Wu, Benoit Boulet

AI总结:

本文提出一种无需训练的再生范式,利用离线整理的对比反思记忆提供纠正指导,通过从头再生突破错误推理,提升大语言模型输出准确性。

摘要:

验证引导的自改进近期 emerged 为提高大语言模型 (LLM) 输出准确性的有前途的方法。然而,现有方法面临推理效率与准确性的权衡:迭代验证-修正过程计算成本高且易陷入错误推理,而 best-of-N 选择需要大量采样而不解决内部模型缺陷。我们提出一种无需训练的再生范式,利用离线整理的对比反思记忆 (RM) 提供纠正指导,同时从头再生有助于突破错误推理。在推理时,该方法执行 RM 引导的自验证,随后进行单次 RM 引导的再生,避免了迭代修正和多样本选择。我们在九个基准测试上评估了该方法,涵盖算法、推理、符号和领域特定任务,适用于小型和大型 LLM。实验结果表明,该方法在保持低计算成本的同时优于先前方法。

英文摘要:

Verification-guided self-improvement has recently emerged as a promising approach to improving the accuracy of large language model (LLM) outputs. However, existing approaches face a trade-off between inference efficiency and accuracy: iterative verification-rectification is computationally expensive and prone to being trapped in faulty reasoning, while best-of-N selection requires extensive sampling without addressing internal model flaws. We propose a training-free regeneration paradigm that leverages an offline-curated contrastive Reflection Memory (RM) to provide corrective guidance, while regenerating from scratch helps break out of faulty reasoning. At inference time, the method performs RM-guided self-verification followed by a single RM-guided regeneration, avoiding both iterative correction and multi-sample selection. We evaluated our method on nine benchmarks that span algorithmic, reasoning, symbolic, and domain-specific tasks in both small- and large-scale LLMs. Experiment results show that our method outperforms prior methods while maintaining low computational cost.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑