无训练再生范式:对比反思记忆引导的自验证与自改进
A Training-Free Regeneration Paradigm: Contrastive Reflection Memory Guided Self-Verification and Self-Improvement
AI总结:
本文提出一种无需训练的再生范式,利用离线整理的对比反思记忆提供纠正指导,通过从头再生突破错误推理,提升大语言模型输出准确性。
摘要:
验证引导的自改进近期 emerged 为提高大语言模型 (LLM) 输出准确性的有前途的方法。然而,现有方法面临推理效率与准确性的权衡:迭代验证-修正过程计算成本高且易陷入错误推理,而 best-of-N 选择需要大量采样而不解决内部模型缺陷。我们提出一种无需训练的再生范式,利用离线整理的对比反思记忆 (RM) 提供纠正指导,同时从头再生有助于突破错误推理。在推理时,该方法执行 RM 引导的自验证,随后进行单次 RM 引导的再生,避免了迭代修正和多样本选择。我们在九个基准测试上评估了该方法,涵盖算法、推理、符号和领域特定任务,适用于小型和大型 LLM。实验结果表明,该方法在保持低计算成本的同时优于先前方法。
英文摘要:
Verification-guided self-improvement has recently emerged as a promising approach to improving the accuracy of large language model (LLM) outputs. However, existing approaches face a trade-off between inference efficiency and accuracy: iterative verification-rectification is computationally expensive and prone to being trapped in faulty reasoning, while best-of-N selection requires extensive sampling without addressing internal model flaws. We propose a training-free regeneration paradigm that leverages an offline-curated contrastive Reflection Memory (RM) to provide corrective guidance, while regenerating from scratch helps break out of faulty reasoning. At inference time, the method performs RM-guided self-verification followed by a single RM-guided regeneration, avoiding both iterative correction and multi-sample selection. We evaluated our method on nine benchmarks that span algorithmic, reasoning, symbolic, and domain-specific tasks in both small- and large-scale LLMs. Experiment results show that our method outperforms prior methods while maintaining low computational cost.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。