基于先例的推理:通过测试时先例学习缓解大推理模型中的过度思考
Precedent-Informed Reasoning: Mitigating Overthinking in Large Reasoning Models via Test-Time Precedent Learning
AI总结:
PIR通过测试时先例学习,优化大推理模型的推理过程,减少冗余探索并提升准确率与效率的平衡。
摘要:
在大型语言模型(LLMs)中进行推理时,往往会出现效率低下且冗长的思维链轨迹,伴随着重复的自我探索和验证,这会增加计算成本,甚至会降低性能。受人类推理模式的启发,人们通过利用过去的相关案例来约束搜索空间并减少试错,我们提出了基于先例的推理(PIR),将LLMs的推理范式从穷举式的自我探索转变为从先例中指导学习。PIR解决了两个关键挑战:采用哪些先例以及如何利用它们。首先,自适应先例选择(APS)为每个问题和LLM构建一个紧凑的先例集,这些先例在语义上相关且对模型具有信息性。它通过语义相似性和模型困惑度的联合得分对示例进行排序,然后根据困惑度减少最大化来调整先例的数量。其次,测试时经验内化(TEI)被视为在先例指导的指令上的测试时学习,通过更新轻量级适配器来内化解决方案模式,并在后续推理中将其作为先验使用。在数学推理、科学问答和代码生成等实验中,PIR一致地缩短了推理轨迹,同时在LLMs中保持或提高了最终准确性,实现了出色的准确率-效率权衡。
英文摘要:
Reasoning in Large Language Models (LLMs) often suffers from inefficient long chain-of-thought traces with redundant self-exploration and validation, which inflate computational costs and even degrade performance. Inspired by human reasoning patterns where people solve new problems by leveraging past related cases to constrain search spaces and reduce trial-and-error, we propose Precedent Informed Reasoning (PIR) transforming LRMs'reasoning paradigm from exhaustive self-exploration to guided learning from precedents. PIR addresses two key challenges: what precedents to adopt and how to utilize them. First, Adaptive Precedent Selection (APS) constructs, for each question and LRM, a compact set of precedents that are both semantically related and informative for the model. It ranks examples by a joint score with semantic similarity and model perplexity, then adapts the amount of precedents to maximize perplexity reduction. Second, Test-time Experience Internalization (TEI) is treated as the test-time learning on precedent-informed instruction, updating lightweight adapters to internalize solution patterns and use them as a prior during subsequent reasoning. Experiments across mathematical reasoning, scientific QA, and code generation demonstrate that PIR consistently shortens reasoning traces while maintaining or improving final accuracy across LLMs, yielding outstanding accuracy-efficiency trade-offs.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。