测试时的自反生成
Self-Reflective Generation at Test Time
AI总结：
提出SRGen框架，通过动态熵阈值识别高不确定性token并训练校正向量，在测试时进行自反生成以纠正概率分布，提升大模型推理的可靠性。
摘要：
大型语言模型（LLM）越来越多地通过长思维链解决复杂推理任务，但其仅前向的自回归生成过程是脆弱的；早期token错误可能级联，这明确需要自反机制。然而，现有的自反要么对完整草稿进行修订，要么通过昂贵的训练学习自我修正，两者本质上都是反应性的且低效。为解决此问题，我们提出测试时的自反生成（SRGen），一种轻量级测试时框架，在不确定点生成前进行反思。在token生成过程中，SRGen利用动态熵阈值识别高不确定性token。对于每个识别的token，它训练一个特定的校正向量，充分利用已生成的上下文进行自反生成以纠正token概率分布。通过回顾性分析部分输出，这种自反使得决策更可靠，从而显著降低高不确定点错误的概率。在具有挑战性的数学推理基准和多种LLM上的评估表明，SRGen能显著增强模型推理能力。此外，我们的发现将SRGen定位为一种即插即用的方法，将反思融入生成过程以实现可靠的LLM推理，在有限开销下取得一致收益，并可与其他训练时（如RLHF）和测试时（如SLOT）技术结合。
英文摘要：
Large language models (LLMs) increasingly solve complex reasoning tasks via long chain-of-thought, but their forward-only autoregressive generation process is fragile; early token errors can cascade, which creates a clear need for self-reflection mechanisms. However, existing self-reflection either performs revisions over full drafts or learns self-correction via expensive training, both fundamentally reactive and inefficient. To address this, we propose Self-Reflective Generation at Test Time (SRGen), a lightweight test-time framework that reflects before generating at uncertain points. During token generation, SRGen utilizes dynamic entropy thresholding to identify high-uncertainty tokens. For each identified token, it trains a specific corrective vector, which fully exploits the already generated context for a self-reflective generation to correct the token probability distribution. By retrospectively analyzing the partial output, this self-reflection enables more trustworthy decisions, thereby significantly reducing the probability of errors at highly uncertain points. Evaluated on challenging mathematical reasoning benchmarks and a diverse set of LLMs, SRGen can significantly strengthen model reasoning. Moreover, our findings position SRGen as a plug-and-play method that integrates reflection into the generation process for reliable LLM reasoning, achieving consistent gains with bounded overhead and can be combined with other training-time (e.g., RLHF) and test-time (e.g., SLOT) techniques.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
