单token验证用于推理正确性估计
One-Token Verification for Reasoning Correctness Estimation
AI总结：
本文提出单token验证方法，用于高效估计推理正确性，通过低秩适应集成到LLM中，减少token使用并提升推理可靠性。
摘要：
近年来，大语言模型（LLMs）在复杂推理任务中的突破性进展，如数学问题解决，带来了显著的成功。提高性能的常见策略是并行思考，即生成多个推理轨迹，然后通过多数投票或最佳-N解码等聚合方案得出最终预测。然而，仍然存在两个关键挑战。首先，多样本解码导致显著的推理延迟，尤其是对于长形式输出。其次，有效机制仍有限，用于可靠评估单个推理轨迹正确性的机制仍然有限。为了解决这些挑战，我们引入了单token验证（OTV），这是一种计算方法，能够在生成过程中单次前向传递中估计推理正确性。OTV通过可学习的token激活，并通过低秩适应集成到LLM中，通过键值缓存探测内部推理信号，支持在生成的任何阶段进行token级正确性估计，而不会干扰主要推理。在数学推理基准测试中的实验表明，OTV在现有验证器中表现一致。此外，通过正确性引导的早期终止，OTV通过减少token使用量高达90%，优先选择更短、更可靠的解决方案。
英文摘要：
Recent breakthroughs in large language models (LLMs) have led to notable successes in complex reasoning tasks, such as mathematical problem solving. A common strategy for improving performance is parallel thinking, in which multiple reasoning traces are generated and the final prediction is made using aggregation schemes like majority voting or best-of-$N$ decoding. However, two key challenges persist. First, multi-sample decoding incurs substantial inference latency, especially for long-form outputs. Second, effective mechanisms for reliably assessing the correctness of individual reasoning traces are still limited. To address these challenges, we introduce One-Token Verification (OTV), a computational method that estimates reasoning correctness in a single forward pass during generation. OTV is activated by a learnable token and integrated into the LLM via low-rank adaptation to probe internal reasoning signals through the key-value cache, supporting token-level correctness estimation at any stage of generation without disrupting primary reasoning. Experiments on mathematical reasoning benchmarks demonstrate that OTV consistently surpasses existing verifiers. Additionally, OTV reduces token usage by up to $90\%$ through correctness-guided early termination, prioritizing shorter, more reliable solutions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
