interwhen：一种可泛化的引导推理模型进行测试时验证的框架
interwhen: A Generalizable Framework for Steering Reasoning Models with Test-time Verification
AI总结：
interwhen提出一种单轨迹验证框架，通过反馈中间推理轨迹引导模型行为，解决验证状态获取和 verifier 缺乏的问题，提升推理代理的任务完成和政策合规性。
摘要：
推理模型生成长序列的中间决策和工具调用，测试时验证对确保正确性至关重要。现有方法或仅验证最终答案，错过早期错误，或依赖分支验证策略探索多轨迹。我们引入interwhen，一种单轨迹验证框架，通过提供反馈引导模型行为。它解决两个关键挑战：首先，给定一组验证器，从推理轨迹获取可验证状态通常需要提示工程或外部任务分解为固定步骤。相反，我们提出一个监控系统，定期轮询推理轨迹并分叉推理模型的推断以恢复中间状态。验证器异步运行于生成过程中，正确执行时开销 negligible，仅在发生违规时介入。其次，除了数学和代码，过程验证的核心挑战是验证器的稀缺性。interwhen通过从自然语言政策文档自动合成验证器解决此问题。给定一个政策，它可以生成代码基于验证器，包括在Lean和z3中可证明正确的验证器。共同，这些贡献产生了一种即插即用的测试时验证系统，可以提高任何推理代理的任务完成和政策合规性。在推理基准中，政策编码数学或逻辑约束时，interwhen在推理模型上实现接近完美的准确性，使用比基线少的token。在具有基于政策的验证器生成的代理基准中，它使SLMs的任务质量改进，无需任何微调，例如Qwen3-30B在tau2-bench电信领域任务完成率从32%提升到87%。代码在https://github.com/microsoft/interwhen。
英文摘要：
Reasoning models produce long traces of intermediate decisions and tool calls, making test-time verification important for ensuring correctness. Existing approaches either verify only the final answer, which misses early errors, or rely on branch-and-verify strategies that explore multiple trajectories. We introduce interwhen, a single-trajectory verification framework that steers model behavior by providing feedback on intermediate reasoning traces. It addresses two key challenges. First, given a set of verifiers, obtaining verifiable states from the reasoning trace typically requires prompt engineering or external task decomposition into fixed steps. Instead, we propose a monitoring system that periodically polls the reasoning trace and forks inference of the reasoning model to recover intermediate states. Verifiers are run asynchronously alongside generation, adding negligible overhead on correct executions and intervening only when violations occur. Second, beyond math and code, a central challenge for process verification is the scarcity of verifiers. interwhen addresses this through automatic verifier synthesis from natural-language policy documents. Given a policy, it can generate code-based verifiers, including provably correct verifiers in Lean and z3. Together, these contributions yield a plug-and-play test-time verification system that can improve task completion and policy compliance of any reasoning agent. On reasoning benchmarks where policies encode mathematical or logical constraints, interwhen achieves near-perfect accuracy for reasoning models using a fraction of the tokens of baselines. On agentic benchmarks with policy-based verifier generation, it enables improvements in task quality for SLMs without any finetuning, e.g., task completion rate of Qwen3-30B jumps from 32% to 87% on the telecom domain in tau2-bench. Code at https://github.com/microsoft/interwhen.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
