arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.06527cs.AI
收藏

HyPER:通过假设路径扩展与缩减实现可扩展的大语言模型推理中的探索与利用平衡

HyPER: Bridging Exploration and Exploitation for Scalable LLM Reasoning with Hypothesis Path Expansion and Reduction

Institute for Artificial Intelligence, Peking University, Beijing(北京大学人工智能研究院);Huawei(华为);School of Integrated Circuits, Peking University, Beijing(北京大学集成电路学院)

Shengxuan Qiu, Haochen Huang, Shuzhang Zhong, Pengfei Zuo, Meng Li

AI总结:

HyPER通过动态扩展-缩减控制问题优化多路径解码,提升推理准确性与计算效率,实验显示在不同基准上准确率提升8-10%,token使用减少25-40%。

摘要:

通过多路径链式思维扩展提升测试时计算量以提高推理准确性,但其效果依赖于探索与利用的权衡。现有方法以刚性方式处理此权衡:树结构搜索通过脆弱的扩展规则硬编码探索,干扰训练后的推理;并行推理过度探索冗余假设路径并依赖弱答案选择。受观察到最优平衡是阶段依赖且正确与错误推理路径往往在后期才分歧的启发,我们将测试时扩展重新表述为一个动态扩展-缩减控制问题。我们提出HyPER,一种无需训练的在线控制策略,用于混合专家模型中的多路径解码,利用轻量级路径统计在固定预算下重新分配计算。HyPER包含一个在线控制器,随着假设池演变从探索转向利用,一个令牌级细化机制,使在生成时高效利用而不需完整路径重采样,以及一个长度和置信度感知的聚合策略以实现可靠的答案时利用。在四个混合专家语言模型上进行的实验表明,HyPER在不同推理基准上始终实现更优的准确性-计算量权衡,准确率提升8到10个百分点,同时减少25到40个百分点的token使用。

英文摘要:

Scaling test-time compute with multi-path chain-of-thought improves reasoning accuracy, but its effectiveness depends critically on the exploration-exploitation trade-off. Existing approaches address this trade-off in rigid ways: tree-structured search hard-codes exploration through brittle expansion rules that interfere with post-trained reasoning, while parallel reasoning over-explores redundant hypothesis paths and relies on weak answer selection. Motivated by the observation that the optimal balance is phase-dependent and that correct and incorrect reasoning paths often diverge only at late stages, we reformulate test-time scaling as a dynamic expand-reduce control problem over a pool of hypotheses. We propose HyPER, a training-free online control policy for multi-path decoding in mixture-of-experts models that reallocates computation under a fixed budget using lightweight path statistics. HyPER consists of an online controller that transitions from exploration to exploitation as the hypothesis pool evolves, a token-level refinement mechanism that enables efficient generation-time exploitation without full-path resampling, and a length- and confidence-aware aggregation strategy for reliable answer-time exploitation. Experiments on four mixture-of-experts language models across diverse reasoning benchmarks show that HyPER consistently achieves a superior accuracy-compute trade-off, improving accuracy by 8 to 10 percent while reducing token usage by 25 to 40 percent.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑