arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.08083cs.CL
收藏

LLMs改进LLMs:测试时扩展的代理发现

LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling

UMD(马里兰大学);UVA(弗吉尼亚大学);WUSTL(华盛顿大学);UNC(北卡罗来纳大学);Google(谷歌);Meta

Tong Zheng, Haolin Liu, Chengsong Huang, Huiwen Bao, Sheng Zhang, Rui Liu, Runpeng Dai, Ruibo Chen, Chenxi Liu, Tianyi Xiong, Xidong Wu, Hongming Zhang, Heng Huang

AI总结:

本文提出AutoTTS框架,通过环境驱动的方法自动发现测试时扩展策略,提升大语言模型性能,实验表明其在数学推理基准上的准确率与成本平衡优于人工设计基线。

摘要:

测试时扩展(TTS)已成为提升大语言模型性能的有效方法,通过在推理时分配额外计算。然而,现有TTS策略大多手工设计:研究者手动设计推理模式并调整启发式方法,留下大量计算分配空间未被探索。我们提出一个环境驱动的框架AutoTTS,改变研究者设计的内容:从个体TTS启发式方法到环境,使TTS策略能够自动发现。AutoTTS的关键在于环境构建:发现环境必须使控制空间可 tractable 并提供廉价、频繁的反馈以支持TTS搜索。作为具体实现,我们将宽度-深度TTS表述为控制器合成,基于预收集的推理轨迹和探测信号,其中控制器决定何时分支、继续、探测、剪枝或停止,并可廉价评估而无需重复LLM调用。我们进一步引入beta参数化使搜索可 tractable,并引入细粒度执行跟踪反馈以提高发现效率,帮助代理诊断TTS程序失败的原因。在数学推理基准上的实验表明,发现的策略在整体准确率-成本平衡上优于强人工设计基线。发现的策略泛化到保留基准和模型规模,而整个发现成本仅39.9美元和160分钟。我们的数据和代码将在https://github.com/zhengkid/AutoTTS开源。

英文摘要:

Test-time scaling (TTS) has become an effective approach for improving large language model performance by allocating additional computation during inference. However, existing TTS strategies are largely hand-crafted: researchers manually design reasoning patterns and tune heuristics by intuition, leaving much of the computation-allocation space unexplored. We propose an environment-driven framework, AutoTTS, that changes what researchers design: from individual TTS heuristics to environments where TTS strategies can be discovered automatically. The key to AutoTTS lies in environment construction: the discovery environment must make the control space tractable and provide cheap, frequent feedback for TTS search. As a concrete instantiation, we formulate width--depth TTS as controller synthesis over pre-collected reasoning trajectories and probe signals, where controllers decide when to branch, continue, probe, prune, or stop and can be evaluated cheaply without repeated LLM calls. We further introduce beta parameterization to make the search tractable and fine-grained execution trace feedback to improve discovery efficiency by helping the agent diagnose why a TTS program fails. Experiments on mathematical reasoning benchmarks show that the discovered strategies improve the overall accuracy--cost tradeoff over strong manually designed baselines. The discovered strategies generalize to held-out benchmarks and model scales, while the entire discovery costs only $39.9 and 160 minutes. Our data, and code will be open-source at https://github.com/zhengkid/AutoTTS.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑