arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.03441cs.LGcs.AI
收藏

机不可失:大语言模型智能体的测试时训练

No Time Like the Present: Agentic Test-Time Training for LLM Agents

Kuaishou Technology(快手科技)

Yanbo Wang, Jinhua Hao, Yuze Shi, Kun Yuan, Ming Sun

AI总结:

研究多轮智能体情节中的连续测试时训练,提出智能体测试时训练方法,通过token级重加权减少重复文本损失,构建并发服务系统,提升了在ALFWorld和SWE-bench Lite上的成功率。

摘要:

大语言模型智能体在长时间情节中常退化。测试时训练可使模型权重适应任务状态,但现有方法多只适应固定输入。本文研究多轮智能体情节中的连续测试时训练,发现更新文本重复可区分不同情况,引入智能体测试时训练方法,构建并发服务系统,提升了成功率。

英文摘要:

LLM agents often degrade over long episodes: as trajectories grow, they revisit explored states, repeat failed actions, and lose strategies that previously worked. Test-time training (TTT) offers a way to adapt model weights to the evolving task state, but existing LLM TTT methods largely adapt once to a fixed input. We study continuous TTT in multi-turn agent episodes, where each update changes the policy that generates later training text. This creates a self-training loop that helps when new trajectory information appears, but can amplify drift when the agent gets stuck and repeatedly trains on similar text. We find that update-text repetition distinguishes these regimes and introduce Agentic Test-Time Training (aTTT), a token-level reweighting method that downweights the loss on tokens appearing in repeated $n$-grams from prior updates while leaving novel tokens fully weighted. To run such updates inside live episodes, we build a concurrent serving system using vLLM's runtime LoRA API, limiting overhead to 1.9$\times$ the no-TTT cost. aTTT improves success by up to 5.0 points on ALFWorld and 4.9 points on SWE-bench Lite. The gains concentrate where models already have task competence but drift over long trajectories, suggesting that aTTT mainly preserves existing competence rather than teaching new abilities.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑