arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.08222cs.AI
收藏

弱驱动学习：弱智能体如何使强智能体更强

Weak-Driven Learning: How Weak Agents make Strong Agents Stronger

Beihang University（北航）；China Telecom eSurfing Cloud（中国电信eSurfing云）

Zehao Chen, Gongxun Li, Tianxiang Ai, Zixuan Huang, Xiaodong Liu, Yifei Li, Wang Zhou, Fuzhen Zhuang, Xianglong Liu, Jianxin Li, Deqing Wang, Yikun Ban

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对大语言模型后训练中的饱和瓶颈，提出WMSS方法，利用模型历史弱检查点通过熵动力学识别可恢复学习差距并进行补偿学习，在数学推理和代码生成任务上实现有效性能提升且无额外推理成本。

摘要：

随着后训练优化成为改进大语言模型的核心，我们观察到一种持续的饱和瓶颈：一旦模型变得高度自信，进一步训练带来的收益递减。虽然现有方法继续强化目标预测，但我们发现信息丰富的监督信号仍然潜藏在模型自身的历史弱状态中。受此观察启发，我们提出WMSS（弱智能体可以使强智能体更强），一种利用弱检查点指导持续优化的后训练范式。通过熵动力学识别可恢复的学习差距，并通过补偿学习强化它们，WMSS使强智能体能够超越传统的后训练饱和。在数学推理和代码生成数据集上的实验表明，使用我们的方法训练的智能体实现了有效的性能提升，同时不产生额外的推理成本。

英文摘要：

As post-training optimization becomes central to improving large language models, we observe a persistent saturation bottleneck: once models grow highly confident, further training yields diminishing returns. While existing methods continue to reinforce target predictions, we find that informative supervision signals remain latent in models' own historical weak states. Motivated by this observation, we propose WMSS (Weak Agents Can Make Strong Agents Stronger), a post-training paradigm that leverages weak checkpoints to guide continued optimization. By identifying recoverable learning gaps via entropy dynamics and reinforcing them through compensatory learning, WMSS enables strong agents to improve beyond conventional post-training saturation. Experiments on mathematical reasoning and code generation datasets show that agents trained with our approach achieve effective performance improvements, while incurring zero additional inference cost.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑