arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2611.05313cs.CL
收藏

MDSKILL:通过多视图蒸馏学习内化技能记忆以实现高效的动态智能体自进化

MDSKILL: Learning to Internalize Skill Memory through Multi-View Distillation for Efficient Dynamic Agent Self-Evolution

Zhejiang University(浙江大学);Kuaishou Technology(快手科技)

Dian Jin, Jinluan Yang, Kairong Han, Baohong Li, Zijing Hu, Yi He, Wenlin Zhong, Yunze Tong, Minqin Zhu, Linchuan Li, Yanan Niu, Kun Kuang

AI总结:

本研究针对智能体自进化中技能记忆蒸馏的噪声与冗余问题,提出MDSKILL多视图蒸馏方法,通过分视图组织技能并结合OPSD整合知识,在无额外开销下于ALFWorld等基准实现性能提升。

摘要:

技能记忆将过往经验编码为文本提示,用于提升自进化智能体的强化学习(RL)效果。与RL互补的在线策略自蒸馏(OPSD)提供密集、细粒度的监督,以从技能引导的轨迹中内化知识并加速策略学习。然而,自生成的记忆常包含噪声监督及功能纠缠、冗余或冲突的知识,导致直接蒸馏困难。在有限的训练预算下,这些问题会阻碍知识内化甚至降低性能。为解决该问题,我们提出MDSKILL——一种用于技能记忆内化的多视图蒸馏方法。MDSKILL将技能记忆组织为三个互补的功能视图(状态 grounding、动作规划与执行控制),并使用动态指标选择高质量技能,以构建特定视图及集成推理分支。随后OPSD跨这些分支传递并整合互补的经验知识到共享策略中,实现高效的技能内化与持续的智能体自进化。我们在三个智能体基准上评估MDSKILL,对比不依赖更强外部教师模型的RL、技能增强及自蒸馏基线。在无额外计算开销的情况下,MDSKILL在ALFWorld和WebShop上较SOTA性能分别提升最高7.1%和8.1%。大量 ablation 实验与进化分析进一步验证了其在知识内化与策略改进中的有效性。代码可在该https URL获取。

英文摘要:

Skill memories encode past experiences as textual prompts to improve reinforcement learning (RL) for self-evolving agents. Complementing RL, On-Policy Self-Distillation (OPSD) provides dense, fine-grained supervision to internalize knowledge from skill-guided trajectories and accelerate policy learning. However, self-generated memories often contain noisy supervision and functionally entangled, redundant, or conflicting knowledge, making direct distillation difficult. Under limited training budgets, these issues can hinder knowledge internalization and even degrade performance. To address this problem, we propose MDSKILL, a multi-view distillation approach for skill memory internalization. MDSKILL organizes skill memories into three complementary functional views (state grounding, action planning, and execution control) and selects high-quality skills using dynamic metrics to construct view-specific and integrated reasoning branches. OPSD then transfers and consolidates complementary experiential knowledge across these branches into a shared policy, enabling efficient skill internalization and continuous agent self-evolution. We evaluate MDSKILL on three agent benchmarks against RL, skill-augmented, and self-distillation baselines that do not rely on stronger external teacher models. Without additional computational overhead, MDSKILL improves upon SOTA performance by up to 7.1% and 8.1% on ALFWorld and WebShop. Extensive ablations and evolution analyses further validate its effectiveness in knowledge internalization and policy improvement. The code is available at https://anonymous.4open.science/r/MDSKILL-CF53.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑