arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.31401cs.CLcs.AIcs.LG
收藏

HERD:用于持续学习的利用演化递归蒸馏框架

HERD: Harness-Evolving Recursive Distillation for Continual Learning

Xun Sun, Thanh Thi Nguyen, Thuy-Trang Vu

AI总结:

HERD是结合参数化与非参数化方法优势的混合持续学习框架,通过双锚点漂移约束蒸馏减少遗忘,在TRACE+17任务流中表现优异。

摘要:

持续学习要求模型在获取新技能和知识的同时不损害现有能力。参数化方法存在灾难性遗忘和可塑性丧失问题;非参数化方法(如提示、技能和harness)利用上下文学习,但受限于上下文长度和有损纯文本压缩。为有效结合两者互补优势,我们提出HERD(Harness-Evolving Recursive Distillation,利用演化递归蒸馏),这是一种混合持续学习框架,可在harness中高效学习并通过蒸馏将经验内化到累积能力中。在harness侧,HERD持续优化技能以捕获可复用模式,同时在有限内存中保留技能未能捕获的高残差样本;在参数侧,HERD采用双锚点的漂移约束蒸馏,大幅降低长递归运行中的遗忘。在TRACE+17的25阶段任务流中,在固定内存和计算预算下,HERD在新任务获取上与最强参数化基线相当,同时表现出最小的遗忘和最强的通用能力保留。

英文摘要:

Continual learning requires a model to keep acquiring new skills and knowledge without compromising its existing capabilities. Parametric methods suffer from catastrophic forgetting and loss of plasticity. Non-parametric methods, such as prompts, skills, and harnesses, leverage in-context learning but are limited by context length and lossy plain-text compression. To effectively combine their complementary strengths, we propose HERD (Harness-Evolving Recursive Distillation), a hybrid continual learning framework that learns efficiently in the harness and internalizes experience into cumulative capabilities via distillation. On the harness side, HERD continually refines its skills to capture reusable patterns, while retaining high-residual examples that the skills fail to capture in a bounded memory. On the parameter side, HERD performs drift-constrained distillation with dual anchors, substantially reducing forgetting over long recursive runs. On the 25-stage task stream of TRACE+17, under fixed memory and compute budgets, HERD matches the strongest parametric baseline in new-task acquisition while exhibiting the least forgetting and the strongest retention of general capabilities.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑