Continual learning requires a model to keep acquiring new skills and knowledge without compromising its existing capabilities. Parametric methods suffer from catastrophic forgetting and loss of plasticity. Non-parametric methods, such as prompts, skills, and harnesses, leverage in-context learning but are limited by context length and lossy plain-text compression. To effectively combine their complementary strengths, we propose HERD (Harness-Evolving Recursive Distillation), a hybrid continual learning framework that learns efficiently in the harness and internalizes experience into cumulative capabilities via distillation. On the harness side, HERD continually refines its skills to capture reusable patterns, while retaining high-residual examples that the skills fail to capture in a bounded memory. On the parameter side, HERD performs drift-constrained distillation with dual anchors, substantially reducing forgetting over long recursive runs. On the 25-stage task stream of TRACE+17, under fixed memory and compute budgets, HERD matches the strongest parametric baseline in new-task acquisition while exhibiting the least forgetting and the strongest retention of general capabilities.
Self-Controlled Dynamic Expansion Model for Continual Learning
Huazhong University of Science and Technology(华中科技大学);University of Electronic Science and Technology of China(电子科技大学);Technische Universität München(慕尼黑工业大学)
Energy-Efficient and Real-Time Sensing for Federated Continual Learning via Sample-Driven Control
School of Electrical and Electronic Engineering, Hanoi University of Science and Technology(电子与电气工程学院,河内科学技术大学);Department of Intelligent Computing and Data Science, VinUniversity(智能计算与数据科学系,文大学);Department of Electrical and Computer Engineering, University of Saskatchewan(电气与计算机工程系,萨斯喀彻温大学);School of Electrical and Data Engineering, University of Technology Sydney(电气与数据工程学院,悉尼技术大学);School of Electricals and Electronics, Hanoi University of Science and Technology(电子与电气工程学院,河内科学技术大学);School of Computer Science and Statistics, Trinity College Dublin(计算机科学与统计学学院,都柏林信任学院)
Boosting LLM Translation Skills without General Ability Loss via Rationale Distillation
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Department of Computer Science, University of Illinois at Chicago(芝加哥大学计算机科学系)
EmbodiedCLBench: Evaluating Continual Learning for Self-Evolving Embodied Agents
School of Intelligence Science and Technology, Peking University(北京大学智能科学与技术学院);State Key Laboratory of General Artificial Intelligence, BIGAI(北京通用人工智能研究院通用人工智能国家重点实验室);University of Hong Kong(香港大学)