arXiv 2506.059852026-05-28cs.LGcs.RO

动态渐进式参数高效专家库混合用于终身机器人学习

Dynamic Mixture of Progressive Parameter-Efficient Expert Library for Lifelong Robot Learning

The University of Hong Kong（香港大学）；Institute of Artificial Intelligence (TeleAI), China Telecom（人工智能研究院（TeleAI），中国电信）；Huawei Cloud Computing Technologies（华为云计算技术）；Ola Dimensions；HKU Shanghai Intelligent Computing Research Center（香港大学上海智能计算研究中心）

Yuheng Lei, Sitong Mao, Shunbo Zhou, Hongyuan Zhang, Xuelong Li, Ping Luo

备注：: Accepted to Transactions on Machine Learning Research (TMLR) at https://openreview.net/forum?id=MHVBrjS8cG . Code is available at https://github.com/HarryLui98/DMPEL

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对终身学习中任务标识不可用和知识隔离问题，提出动态渐进式参数高效专家库混合（DMPEL），通过构建低秩专家库和轻量路由器实现灵活的前向迁移，并引入专家系数回放缓解遗忘，在LIBERO基准上以最少可训练参数和存储超越现有方法。

摘要：

一个通用智能体必须在其生命周期中持续学习和适应，实现高效的前向迁移，同时最小化灾难性遗忘。先前在主导的预训练-微调范式中的工作探索了用于单任务适应的参数高效微调，通过少量参数有效引导冻结的预训练模型。然而，在终身学习背景下，这些方法依赖于测试时任务标识符这一不切实际的假设，并限制了孤立适配器之间的知识共享。为解决这些限制，我们提出了用于终身机器人学习的动态渐进式参数高效专家库混合（DMPEL）。DMPEL逐步构建一个低秩专家库，并采用轻量路由器将专家动态组合成端到端策略，从而实现灵活高效的终身前向迁移。此外，通过利用微调参数的模块化结构，我们引入了专家系数回放，引导路由器准确检索先前遇到任务的冻结专家。该技术缓解了遗忘，同时相比对整个策略进行经验回放，显著节省存储和计算。在终身机器人学习基准LIBERO上的大量实验表明，我们的框架在持续适应过程中的成功率上优于最先进的终身学习方法，同时使用了最少的可训练参数和存储。

英文摘要：

A generalist agent must continuously learn and adapt throughout its lifetime, achieving efficient forward transfer while minimizing catastrophic forgetting. Previous work within the dominant pretrain-then-finetune paradigm has explored parameter-efficient fine-tuning for single-task adaptation, effectively steering a frozen pretrained model with a small number of parameters. However, in the context of lifelong learning, these methods rely on the impractical assumption of a test-time task identifier and restrict knowledge sharing among isolated adapters. To address these limitations, we propose Dynamic Mixture of Progressive Parameter-Efficient Expert Library (DMPEL) for lifelong robot learning. DMPEL progressively builds a low-rank expert library and employs a lightweight router to dynamically combine experts into an end-to-end policy, enabling flexible and efficient lifelong forward transfer. Furthermore, by leveraging the modular structure of the fine-tuned parameters, we introduce expert coefficient replay, which guides the router to accurately retrieve frozen experts for previously encountered tasks. This technique mitigates forgetting while being significantly more storage- and computation-efficient than experience replay over the entire policy. Extensive experiments on the lifelong robot learning benchmark LIBERO demonstrate that our framework outperforms state-of-the-art lifelong learning methods in success rates during continual adaptation, while utilizing minimal trainable parameters and storage.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。