面向具身持续学习的分层任务感知多模态增量LoRA专家混合方法
Hierarchical-Task-Aware Multi-modal Mixture of Incremental LoRA Experts for Embodied Continual Learning
AI总结：
针对具身智能持续学习中高层规划能力不足与灾难性遗忘问题，提出分层设置HEC和Task-aware MoILE方法，通过聚类识别任务、双级路由选择LoRA专家，并利用SVD保留关键参数，有效减少旧任务遗忘。
摘要：
以往的具身智能持续学习设置侧重于根据人类指令执行低层动作，忽略了学习高层规划与多层次知识的能力。为解决这些问题，我们提出了分层具身持续学习设置（HEC），将智能体的持续学习过程划分为两个层次：高层指令与低层动作，并定义了五个具身持续学习子设置。基于这些设置，我们引入了任务感知的增量LoRA专家混合方法（Task-aware MoILE）。该方法通过对视觉-文本嵌入进行聚类来实现任务识别，并使用任务级路由器和令牌级路由器选择合适的LoRA专家。为有效解决灾难性遗忘问题，我们对先前任务获得的LoRA参数应用奇异值分解（SVD），保留关键成分，同时正交地训练其余部分。实验结果表明，与其他方法相比，我们的方法在减少旧任务遗忘方面表现突出，能够有效支持智能体在持续学习新任务的同时保留先前知识。
英文摘要：
Previous continual learning setups for embodied intelligence focused on executing low-level actions based on human commands, neglecting the ability to learn high-level planning and multi-level knowledge. To address these issues, we propose the Hierarchical Embodied Continual Learning Setups (HEC) that divide the agent's continual learning process into two layers: high-level instructions and low-level actions, and define five embodied continual learning sub-setups. Building on these setups, we introduce the Task-aware Mixture of Incremental LoRA Experts (Task-aware MoILE) method. This approach achieves task recognition by clustering visual-text embeddings and uses both a task-level router and a token-level router to select the appropriate LoRA experts. To effectively address the issue of catastrophic forgetting, we apply Singular Value Decomposition (SVD) to the LoRA parameters obtained from prior tasks, preserving key components while orthogonally training the remaining parts. The experimental results show that our method stands out in reducing the forgetting of old tasks compared to other methods, effectively supporting agents in retaining prior knowledge while continuously learning new tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
