基于相似性感知的专家混合模型用于数据高效持续学习
Similarity-Aware Mixture-of-Experts for Data-Efficient Continual Learning
AI总结：
本文提出一种适应性混合专家框架，通过增量全局池化和实例级提示遮蔽，解决数据有限和任务重叠带来的持续学习挑战，提升样本效率。
摘要：
机器学习模型在部署后常需适应新数据，因现实动态结构或无结构变化。持续学习（CL）框架支持连续模型适应，但现有方法多假设每个任务有足够的数据或任务不重叠。本文针对更一般的情况，即每个任务可能数据有限，任务任意重叠。此设置更具挑战性，因数据稀缺需有效上下文化通用知识并高效跨任务知识转移，而无结构任务重叠易导致负知识转移。本文提出适应性混合专家（MoE）框架，通过增量全局池化和实例级提示遮蔽，缓解提示关联噪声并分解任务样本为对齐当前提示（在分布）和需新提示（出分布）。实验显示方法在不同数据量和任务相似性下提升样本效率并广泛适用。
英文摘要：
Machine learning models often need to adapt to new data after deployment due to structured or unstructured real-world dynamics. The Continual Learning (CL) framework enables continuous model adaptation, but most existing approaches either assume each task contains sufficiently many data samples or that the learning tasks are non-overlapping. In this paper, we address the more general setting where each task may have a limited dataset, and tasks may overlap in an arbitrary manner without a priori knowledge. This general setting is substantially more challenging for two reasons. On the one hand, data scarcity necessitates effective contextualization of general knowledge and efficient knowledge transfer across tasks. On the other hand, unstructured task overlapping can easily result in negative knowledge transfer. To address the above challenges, we propose an adaptive mixture-of-experts (MoE) framework over pre-trained models that progressively establishes similarity awareness among tasks. Our design contains two innovative algorithmic components: incremental global pooling and instance-wise prompt masking. The former mitigates prompt association noise through gradual prompt introduction over time. The latter decomposes incoming task samples into those aligning with current prompts (in-distribution) and those requiring new prompts (out-of-distribution). Together, our design strategically leverages potential task overlaps while actively preventing negative mutual interference in the presence of per-task data scarcity. Experiments across varying data volumes and inter-task similarity show that our method enhances sample efficiency and is broadly applicable.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
