FLAME：适应性专家混合用于连续多模态多任务学习
FLAME: Adaptive Mixture-of-Experts for Continual Multimodal Multi-Task Learning
AI总结：
FLAME提出了一种可扩展的专家混合框架，支持多任务预训练和连续学习，通过模态特定路由器处理不同模态的任务，同时利用低秩内存子空间压缩专家知识，提升参数效率和减少灾难性遗忘。
摘要：
现实世界中跨多个领域的模型部署需要多模态模型在两个互补的模式下运行：（1）多任务预训练，任务在设计时同时存在，相关任务可以相互借鉴表示能力；（2）持续适应，部署后出现新任务，涉及以前未见过的模态组合。然而，单独的模式不足以满足需求：预训练任务集从不完整，而跳过联合训练会失去可训练任务间的转移收益和效率。稀疏专家混合（MoE）是满足这种双重需求的自然选择：稀疏激活使模块化容量扩展成为可能，而路由将模态层面的计算与任务层面的组成解耦。在本文中，我们提出了一种可扩展的MoE框架，用于在灵活的模态组合上进行多任务预训练和连续学习。该框架旨在通过利用模态特定的路由器来支持训练多模态任务，这些路由器在不同任务中处理每个模态的标记。此外，它通过将积累的专家知识压缩到低秩内存子空间中，同时仅扩展轻量级路由器，从而在固定容量的MoE中实现连续学习。我们在多个医疗多模态基准上验证了该方法的有效性。它在多任务预训练性能上具有竞争力，同时缓解了灾难性遗忘并提高了参数效率。
英文摘要：
Real-world model deployment across multiple domains requires multimodal models to operate under two complementary regimes: (1) multi-task pretraining, tasks are co-available at design time where related tasks could borrow representational strength from one another, (2) continual adaptation, in which new tasks emerge after deployment with previously unseen modality combinations. However, neither regime alone suffices: the pretraining task set is never exhaustive, while bypassing joint training forfeits the transfer gains and efficiency among co-trainable tasks. Sparse Mixture-of-Experts (MoE) is a natural fit for this dual requirement: sparse activation enables modular capacity expansion as new tasks arrive, while routing decouples modality-level computation from task-level composition. In this work, we propose a scalable MoE framework for multitask pretraining and continual learning across flexible modality combinations. The framework is designed to support training on multimodal tasks with diverse modality configurations by leveraging modality-specific routers that process tokens from each modality across tasks. Furthermore, it enables continual learning over sequential multimodal tasks within a fixed-capacity MoE by compressing accumulated expert knowledge into low-rank memory subspaces, while expanding only the lightweight routers. We validate the effectiveness of our method on multiple healthcare multimodal benchmarks. It demonstrates competitive multitask pretraining performance while alleviating catastrophic forgetting and improving parameter efficiency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
