LiME：轻量级专家混合用于高效的多模态多任务学习
LiME: Lightweight Mixture of Experts for Efficient Multimodal Multi-task Learning
AI总结：
LiME通过轻量级调制实现专家专业化，减少可训练参数并提升泛化能力，实验表明其在多模态多任务基准上性能优异且训练更快。
摘要：
MoE-PEFT方法通过混合专家与参数高效微调实现多任务适应，但需要每个专家单独的适配器，导致可训练参数随专家数量线性增长，限制了其在适配器架构中的应用。我们提出LiME（轻量级专家混合），通过轻量级调制而非适配器复制实现专家专业化。与单独适配器不同，LiME使用单一共享PEFT模块，并通过轻量级专家向量调制其输出，减少专家参数并适用于任何PEFT方法。值得注意的是，LiME引入零参数路由，利用现有冻结和适配的表示消除通常每层所需的学得路由参数。理论上，我们证明（i）更多专家保留更多任务相关信息，（ii）调制近似全专家特定PEFT并具有有界误差。LiME进一步结合基于路由信心的n-gram窗口路由和自适应专家选择（Auto Top-K）。在MMT-47多模态多任务基准（包含47个任务，涵盖文本、图像和视频）上的实验表明，LiME在使用最多4倍更少可训练参数和高达29%更快的训练速度的情况下，实现了竞争或更优的性能。
英文摘要：
MoE-PEFT methods combine Mixture of Experts with parameter-efficient fine-tuning for multi-task adaptation, but require separate adapters per expert causing trainable parameters to scale linearly with expert count and limiting applicability to adapter-based architectures. We propose LiME (Lightweight Mixture of Experts), which achieves expert specialization through lightweight modulation rather than adapter replication. Instead of separate adapters, LiME uses a single shared PEFT module and modulates its output with lightweight expert vectors, reducing expert parameters while generalizing to any PEFT method. Notably, LiME introduces zero-parameter routing by leveraging existing frozen and adapted representations eliminating learned router parameters typically required per layer. Theoretically, we prove that (i) more experts preserve more task-relevant information and (ii) modulation approximates full expert-specific PEFT with bounded error. LiME further incorporates n-gram windowed routing and adaptive expert selection (Auto Top-K) based on routing confidence. Experiments on MMT-47, a multimodal multi-task benchmark with 47 tasks spanning text, image, and video, demonstrate that LiME achieves competitive or superior performance while using up to 4x fewer trainable parameters and up to 29% faster training compared to corresponding MoE-PEFT baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
