AutoAdapt：自动领域发现实现低成本可扩展性
AutoAdapt: Automatic Domain Discovery Enables Low-Cost Extensibility
浏览论文内容
中文总结 AI 辅助
AutoAdapt通过自动发现潜在领域并独立训练每个领域的LoRA适配器，实现无需全模型重训的模块化扩展，在14个基准上达到与全局训练相当的性能。
中文摘要 AI 辅助
指令调优模型被部署在领域异构且不断演变的环境中，然而添加新领域或数据通常需要昂贵的重新训练。我们提出了AutoAdapt，一个模块化框架，通过针对性的单适配器训练来整合新领域和数据，而无需修改其他适配器。该框架自动发现潜在领域，利用这些领域并行独立地训练每个领域的低秩适配（LoRA）适配器，并执行无参数路由。在14个领域特定基准测试和GPT-4o成对评估中，AutoAdapt达到了与在所有领域上训练的LoRA适配器相当的性能，而无需进行全模型重新训练。我们还发现了跨独立发现方法的特化效应收敛的证据。总体而言，在每个适配器自身领域上训练可从根本上防止领域干扰，从而实现模块化、无分类法的领域特化，而不会导致总体性能损失或全模型重新训练。
英文摘要
Instruction-tuned models are deployed into environments where domains are heterogeneous and evolve, yet adding new domains or data typically requires costly retraining. We present AutoAdapt, a modular framework that incorporates new domains and data via targeted single-adapter training without modifying other adapters. The framework automatically discovers latent domains, uses them to train per-domain Low-Rank Adaptation (LoRA) adapters independently in parallel and performs parameter-free routing. Across 14 domain-specific benchmarks and GPT-4o pairwise judgements, AutoAdapt achieves parity with a LoRA adapter trained on all domains without requiring full-model retraining. We also find evidence of specialisation effect convergence across independent discovery methods. Overall, training each adapter on its own domain prevents domain interference by construction, thus enabling modular, taxonomy-free domain specialisation without aggregate performance loss or full model retraining.
发表机构
- University of Limerick（利莫瑞克大学）
机构由 AI 辅助整理，请以论文原文为准。
