发表机构
University of Limerick（利默里克大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
SemiAdapt-Instruct是一种模块化指令微调框架，可发现潜在指令领域并训练对应LoRA适配器，无需全模型微调即可扩展，性能优于全模型微调且具备整体方法无法实现的可扩展性。
AI 中文摘要
经过指令微调的大型语言模型（LLM）被部署到领域不断演变的环境中，然而在不进行完整重新训练的情况下扩展微调模型的能力仍是一个未解决的实际挑战。我们提出了SemiAdapt-Instruct，这是一个模块化框架，可发现潜在的指令领域、并行训练各领域的LoRA适配器，并执行无参数路由，通过单适配器训练纳入新领域而无需修改现有组件。SemiAdapt-Instruct在ROUGE-L和LLM-as-a-judge评估的所有配置下均优于全模型微调，同时与单LoRA微调性能相当，并提供了整体方法无法实现的可扩展性。我们通过展示用新领域数据更新单个适配器的性能优于所有整体基线，从经验上证明了这种可扩展性。我们的研究还发现，独立的发现方法会收敛到相同的对专业化友好的领域。这些发现表明，将异构指令数据分解为潜在领域可实现可扩展的NLP系统，在领域演变时仅需针对性的单适配器更新，无需进行全模型重新训练。
英文摘要
Instruction-tuned LLMs are deployed into environments where domains evolve, yet extending a fine-tuned model's capabilities without full retraining remains an unsolved practical challenge. We present SemiAdapt-Instruct, a modular framework that discovers latent instruction domains, trains per-domain LoRA adapters in parallel, and performs parameter-free routing, incorporating new domains via single-adapter training without modifying existing components. SemiAdapt-Instruct outperforms full model fine-tuning across all configurations on both ROUGE-L and LLM-as-a-judge evaluation, while matching single LoRA fine-tuning and delivering extensibility that monolithic approaches cannot provide. We empirically demonstrate this extensibility by showing that updating a single adapter with new domain data outperforms all monolithic baselines. Our study also finds that independent discovery methods converge on the same specialisation-friendly domains. These findings demonstrate that decomposing heterogeneous instruction data into latent domains enables extensible NLP systems where evolving domains require only targeted single-adapter updates, eliminating the need for full model retraining.
