Multi LoRA 遇见视觉：合并多个适配器以创建多任务模型
Multi LoRA Meets Vision: Merging multiple adapters to create a multi task model
AI总结：
本文研究将多个计算机视觉任务上训练的 LoRA 适配器合并为一个多任务模型，无需重新训练即可减少推理时间，并在最多三个适配器合并实验中验证其性能可与头部微调相当或更优。
摘要：
参数高效微调（PEFT）方法广泛用于大语言模型（LLM）和计算机视觉中的生成模型。特别是，在推理过程中可以使用多个此类方法来改变基础模型的行为。在本文中，我们研究了在计算机视觉任务上训练的多个 LoRA 适配器是否可以在推理过程中合并使用而不损失性能。通过实现这一点，只需合并不同的 LoRA 即可创建多任务模型。合并这些适配器将减少推理时间，并且不需要任何额外的重新训练。我们在六个不同的任务上训练了适配器，并评估了它们合并在一起时的性能。为了进行比较，我们使用了一个具有冻结主干网络并微调其头部的模型。我们的结果表明，即使使用简单的合并技术，通过合并适配器来创建多任务模型也是可行的，在某些情况下性能会略有下降。在我们的实验中，我们最多将三个适配器合并在一起。根据任务以及适配器训练所用数据的相似性，合并后的模型可以优于头部微调。我们观察到，与在相似数据集上训练的模型相比，在不相似数据集上训练的 LoRA 往往表现更好。
英文摘要：
Parameter efficient finetuning (PEFT) methods are widely used in LLMs and generative models in computer vision. Especially one can use multiple of these during inference to change the behavior of the base model. In this paper we investigated whether multiple LoRA adapters trained on computer vision tasks can be merged together and used during inference without loss in performance. By achieving this, multitask models can be created just by merging different LoRAs. Merging these will reduce inference time and it will not require any additional retraining. We have trained adapters on six different tasks and evaluated their performance when they are merged together. For comparison we used a model with a frozen backbone and finetuned its head. Our results show that even with simple merging techniques creating a multitask model by merging adapters is achievable by slightly loosing performance in some cases. In our experiments we merged up to three adapters together. Depending on the task and the similarity of the data adapters were trained on, merges can outperform head finetuning. We have observed that LoRAs trained with dissimilar datasets tend to perform better compared to model trained on similar datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
