HyperLoader：将基于超网络的LoRA和Adapter层集成到多任务Transformer中以进行序列标注
HyperLoader: Integrating Hypernetwork-Based LoRA and Adapter Layers into Multi-Task Transformers for Sequence Labelling
AI总结：
提出HyperLoader，利用超网络基于任务和层位置生成LoRA与Adapter权重，有效缓解多任务干扰，在高低资源场景下超越全量微调并取得最佳平均性能。
摘要：
我们提出了HyperLoader，这是一种在多任务环境中结合不同参数高效微调方法的简单方法。为实现该目标，我们的模型利用超网络，基于任务、transformer层及其在该层内的位置来生成这些模块的权重。我们的方法结合了多任务学习的优势，通过捕获所有任务的结构，同时通过将任务特定知识封装在生成的权重中来缓解任务干扰问题，并具备结合不同参数高效方法以超越全量微调的优势。我们提供了实证证据，表明HyperLoader在大多数数据集上优于先前方法，并在高资源和低资源场景下均获得了跨任务的最佳平均性能。
英文摘要：
We present HyperLoader, a simple approach that combines different parameter-efficient fine-tuning methods in a multi-task setting. To achieve this goal, our model uses a hypernetwork to generate the weights of these modules based on the task, the transformer layer, and its position within this layer. Our method combines the benefits of multi-task learning by capturing the structure of all tasks while reducing the task interference problem by encapsulating the task-specific knowledge in the generated weights and the benefits of combining different parameter-efficient methods to outperform full-fine tuning. We provide empirical evidence that HyperLoader outperforms previous approaches in most datasets and obtains the best average performance across tasks in high-resource and low-resource scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
