HyperGrid：具有网格可分解超投影的高效多任务Transformers
HyperGrid: Efficient Multi-Task Transformers with Grid-wise Decomposable Hyper Projections
摘要：
在自然语言理解任务上实现最先进的性能通常依赖于为每个任务微调新模型。因此，这种方法导致更高的总体参数成本，以及为多个模型提供服务的更高的技术维护。学习一个能够很好地完成所有任务的单一多任务模型一直是一个具有挑战性但又有吸引力的命题。在本文中，我们提出了\textsc{HyperGrid}，一种新的方法，高效的多任务学习。所提出的方法是基于一个可分解的超网络，学习网格明智的预测，帮助专门的区域在不同的任务的权重矩阵。为了构建建议的超网络，我们的方法学习全局（任务不可知）状态和局部任务特定状态之间的相互作用和组合。我们将我们提出的\textsc{HyperGrid}应用于当前最先进的T5模型，当仅使用单个多任务模型时，在GLUE和SuperGLUE基准测试中表现出强大的性能。我们的方法有助于弥合微调和多任务学习方法之间的差距。
英文摘要：
Achieving state-of-the-art performance on natural language understanding tasks typically relies on fine-tuning a fresh model for every task. Consequently, this approach leads to a higher overall parameter cost, along with higher technical maintenance for serving multiple models. Learning a single multi-task model that is able to do well for all the tasks has been a challenging and yet attractive proposition. In this paper, we propose \textsc{HyperGrid}, a new approach for highly effective multi-task learning. The proposed approach is based on a decomposable hypernetwork that learns grid-wise projections that help to specialize regions in weight matrices for different tasks. In order to construct the proposed hypernetwork, our method learns the interactions and composition between a global (task-agnostic) state and a local task-specific state. We apply our proposed \textsc{HyperGrid} on the current state-of-the-art T5 model, demonstrating strong performance across the GLUE and SuperGLUE benchmarks when using only a single multi-task model. Our method helps bridge the gap between fine-tuning and multi-task learning approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
