Plug-Tagger：一个基于语言模型的可插入序列标记框架
Plug-Tagger: A Pluggable Sequence Labeling Framework Using Language Models
摘要：
即插即用功能使深度学习模型能够很好地适应不同的任务，而无需修改任何参数。最近，前缀调优被证明是一种即插即用的方法，通过简单地将相应的连续向量插入到输入中来完成各种文本生成任务。然而，序列标记任务无效现有的即插即用的方法，因为不同的标签集的需求变化的模型分类器的架构。在这项工作中，我们提出使用标签词预测而不是分类，以完全重用序列标记任务的预训练模型的架构。具体而言，对于每个任务，首先通过为每个类别分别选择高频词来构建标签词集合，然后，将任务特定的向量插入到输入中并进行优化以操纵模型预测朝向对应的标签词。因此，通过简单地切换输入上的插件向量，允许冻结的预训练语言模型执行不同的任务。在三个序列标记任务上的实验结果表明，该方法的性能可以达到与标准微调相当的性能，仅需0.1%的任务特定参数。此外，我们的方法比非即插即用方法快70倍，同时在资源受限的情况下切换不同的任务。
英文摘要：
Plug-and-play functionality allows deep learning models to adapt well to different tasks without requiring any parameters modified. Recently, prefix-tuning was shown to be a plug-and-play method on various text generation tasks by simply inserting corresponding continuous vectors into the inputs. However, sequence labeling tasks invalidate existing plug-and-play methods since different label sets demand changes to the architecture of the model classifier. In this work, we propose the use of label word prediction instead of classification to totally reuse the architecture of pre-trained models for sequence labeling tasks. Specifically, for each task, a label word set is first constructed by selecting a high-frequency word for each class respectively, and then, task-specific vectors are inserted into the inputs and optimized to manipulate the model predictions towards the corresponding label words. As a result, by simply switching the plugin vectors on the input, a frozen pre-trained language model is allowed to perform different tasks. Experimental results on three sequence labeling tasks show that the performance of the proposed method can achieve comparable performance with standard fine-tuning with only 0.1\% task-specific parameters. In addition, our method is up to 70 times faster than non-plug-and-play methods while switching different tasks under the resource-constrained scenario.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
