Decoder Tuning:将高效语言理解作为解码过程
Decoder Tuning: Efficient Language Understanding as Decoding
AI总结:
针对模型即服务下输入端适配缺乏梯度且查询成本高的问题,提出Decoder Tuning(DecT),通过在输出端优化任务特定的解码器网络,仅需几秒训练和单次查询,即以200倍加速显著超越现有算法。
摘要:
随着预训练模型(PTMs)规模的不断增长,仅向用户提供推理API的做法日益普遍,即模型即服务(MaaS)设置。为了在模型参数冻结的情况下适配PTMs,目前的大多数方法侧重于输入端,寻求强大的提示来激发模型给出正确答案。然而,我们认为由于缺乏梯度信号,输入端适配可能很困难,并且它们通常需要数千次API查询,导致高昂的计算和时间成本。鉴于此,我们提出了Decoder Tuning(DecT),相反地在输出端优化任务特定的解码器网络。具体而言,DecT首先提取由提示激发的输出分数以进行初始预测。在此基础上,我们在输出表示上训练一个额外的解码器网络,以整合后验数据知识。通过基于梯度的优化,DecT可以在几秒内完成训练,并且每个样本仅需一次PTM查询。在实验中,我们进行了广泛的自然语言理解实验,结果表明DecT显著优于当前最先进的算法,并实现了$200\ imes$的加速。
英文摘要:
With the evergrowing sizes of pre-trained models (PTMs), it has been an emerging practice to only provide the inference APIs for users, namely model-as-a-service (MaaS) setting. To adapt PTMs with model parameters frozen, most current approaches focus on the input side, seeking for powerful prompts to stimulate models for correct answers. However, we argue that input-side adaptation could be arduous due to the lack of gradient signals and they usually require thousands of API queries, resulting in high computation and time costs. In light of this, we present Decoder Tuning (DecT), which in contrast optimizes task-specific decoder networks on the output side. Specifically, DecT first extracts prompt-stimulated output scores for initial predictions. On top of that, we train an additional decoder network on the output representations to incorporate posterior data knowledge. By gradient-based optimization, DecT can be trained within several seconds and requires only one PTM query per sample. Empirically, we conduct extensive natural language understanding experiments and show that DecT significantly outperforms state-of-the-art algorithms with a $200\times$ speed-up.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。