arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.11239cs.CL
收藏

DQ-BART:基于联合蒸馏和Quantization的高效序列到序列模型

DQ-BART: Efficient Sequence-to-Sequence Model via Joint Distillation and Quantization

AWS AI Labs(亚马逊云科技人工智能实验室);University of Pennsylvania(宾夕法尼亚大学);Cornell University(康奈尔大学)

Zheng Li, Zijian Wang, Ming Tan, Ramesh Nallapati, Parminder Bhatia, Andrew Arnold, Bing Xiang, Dan Roth

摘要:

像BART和T5这样的大规模预训练序列到序列模型在许多生成NLP任务上实现了最先进的性能。然而,这种模型在资源受限的情况下,由于其大的内存需求和高延迟带来了很大的挑战。为了缓解这个问题,我们建议联合蒸馏和蒸馏模型,其中知识从全精度教师模型转移到量化和蒸馏的低精度学生模型。实证分析表明,尽管生成任务具有挑战性,但我们能够在多个摘要和QA数据集上实现16.5倍的模型足迹压缩比,相对于全精度同行,性能下降很少。我们进一步将压缩比的限制提高到27.7倍,并使用预训练模型为生成任务提供了性能-效率权衡。据我们所知,这是第一项旨在有效地提取和重新排序用于语言生成任务的序列到序列预训练模型的工作。

英文摘要:

Large-scale pre-trained sequence-to-sequence models like BART and T5 achieve state-of-the-art performance on many generative NLP tasks. However, such models pose a great challenge in resource-constrained scenarios owing to their large memory requirements and high latency. To alleviate this issue, we propose to jointly distill and quantize the model, where knowledge is transferred from the full-precision teacher model to the quantized and distilled low-precision student model. Empirical analyses show that, despite the challenging nature of generative tasks, we were able to achieve a 16.5x model footprint compression ratio with little performance drop relative to the full-precision counterparts on multiple summarization and QA datasets. We further pushed the limit of compression ratio to 27.7x and presented the performance-efficiency trade-off for generative tasks using pre-trained models. To the best of our knowledge, this is the first work aiming to effectively distill and quantize sequence-to-sequence pre-trained models for language generation tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑