arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1910.06188cs.CLcs.LG
收藏

Q8BERT:量化8位BERT

Q8BERT: Quantized 8Bit BERT

Intel AI Lab(英特尔人工智能实验室)

Ofir Zafrir, Guy Boudoukh, Peter Izsak, Moshe Wasserblat

摘要:

最近,预训练的基于Transformer的语言模型,如BERT和GPT,在许多自然语言处理(NLP)任务中表现出很大的改进。然而,这些模型包含大量的参数。更大更精确的模型的出现,如GPT2和Megatron,表明大型预训练Transformer模型的趋势。然而,在生产环境中使用这些大型模型是一项复杂的任务,需要大量的计算、内存和电源资源。在这项工作中,我们展示了如何在BERT的微调阶段执行量化感知训练,以便通过$4\times$压缩BERT,同时将准确性损失降到最低。此外,所产生的量化模型,可以加快推理速度,如果它是最佳的支持8位的硬件。

英文摘要:

Recently, pre-trained Transformer based language models such as BERT and GPT, have shown great improvement in many Natural Language Processing (NLP) tasks. However, these models contain a large amount of parameters. The emergence of even larger and more accurate models such as GPT2 and Megatron, suggest a trend of large pre-trained Transformer models. However, using these large models in production environments is a complex task requiring a large amount of compute, memory and power resources. In this work we show how to perform quantization-aware training during the fine-tuning phase of BERT in order to compress BERT by $4\times$ with minimal accuracy loss. Furthermore, the produced quantized model can accelerate inference speed if it is optimized for 8bit Integer supporting hardware.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑