arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2108.02765cs.CL
收藏

开放域问答中可扩展推理的解耦Transformer

Decoupled Transformer for Scalable Inference in Open-domain Question Answering

Haytham ElFadeel, Stan Peshterliev

摘要:

大型Transformer模型,如BERT,在开放域问答(QA)的机器阅读理解(MRC)中实现了最先进的结果。然而,Transformers具有很高的推理计算成本,这使得它们很难应用于语音助理等应用的在线QA系统。为了减少计算成本和延迟,我们提出了将Transformer MRC模型解耦为输入分量和交叉分量。解耦允许部分表示计算离线执行并缓存以供在线使用。为了保持解耦的Transformer的准确性,我们从标准的Transformer模型设计了一个知识蒸馏目标。此外,我们引入了学习表示压缩层,这有助于将缓存的存储需求减少四倍。在SQUAD 2.0数据集上的实验中,与标准Transformer相比,解耦的Transformer将开域MRC的计算成本和延迟降低了30-40%,F1分数仅差1.2分。

英文摘要:

Large transformer models, such as BERT, achieve state-of-the-art results in machine reading comprehension (MRC) for open-domain question answering (QA). However, transformers have a high computational cost for inference which makes them hard to apply to online QA systems for applications like voice assistants. To reduce computational cost and latency, we propose decoupling the transformer MRC model into input-component and cross-component. The decoupling allows for part of the representation computation to be performed offline and cached for online use. To retain the decoupled transformer accuracy, we devised a knowledge distillation objective from a standard transformer model. Moreover, we introduce learned representation compression layers which help reduce by four times the storage requirement for the cache. In experiments on the SQUAD 2.0 dataset, a decoupled transformer reduces the computational cost and latency of open-domain MRC by 30-40% with only 1.2 points worse F1-score compared to a standard transformer.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑