AdapLeR：通过自适应长度缩减加速推理
AdapLeR: Speeding up Inference by Adaptive Length Reduction
摘要：
预训练的语言模型在各种下游任务中表现出出色的性能。但是，这通常以高延迟和计算为代价，阻碍了它们在资源有限的环境中的使用。在这项工作中，我们提出了一种新的方法来减少BERT的计算成本，在下游性能的损失最小。我们的方法通过层动态地消除了贡献较少的令牌，从而缩短了长度，从而降低了计算成本。为了确定每个标记表示的重要性，我们使用基于梯度的显着性方法为每个层训练一个贡献预测器。我们在几个不同的分类任务上的实验显示，在推理时间内，速度提高了22倍，而性能没有太大的牺牲。我们还使用ERASER基准测试中的人类注释来验证我们方法中所选令牌的质量。与其他广泛使用的选择重要标记的策略（如显着性和注意力）相比，我们提出的方法在生成合理性时具有显着较低的误报率。我们的代码可以在https://github.com/amodaresi/AdapLeR上免费获得。
英文摘要：
Pre-trained language models have shown stellar performance in various downstream tasks. But, this usually comes at the cost of high latency and computation, hindering their usage in resource-limited settings. In this work, we propose a novel approach for reducing the computational cost of BERT with minimal loss in downstream performance. Our method dynamically eliminates less contributing tokens through layers, resulting in shorter lengths and consequently lower computational cost. To determine the importance of each token representation, we train a Contribution Predictor for each layer using a gradient-based saliency method. Our experiments on several diverse classification tasks show speedups up to 22x during inference time without much sacrifice in performance. We also validate the quality of the selected tokens in our method using human annotations in the ERASER benchmark. In comparison to other widely used strategies for selecting important tokens, such as saliency and attention, our proposed method has a significantly lower false positive rate in generating rationales. Our code is freely available at https://github.com/amodaresi/AdapLeR .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
