基于置信自适应Transformers的一致加速推理
Consistent Accelerated Inference via Confident Adaptive Transformers
摘要：
我们开发了一种新的方法，自信地加速推理的大型和昂贵的多层Transformers，现在无处不在的自然语言处理（NLP）。摊销或近似计算方法提高了效率，但可能带来不可预测的性能成本。在这项工作中，我们提出了CAT-自信的自适应Transformers -在其中，我们同时提高计算效率，同时保证与原始模型的一致性与高置信度的指定程度。我们的方法在中间层之上训练额外的预测头，并使用Meta一致性分类器动态决定何时停止向每个输入分配计算工作。为了校准我们的早期预测停止规则，我们制定了一个独特的扩展共形预测。我们证明了这种方法在四个分类和回归任务的有效性。
英文摘要：
We develop a novel approach for confidently accelerating inference in the large and expensive multilayer Transformers that are now ubiquitous in natural language processing (NLP). Amortized or approximate computational methods increase efficiency, but can come with unpredictable performance costs. In this work, we present CATs -- Confident Adaptive Transformers -- in which we simultaneously increase computational efficiency, while guaranteeing a specifiable degree of consistency with the original model with high confidence. Our method trains additional prediction heads on top of intermediate layers, and dynamically decides when to stop allocating computational effort to each input using a meta consistency classifier. To calibrate our early prediction stopping rule, we formulate a unique extension of conformal prediction. We demonstrate the effectiveness of this approach on four classification and regression tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
