DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference

DE$^3$-BERT: 基于原型网络的距离增强型BERT早期退出

DE$^3$-BERT: Distance-Enhanced Early Exiting for BERT based on Prototypical Networks

DE$^3$-BERT通过结合局部和全局信息提升BERT的早期退出效果，实验表明其在不同加速比例下性能更优。

SmartBERT：一种促进动态提前退出机制以加速BERT推理的方法

SmartBERT: A Promotion of Dynamic Early Exiting Mechanism for Accelerating BERT Inference

SmartBERT通过在每个BERT层添加跳过门和退出操作符，结合跨层对比学习与硬权重机制，实现动态层跳过与提前退出，在GLUE八个数据集上减少2-3倍计算量且准确率损失极小。

RomeBERT: Robust Training of Multi-Exit BERT

BERT has achieved superior performances on Natural Language Understanding (NLU) tasks. However, BERT possesses a large number of parameters and demands certain resources to deploy. For acceleration, Dynamic Early Exiting for BERT (DeeBERT) has been proposed recently, which incorporates multiple exits and adopts a dynamic early-exit mechanism to ensure efficient inference. While obtaining an efficiency-performance tradeoff, the performances of early exits in multi-exit BERT are significantly worse than late exits. In this paper, we leverage gradient regularized self-distillation for RObust trai…

Magic Pyramid: Accelerating Inference with Early Exiting and Token Pruning

Pre-training and then fine-tuning large language models is commonly used to achieve state-of-the-art performance in natural language processing (NLP) tasks. However, most pre-trained models suffer from low inference speed. Deploying such large models to applications with latency constraints is challenging. In this work, we focus on accelerating the inference via conditional computations. To achieve this, we propose a novel idea, Magic Pyramid (MP), to reduce both width-wise and depth-wise computation via token pruning and early exiting for Transformer-based models, particularly BERT. The forme…

COST-EFF: Collaborative Optimization of Spatial and Temporal Efficiency with Slenderized Multi-exit Language Models

Transformer-based pre-trained language models (PLMs) mostly suffer from excessive overhead despite their advanced capacity. For resource-constrained devices, there is an urgent need for a spatially and temporally efficient model which retains the major capacity of PLMs. However, existing statically compressed models are unaware of the diverse complexities between input instances, potentially resulting in redundancy and inadequacy for simple and complex inputs. Also, miniature models with early exiting encounter challenges in the trade-off between making predictions and serving the deeper layer…

EPEE：迈向生物医学中的高效与有效基础模型

EPEE: Towards Efficient and Effective Foundation Models in Biomedicine

针对基础模型在生物医学推理中的高延迟与过度思考问题，提出混合提前退出策略EPEE，在保持或提升准确性的同时显著降低推理时间，为实时临床决策提供高效可靠的解决方案。