用于深度神经网络注意力机制的数据驱动全局稀疏性
Data-Informed Global Sparseness in Attention Mechanisms for Deep Neural Networks
AI总结:
该研究提出注意力剪枝(AP)框架,通过观测固定数据集的注意力模式生成全局稀疏掩码,可大幅降低不同NLP任务中注意力模型的计算量与内存需求,同时保持性能,还揭示了自注意力与交叉注意力的模式差异。
摘要:
注意力机制在自然语言处理(NLP)的神经革命中发挥着关键作用。随着基于注意力的模型不断发展,研究人员已开发出多种剪枝技术来识别并利用稀疏性,以提升这类模型的效率。现有研究大多聚焦于硬编码注意力模式,或基于训练数据剪枝注意力权重。我们提出了注意力剪枝(Attention Pruning,AP)框架,该框架在固定数据集上观测注意力模式,并生成全局稀疏掩码。在语言建模任务中,AP可节省90%的注意力计算量;在机器翻译和GLUE任务中,可节省约50%的注意力计算量,同时保持结果质量。我们的方法揭示了自注意力模式与交叉注意力模式之间的重要差异,可为未来NLP研究提供指导。该框架能够降低任意基于注意力的模型的延迟与内存需求,助力为现有或新型NLP应用开发更优模型。我们已通过搭载Triton GPU内核的编码器Transformer模型和自回归Transformer模型验证了该框架的有效性,并将代码公开在https://github.com/irugina/AP。
英文摘要:
Attention mechanisms play a crucial role in the neural revolution of Natural Language Processing (NLP). With the growth of attention-based models, several pruning techniques have been developed to identify and exploit sparseness, making these models more efficient. Most efforts focus on hard-coding attention patterns or pruning attention weights based on training data. We propose Attention Pruning (AP), a framework that observes attention patterns in a fixed dataset and generates a global sparseness mask. AP saves 90% of attention computation for language modeling and about 50% for machine translation and GLUE tasks, maintaining result quality. Our method reveals important distinctions between self- and cross-attention patterns, guiding future NLP research. Our framework can reduce both latency and memory requirements for any attention-based model, aiding in the development of improved models for existing or new NLP applications. We have demonstrated this with encoder and autoregressive transformer models using Triton GPU kernels and make our code publicly available at https://github.com/irugina/AP.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。