Jump to Conclusions: 用线性变换为Transformer捷径加速
Jump to Conclusions: Short-Cutting Transformers With Linear Transformations
AI总结:
本文提出用线性变换将Transformer中间层隐藏表示转换为最终表示的方法,可窥探模型早期预测能力,还能提升早期退出策略的层节省效率,且发现注意力模块对线性近似容忍度最高。
摘要:
基于Transformer的语言模型会在每一层生成输入的隐藏表示,但仅使用最后一层的表示进行预测,这使得模型的内部决策过程及中间表示的效用难以被观测。一种厘清该问题的思路是将隐藏表示转换为最终表示,跳过中间的Transformer计算。本文提出一种实现该转换的简单方法——使用线性变换。这种近似效果远超当前在最后层空间中检查所有层隐藏表示的通用做法。此外,在语言建模场景下,我们的方法能在不同模型规模、架构和数据分布下,从隐藏层生成更准确的预测。这使得我们可以“窥探”中间表示,结果显示GPT-2和BERT通常在早期层就已经能预测最终输出。随后我们验证了该方法在近期早期退出策略中的实用性:例如,当目标为保留95%的准确率时,我们的方法为GPT-2额外节省7.9%的层,为BERT额外节省5.4%的层。最后,我们将方法扩展到子模块的线性近似,发现注意力模块对这种变化的容忍度最高。我们的代码和学习到的映射已公开在https://github.com/sashayd/mat。
英文摘要:
Transformer-based language models create hidden representations of their inputs at every layer, but only use final-layer representations for prediction. This obscures the internal decision-making process of the model and the utility of its intermediate representations. One way to elucidate this is to cast the hidden representations as final representations, bypassing the transformer computation in-between. In this work, we suggest a simple method for such casting, using linear transformations. This approximation far exceeds the prevailing practice of inspecting hidden representations from all layers, in the space of the final layer. Moreover, in the context of language modeling, our method produces more accurate predictions from hidden layers, across various model scales, architectures, and data distributions. This allows "peeking" into intermediate representations, showing that GPT-2 and BERT often predict the final output already in early layers. We then demonstrate the practicality of our method to recent early exit strategies, showing that when aiming, for example, at retention of 95% accuracy, our approach saves additional 7.9% layers for GPT-2 and 5.4% layers for BERT. Last, we extend our method to linearly approximate sub-modules, finding that attention is most tolerant to this change. Our code and learned mappings are publicly available at https://github.com/sashayd/mat.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。