Transformers commonly exhibit an attention sink: disproportionately high attention to the first position. We study this behavior in GPT-2-style models with learned query biases and absolute positional embeddings. Combining structural analysis with causal interventions, validated across natural-language, mathematical, and code inputs, we find that the sink arises from the interaction among (i) a learned query bias, (ii) the first-layer MLP transformation of the positional encoding, and (iii) structure in the key projection. Crucially, each component we identify is individually dispensable: architectures omitting each of them robustly exhibit sinks. This indicates that attention sinks may arise through distinct circuits across architectures. These findings inform mitigation of sinks, and motivate broader investigation into why sinks emerge.
Attention Sink in Transformers: A Survey on Utilization, Interpretation, and Mitigation
Tsinghua University(清华大学);Meituan LongCat Team(美团LongCat团队);The University of Hong Kong(香港大学);University of Michigan(密歇根大学);Xiamen University(厦门大学);The Ohio State University(俄亥俄州立大学);Columbia University(哥伦比亚大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);The Hong Kong Polytechnic University(香港理工大学)
Attention Sinks and Outliers in Attention Residuals
Department of Computer Science, Northwestern University(西北大学计算机科学系);Department of Computer Science and Engineering, University of Michigan(密歇根大学计算机科学与工程系);Department of Statistics and Data Science, University of California Los Angeles(加州大学洛杉矶分校统计与数据科学系);Department of Electrical and Computer Engineering, University of California San Diego(加州圣地亚哥大学电气与计算机工程系);Department of Computer Science, Rutgers University-New Brunswick(新泽西州立大学鲁特学院计算机科学系);Department of Computer Science and Engineering, Texas A&M University(德克萨斯农工大学计算机科学与工程系);Department of Computer Science, Columbia University(哥伦比亚大学计算机科学系)
Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias
AMAP, Alibaba Group(阿里巴巴集团AMAP实验室);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);University of Southern California(南加州大学);University of Michigan, Ann Arbor(密歇根大学安娜堡分校)