We investigate the origins of massive activations in large language models (LLMs) and identify a specific layer named the \textbf{Massive Emergence Layer (ME Layer)}, that is consistently observed across model families, where massive activations first emerge and subsequently propagate to deeper layers through residual connections. We show that, within the ME Layer both the RMSNorm and the FFN parameters jointly contribute to the emergence of massive activations. Once formed, the massive activation token representation remains largely invariant across layers, reducing the diversity of hidden representations passed to the attention module. Motivated by this limitation, we propose a simple and effective method to reduce the rigidity of the massive activation token. Our approach consistently improves LLM performance across multiple tasks, including instruction following and math reasoning, in both training free and fine tuning settings. Moreover, we show that our method mitigates attention sinks by selectively weakening their influence, elucidating their origin at the hidden state level and shedding new light on principled mitigation strategies.
Massive Activations in Hybrid Linear Attention Large Language Models: Pre-Attention Spikes and Inter-Spike Plateaus
Startlux(星创公司);Tsinghua University(清华大学);University of Chinese Academy of Sciences(中国科学院大学);The University of Hong Kong(香港大学);University of Sydney(悉尼大学);Columbia University(哥伦比亚大学)
NeuronScope: A Multi-Agent Framework for Explaining Polysemantic Neurons in Language Models
Wuhan University(武汉大学);Hong Kong Baptist University(香港 Baptist大学);New Jersey Institute of Technology(新泽西理工学院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))
DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMs
Institute of Automation, CAS(中国科学院自动化研究所);City University of Hong Kong(香港城市大学);Tsinghua University(清华大学);Zhejiang University(浙江大学);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
Circuit Insights: Towards Interpretability Beyond Activations
Department of Artificial Intelligence, Fraunhofer Heinrich Hertz Institute(人工智能系,弗劳恩霍夫海因里希·赫兹研究所);Department of Electrical Engineering and Computer Science, Technische Universität Berlin(电气工程与计算机科学系,柏林技术大学);BIFOLD - Berlin Institute for the Foundations of Learning and Data(柏林学习与数据基础研究所);Centre of eXplainable Artificial Intelligence, Technological University Dublin(可解释人工智能中心,都柏林技术大学)
Universal Activation Verbalizer: A Unified Framework for Cross-Model Activation Explanation
New Jersey Institute of Technology(新泽西理工学院);University of North Carolina at Charlotte(北卡罗来纳大学夏洛特分校);Cisco Research(思科研究);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))