We observe an empirical phenomenon in Large Language Models (LLMs) -- very few activations exhibit significantly larger values than others (e.g., 100,000 times larger). We call them massive activations. First, we demonstrate the widespread existence of massive activations across various LLMs and characterize their locations. Second, we find their values largely stay constant regardless of the input, and they function as indispensable bias terms in LLMs. Third, these massive activations lead to the concentration of attention probabilities to their corresponding tokens, and further, implicit bias terms in the self-attention output. Last, we also study massive activations in Vision Transformers. Code is available at https://github.com/locuslab/massive-activations.
NeuronScope: A Multi-Agent Framework for Explaining Polysemantic Neurons in Language Models
Wuhan University(武汉大学);Hong Kong Baptist University(香港 Baptist大学);New Jersey Institute of Technology(新泽西理工学院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))
Massive Activations in Hybrid Linear Attention Large Language Models: Pre-Attention Spikes and Inter-Spike Plateaus
Startlux(星创公司);Tsinghua University(清华大学);University of Chinese Academy of Sciences(中国科学院大学);The University of Hong Kong(香港大学);University of Sydney(悉尼大学);Columbia University(哥伦比亚大学)
AWEQ: Post-Training Quantization with Activation-Weight Equalization for Large Language Models
School of Computer Science and Technology, Jilin University(吉林大学计算机科学与技术学院);Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University(吉林大学符号计算与知识工程教育部重点实验室)
Interpretable Steering of Large Language Models with Feature Guided Activation Additions
Raffles Science Institute(莱佛士科学学院);Raffles Institution(莱佛士书院);Nous Research(努斯研究机构);Centre for Frontier AI Research (CFAR)(前沿人工智能研究中心);Agency for Science Technology and Research (A*STAR)(新加坡科技研究局)
Unraveling Babel: Exploring Multilingual Activation Patterns of LLMs and Their Applications
Liangzhu Laboratory, Zhejiang University(浙江大学良渚实验室);AI Research Center, WeDoctor Cloud(微医云AI研究中心);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);School of Public Health, Zhejiang University(浙江大学公共卫生学院)
Polynomial Composition Activations: Unleashing the Dynamics of Large Language Models
School of Mathematical Sciences, Peking University(北京大学数学科学学院);Seed-Foundation-Model, ByteDance(字节跳动种子大模型部);Capital University of Economics and Business(首都经济贸易大学)
Enhancing Cross-task Transfer of Large Language Models via Activation Steering
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学耿丽人工智能学院);Department of Computer Science, National University of Singapore(新加坡国立大学计算机科学系);Ant Group(蚂蚁集团)
Internal Value Alignment in Large Language Models through Controlled Value Vector Activation
University of Science and Technology of China(中国科学技术大学);State Key Laboratory of Cognitive Intelligence(认知智能国家重点实验室);Gaoling School of Artificial Intelligence(光明人工智能学院);Renmin University of China(中国人民大学);Beijing Key Laboratory of Research on Large Models and Intelligent Governance(北京大型模型与智能治理重点实验室);Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE(下一代智能搜索与推荐工程技术研究中心);Tsinghua University(清华大学);Huawei Technologies Co. Ltd(华为技术有限公司)
Controlling Large Language Models Through Concept Activation Vectors
Key Lab of Intelligent Information Processing of Chinese Academy of Sciences(中国科学院智能信息处理重点实验室);Key Lab of AI Safety of Chinese Academy of Sciences(中国科学院人工智能安全重点实验室);University of Chinese Academy of Sciences(中国科学院大学);Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院)
Activation-Informed Merging of Large Language Models
Massachusetts Institute of Technology(麻省理工学院);Stony Brook University(石溪大学);MIT-IBM Watson AI Lab & Red Hat AI Innovation(MIT-IBM Watson AI实验室及Red Hat AI创新)