While explainable artificial intelligence (XAI) for large language models (LLMs) remains an evolving field with many unresolved questions, increasing regulatory pressures have spurred interest in its role in ensuring transparency, accountability, and privacy-preserving machine unlearning. Despite recent advances in XAI have provided some insights, the specific role of attention layers in transformer based LLMs remains underexplored. This study investigates the memory mechanisms instantiated by attention layers, drawing on prior research in psychology and computational psycholinguistics that links Transformer attention to cue based retrieval in human memory. In this view, queries encode the retrieval context, keys index candidate memory traces, attention weights quantify cue trace similarity, and values carry the encoded content, jointly enabling the construction of a context representation that precedes and facilitates memory retrieval. Guided by the Encoding Specificity Principle, we hypothesize that the cues used in the initial stage of retrieval are instantiated as keywords. We provide converging evidence for this keywords-as-cues hypothesis. In addition, we isolate neurons within attention layers whose activations selectively encode and facilitate the retrieval of context-defining keywords. Consequently, these keywords can be extracted from identified neurons and further contribute to downstream applications such as unlearning.
Bayesian Attention Mechanism: A Probabilistic Framework for Positional Encoding and Context Length Extrapolation
MALTA – Machine Learning Theory and Applications Lab, PUCRS – Porto Alegre, Brazil(MALTA机器学习理论与应用实验室,PUCRS-波士顿实验室,巴西);Kunumi Institute, Brazil(昆米研究所,巴西)
Remember to Forget: Gated Adaptive Positional Encoding
Department of Computer Science and Technology, University of Cambridge(计算机科学与技术系,剑桥大学);Department of Computer, Control and Management Engineering, Sapienza University(计算机、控制与管理工程系,萨皮恩扎大学);Department of Information Engineering, University of Padova(信息工程系,帕多瓦大学)
A Multimodal RAG Framework for Housing Damage Assessment: Collaborative Optimization of Image Encoding and Policy Vector Retrieval
University of California, Berkeley(加州大学伯克利分校);Blum Center for Developing Economies(布鲁姆发展中经济体中心);Icahn School of Medicine at Mount Sinai(西奈山伊坎医学院);McKelvey School of Engineering, Washington University in St. Louis(圣路易斯华盛顿大学麦凯维工程学院)
Causality-Induced Positional Encoding for Transformer-Based Representation Learning of Non-Sequential Features
Department of Computer Science, Emory University(埃默里大学计算机科学系);School of Statistics and Mathematics, Zhongnan University of Economics and Law(中南财经政法大学统计与数学学院);School of Medicine, Department of Human Genetics, Emory University(埃默里大学医学院人类遗传学系)