Large Language Models (LLMs), trained on massive corpora with billions of parameters, show unprecedented performance in various fields. Though surprised by their excellent performances, researchers also noticed some special behaviors of those LLMs. One of those behaviors is memorization, in which LLMs can generate the same content used to train them. Though previous research has discussed memorization, the memorization of LLMs still lacks explanation, especially the cause of memorization and the dynamics of generating them. In this research, we comprehensively discussed memorization from various perspectives and extended the discussion scope to not only just the memorized content but also less and unmemorized content. Through various studies, we found that: (1) Through experiments, we revealed the relation of memorization between model size, continuation size, and context size. Further, we showed how unmemorized sentences transition to memorized sentences. (2) Through embedding analysis, we showed the distribution and decoding dynamics across model size in embedding space for sentences with different memorization scores. The n-gram statistics analysis presents d (3) An analysis over n-gram and entropy decoding dynamics discovered a boundary effect when the model starts to generate memorized sentences or unmemorized sentences. (4)We trained a Transformer model to predict the memorization of different models, showing that it is possible to predict memorizations by context.
The Reasoning-Memorization Interplay in Language Models Is Mediated by a Single Direction
New York University(纽约大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);McGill University(麦吉尔大学);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);University of Toronto(多伦多大学);Vector Institute(矢量研究所)
Emergent and Predictable Memorization in Large Language Models
Booz Allen Hamilton(博思艾伦汉密尔顿);EleutherAI;Yale University(耶鲁大学);Ohio State University(俄亥俄州立大学);Stability AI;University of Maryland, Baltimore County(马里兰大学巴尔的摩县分校)
Unintended Memorization of Sensitive Information in Fine-Tuned Language Models
TUM University Hospital(慕尼黑工业大学医院);Technical University of Munich (TUM)(慕尼黑技术大学);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Department of Computing, Imperial College London(伦敦帝国学院计算系)
Memorization in Large Language Models in Medicine: Prevalence, Characteristics, and Implications
Yale University(耶鲁大学);Imperial College London(帝国理工学院);University of Texas Health Science at Houston(德克萨斯大学休斯顿健康科学中心);University of California, San Diego(加利福尼亚大学圣迭戈分校);National Library of Medicine, National Institutes of Health(美国国立卫生研究院国家医学图书馆);Vanderbilt University Medical Center(范德堡大学医学中心);Vanderbilt University(范德堡大学);National University of Singapore(新加坡国立大学)
Memorization $\neq$ Understanding: Do Large Language Models Have the Ability of Scenario Cognition?
School of Computer and Information Technology, Shanxi University(山西大学计算机与信息学院);Information Systems Technology and Design, Singapore University of Technology and Design(新加坡科技设计大学信息系统技术与设计)
Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Data
University of California, Santa Barbara(加利福尼亚大学圣巴巴拉分校);Allen Institute for AI(艾伦人工智能研究所);University of Washington(华盛顿大学);SynthLabs;Carnegie Mellon University(卡内基梅隆大学)
College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);State Key Laboratory of Transvascular Implantation Devices and TIDRI(血管植入设备国家重点实验室和TIDRI);Zhejiang Key Laboratory of Medical Imaging Artificial Intelligence(浙江医学影像人工智能重点实验室);School of Data Science of Engineering, East China Normal University(华东师范大学工程数据科学学院);Second Affiliated Hospital and Liangzhu Laboratory, Zhejiang University School of Medicine(浙江大学医学院第二附属医院和良渚实验室);Alibaba Group(阿里巴巴集团)
FastMem: Fast Memorization of Prompt Improves Context Awareness of Large Language Models
KU Leuven(KU Leuven(天主教鲁汶大学));University of Science and Technology of China(中国科学技术大学);Institute for Advanced Algorithms Research, Shanghai(上海高级算法研究院);National University of Singapore(新加坡国立大学)
Unlocking Memorization in Large Language Models with Dynamic Soft Prompting
George Mason University(乔治梅森大学);GE Healthcare(GE医疗);University of Pittsburgh(匹兹堡大学);William and Mary(威廉玛丽学院);Southern University of Science and Technology(南方科技大学);Florida State University(佛罗里达州立大学)
Beyond Memorization: The Challenge of Random Memory Access in Language Models
Sea AI Lab(Sea AI实验室);National University of Singapore(新加坡国立大学);Carnegie Mellon University(卡内基梅隆大学);Institute of Computing Technology, CAS(中国科学院计算技术研究所)
KMIR: A Benchmark for Evaluating Knowledge Memorization, Identification and Reasoning Abilities of Language Models
Huawei Poisson Laboratory(华为泊松实验室);Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Hong Kong University of Science and Technology(香港科技大学)
Reason to Rote: Rethinking Memorization in Reasoning
Department of ICS, Utrecht University(信息科学与计算系,乌特勒支大学);MaiNLP, Center for Information and Language Processing, LMU Munich(MaiNLP,信息与语言处理中心,慕尼黑大学);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Saarland Informatics Campus, Saarland University(萨尔兰州信息科学校区,萨尔兰州大学)
Beyond Frequency: The Role of Redundancy in Large Language Model Memorization
Semyung University(_semyung大学);University of Copenhagen(哥本哈根大学);University of Washington(华盛顿大学);SAIBD, Hefei University(合肥大学);School of Information Science and Engineering, Lanzhou University(兰州大学信息科学与工程学院)
Beyond Memorization: Assessing Semantic Generalization in Large Language Models Using Phrasal Constructions
Georgetown University(乔治城大学);University of Bath(巴斯大学);DEVCOM U.S. Army Research Laboratory(美国陆军研究实验室);University of Maryland, College Park(马里兰大学学院公园分校)