As scaling laws push the training of frontier large language models (LLMs) toward ever-growing data requirements, training pipelines are approaching a regime where much of the publicly available online text may be consumed. At the same time, widespread LLM usage increases the volume of machine-generated content on the web; together, these trends raise the likelihood of generated text re-entering future training corpora, increasing the associated risk of performance degradation often called model collapse. In practice, model developers address this concern through data cleaning, watermarking, synthetic-data policies, or, in some cases, blissful ignorance. However, the problem of model collapse in generative models has not been examined from a learning-theoretic perspective: we study it through the theoretical lens of the language generation in the limit framework, introducing a replay adversary that augments the example stream with the generator's own past outputs. Our main contribution is a fine-grained learning-theoretic characterization of when replay fundamentally limits generation: while replay is benign for the strongest notion of uniform generation, it provably creates separations for the weaker notions of non-uniform generation and generation in the limit. Interestingly, our positive results mirror heuristics widely used in practice, such as data cleaning, watermarking, and output filtering, while our separations show when these ideas can fail.
Rooted Absorbed Prefix Trajectory Balance with Submodular Replay for GFlowNet Training
Courant Institute School of Mathematics, Computing, and Data Science, New York University(纽约大学Courant研究所数学、计算与数据科学学院);Courant Institute School of Mathematics, Computing(纽约大学Courant研究所数学、计算与数据科学学院);Data Science, New York University(纽约大学数据科学学院)
Drift and Dependence: Layer-wise Information-Theoretic Bounds for Replay-Based Continual Learning
School of Computer Science and Technology, Xi’an Jiaotong University(西安交通大学计算机科学与技术学院);Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系)
Information-Theoretic Generalization Bounds of Replay-based Continual Learning
School of Computer Science and Technology, Xi’an Jiaotong University(西安交通大学计算机科学与技术学院);Department of Oncology, University of Cambridge(剑桥大学肿瘤学系);China Telecom, China(中国电信);Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系)
Provable Effects of Data Replay in Continual Learning: A Feature Learning Perspective
School of Information Science and Technology, USTC and Institute of Artificial Intelligence, HCNSC(信息科学与技术学院,中国科学技术大学及人工智能研究所);Department of Computer Science and Engineering, SUNY at Buffalo(计算机科学与工程系,纽约州立大学布法罗分校)
Replay What Matters: Off-Policy Replay for Efficient LLM Reinforcement Unlearning
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Glasgow(格拉斯哥大学)
Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University(多媒体信息处理国家重点实验室,计算机学院,北京大学);LLM-Core Xiaomi(小米LLM-Core)
Exploring Information-Theoretic Metrics Associated with Neural Collapse in Supervised Training
School of Computer and Communication Engineering, University of Science and Technology Beijing(北京科技大学计算机与通信工程学院);Tsinghua University(清华大学);Shanghai AI Laboratory(上海人工智能实验室)
Universitat Politècnica de Catalunya(加泰罗尼亚理工大学);University of Oxford(牛津大学);Barcelona Supercomputing Center(巴塞罗那超级计算中心);Institut de Ciències del Mar(海洋科学研究所)