Scaling laws for language model training traditionally characterize how performance scales with model size and dataset volume. Prior work has explored architecture variants and data treatments such as dataset filtering and noise injection in language model pretraining; however, these studies have not formalized data quality within a principled scaling law. We introduce a dimensionless data-quality parameter Q, and propose a quality-aware scaling law extending the Chinchilla framework to predict loss as a joint function of model size, data volume, and data quality. The law is motivated by an effective-sample-size and information-theoretic view of noisy or redundant corpora, and it admits two practical estimators for Q: (i) a corruption rate proxy and (ii) a deficiency measure. Through synthetic experiments in neural machine translation and autoregressive modeling -- where we systematically control data quality via multiple levels of noise injection variation -- we show that loss scales predictably with data quality and that higher-quality data can substantially reduce model size and hence compute requirements. Our results demonstrate a sublinear decay of effective data with quality and robustness to moderate data corruption; out-of-sample evaluations further validate the predictive form of the law. Unlike prior empirical analyses, our work establishes an explicit, generalizable law for data quality, offering concrete guidance for balancing data curation effort and model scale in large-scale pretraining.
LLMs on the Line: Data Determines Loss-to-Loss Scaling Laws
Max Planck Institute for Intelligent Systems（智能系统马克斯·普朗克研究所）；ELLIS Institute Tübingen（图宾根ELLIS研究所）；Tübingen AI Center（图宾根人工智能中心）；University of Tübingen（图宾根大学）
EfficientLLM: Scalable Pruning-Aware Pretraining for Architecture-Agnostic Edge Language Models
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；University of Oxford（牛津大学）；Peking University（北京大学）
Deriving Scaling Laws for OpenEuroLLM Models: Learning Rate, Batch Size and Loss
Eindhoven University of Technology（埃因霍温理工大学）；Georgia Institute of Technology（佐治亚理工学院）；University of Freiburg（弗赖堡大学）；LAION；Open-Ψ (Open-Sci) Collective（Open-Ψ（开放科学）集体）；University of Turku（图尔库大学）；OpenEuroLLM
Uni-Mol2: Exploring Molecular Pretraining Model at Scale
DP Technology（深势科技）；AI for Science Institute, Beijing（北京AI for Science研究院）；School of Mathematical Sciences, Peking University（北京大学数学科学学院）；Center for Machine Learning Research, Peking University（北京大学机器学习研究中心）
Scaling laws in wearable human activity recognition
Google DeepMind（谷歌DeepMind）；Google LLC School of Engineering & Informatics, University of Sussex, UK（谷歌 LLC 工程与信息学学院，Sussex 大学，英国）；Google LLC Computer Laboratory, University of Cambridge, UK（谷歌 LLC 计算实验室，剑桥大学，英国）；Google Research Queen’s University, Canada（谷歌研究 Queen’s 大学，加拿大）
Physics-Informed Generative Solver: Bridging Data-Driven Priors and Conservation Laws for Stable Spatiotemporal Field Reconstruction
School of Advanced Manufacturing Engineering, Nanjing University（南京大学先进制造工程学院）；National Laboratory of Solid State Microstructures, Nanjing University（南京大学固态微结构国家实验室）；Suzhou Acoustics Industry Technology Research Institute Co., Ltd.（苏州声学工业技术研究所有限公司）；School of Mechanical and Electric Engineering, Soochow University（苏州大学机械与电子工程学院）；Shishan Laboratory, Nanjing University（仙山实验室）
Discovering ordinary differential equations that govern time-series
Helmholtz AI（亥姆霍兹人工智能中心）；Technical University of Munich（慕尼黑工业大学）；Apple（苹果公司）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；DeepMind（深度思维）；OpenAI（开放人工智能）
The Model Shape Behind Scaling Laws: A Gradient Superposition Perspective
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究院）；New Jersey Institute of Technology（新泽西理工学院）
A Nationwide Japanese Medical Claims Foundation Model: Balancing Model Scaling and Task-Specific Computational Efficiency
Graduate School of Medicine, Kyoto University（京都大学医学研究科）；Medical Data Vision Co., Ltd.（医疗数据视觉株式会社）；IQVIA Solutions Japan G.K.（IQVIA日本解决方案公司）；DeSC Healthcare, Inc.（DeSC医疗公司）
Fast Catch-Up, Late Switching: Optimal Batch Size Scheduling via Functional Scaling Laws
School of Mathematical Sciences, Peking University（北京大学数学科学学院）；Center for Machine Learning Research, Peking University（北京大学机器学习研究中心）；School of Computer Science, Shanghai Jiao Tong University（上海交通大学计算机科学学院）；State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China（中国科学技术大学认知智能国家重点实验室）；Meituan, Beijing（美团（北京））；AI for Science Institute, Beijing（北京人工智能科学研究院）
