指导现代模型训练的扩展定律最初是在数据丰富的单一周期预训练条件下校准的。主导的扩展定律形式，即Chinchilla的L = E + A/N^α + B/D^β，存在三个结构性限制：当唯一数据减少时，它会发散而非在无信息基线处饱和；当容量超过数据时，它无法表示过拟合；它将总示例数与可用唯一示例数混为一谈。我们提出了一种闭式扩展，L(N, D, T) = E + (L_0 - E)h/(1+h) with h = a/N^α + b/T^β + cN^γ/D^δ，将损失分解为欠容量、欠训练和过拟合项。它在不可约损失E和由损失类型固定的无信息基线L_0之间饱和，并在数据丰富的单一周期极限下还原为Chinchilla。我们验证了该公式在四个跨四个架构家族（MLPs、ResNets、傅里叶神经算子和转换器）的多周期实验中，覆盖视觉、科学ML和语言领域，并重新拟合了五个已发布LLM扩展定律网格。外推到比拟合时间看到的计算更高和唯一数据更大的情况下，我们的形式在我们评估的每个已发布LLM网格和我们构建的大多数实验单元中均实现了最先进的RMSE。一旦校准，该形式允许成本意识的分配，当数据免费时恢复Chinchilla的最优解，并在数据变得昂贵时转向更小的语料库和更多的周期。
英文摘要：
The scaling laws guiding modern model training were calibrated for a single regime: data-rich, single-epoch pretraining. The dominant such scaling law form, Chinchilla's $L = E + A/N^α+ B/D^β$, has three structural limitations outside that regime: it diverges as unique data shrinks instead of saturating at the uninformed baseline; it cannot represent overfitting when capacity exceeds the data; and it conflates total examples seen with unique examples available. We propose a closed-form extension, $L(N, D, T) = E + (L_0 - E)\,h/(1+h)$ with $h = a/N^α+ b/T^β+ c\,N^γ/D^δ$, that decomposes loss into undercapacity, undertraining, and overfitting terms. It saturates between the irreducible loss $E$ and an uninformed baseline $L_0$ fixed by the loss type, and reduces to Chinchilla in the data-rich, single-epoch limit. We validate it on four multi-epoch experiments spanning four architecture families (MLPs, ResNets, Fourier neural operators, and transformers) across vision, scientific ML, and language domains, and refit it to five published LLM scaling-law grids. Extrapolating to higher compute and larger unique data than seen at fit time, our form achieves state-of-the-art RMSE on every published LLM grid we evaluate and on most cells of our constructed experiments. Once calibrated, the form admits a cost-aware allocation that recovers Chinchilla's optimum when data is free and shifts toward smaller corpora and more epochs as data grows expensive.
Adaptive Correction for Ensuring Conservation Laws in Neural Operators
Department of Applied Mathematics and Theoretical Physics, University of Cambridge, UK（应用数学与理论物理系，剑桥大学）；Department of Mathematical Sciences, University of Bath, UK（数学科学系，巴斯大学）
Projected Neural Differential Equations for Learning Constrained Dynamics
Technical University of Munich（慕尼黑工业大学）；Potsdam Institute for Climate Impact Research（波茨坦气候影响研究所）；California Institute of Technology（加州理工学院）；Helmholtz Munich（亥姆霍兹慕尼黑中心）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；University of Exeter（埃克塞特大学）
Effective Frontiers: A Unification of Neural Scaling Laws
Department of Mathematics and Statistics, Xi'an Jiaotong University（西安交通大学数学与统计学系）；Department of Artificial Intelligence, Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学人工智能学院）；University of Chinese Academy of Sciences（中国科学院大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
