发表机构
School of Civil and Commercial Law, Southwest University of Political Science and Law(西南政法大学民商法学院)
机构由 AI 辅助整理,请以论文原文为准。AI 中文总结
该研究提出耦合缩放框架,用于解释相同数据训练的系统因架构或优化差异导致的缩放不同,推导残差指数公式并提出两项检验,确定了分离测量几何与缩放拟合的析因检验所需控制变量。
AI 中文摘要
现有理论从数据几何或特定的数据-模型频谱推导神经缩放规律,但在相同数据上训练的系统,当架构或优化改变其可高效获取的表征时,缩放表现会有所不同。我们提出了耦合缩放(Coupled Scaling),这是一种任务条件化框架,其中有限预算下的缩放取决于任务结构与架构-优化系统可访问几何之间的关系。在可解的模式截断模型中,损失可分为架构支持范围外的目标能量和未解决的支持尾部。对于任意优先级顺序,残差介于最佳N个支持尾部与最大完成高价值前缀之外的尾部之间。若累积尾部和覆盖率对数率为γ_{A,T}和ρ_{A,O,T},则残差指数位于[ρ_{A,O,T}γ_{A,T},γ_{A,T}]区间内。在受限前缀外增益条件下,完成的前缀起速率决定作用,此时α_{A,O,T}=ρ_{A,O,T}γ_{A,T};当a_{A,T,j}≍j^{-b_{A,T}}时,可得α_{A,O,T}=ρ_{A,O,T}(b_{A,T}-1)。固定核特化方法从与损失拟合无关的任务加权谱测度的近零尾部推导训练时间指数。该框架将架构支持与有限预算获取分离开来,并提出两项检验:静态任务相关几何应在共同预算下跟踪损失,而多尺度几何应跟踪特定耦合的指数排序,包括在不同任务间的反转。对已发布涌现轨迹的审查确定了直接析因检验所需的控制变量,该检验可将几何与缩放拟合分开测量。
英文摘要
We ask when two learning systems trained on the same task under a common resource protocol should share a scaling rate and when their rates should differ. Coupled Scaling answers this through representational accessibility: the task-relevant geometry that a specified architecture-optimization system can reach and how that geometry is acquired as resources grow. In an orthogonal model, unsupported target energy sets the asymptotic floor, while unacquired supported energy sets the finite-budget residual. When acquisition can skip high-value directions, the largest fully acquired prefix no longer determines a unique exponent. For target powers $a_j \asymp j^{-b}$, $b>1$, and prefix log-growth rate $0<ρ\leq1$, the sharp attainable interval is $[ρ(b-1),\min\{b-1,ρb\}]$, and every rate in this interval is realized by a fixed acquisition order on a common support. A rank-window condition identifies when the product law $α=ρ(b-1)$ applies, while a fixed-kernel specialization expresses the same task-system dependence through the task-weighted spectral tail. We then audit released capability trajectories to diagnose measurement and comparison effects and propose a staged test in which independently measured geometry predicts held-out loss, scaling rates, and cross-task reversals.
Comments29 pages, 2 figures. Reanalysis code and reproducibility artifacts: https://github.com/quintonvina/coupled-scaling/tree/v1.0-reanalysis