学习的算法信息动力学：一个经认证的、可微的复杂性控制器用于Grokking

中文摘要 AI 辅助

算法信息动力学（AID）通过扰动系统并测量算法复杂性的变化来研究系统，但其通常的估计器——块分解方法——是分段常数的，将微积分限制在有限差分上。我们使用 $K^{\mathrm{CDM}}_{\mathrm{s}F}$，一个经认证的、可微的估计器，将微积分引入学习动力学：grokking，其中已知一个复杂性序参量但尚未使其发挥作用。除了瞬态损失脉冲外，该估计器成为一个控制器，在Levin的描述长度对时间意义上加速grokking，在一个数据相关的奥卡姆边界内，其有限尺寸趋势 $f_c\sim\ln p/p$ 与优惠券收集器解释一致。消融实验表明，复杂性门在拯救失败种子方面与训练损失门匹配，但干预减少27%；在测试的信号中，只有映射复杂性标记了转变的完成；经认证的先验和逐参数 $\nabla K$ 归因都是可替代的（均匀先验传感器做出比特相同的门决策，随机支持在稀疏阈值以上匹配 $\nabla K$ 选择的）；直接场扰动显示对奥卡姆场的成核样响应（在探测幅度上未分辨出线性区域，因此这些测量不证明涨落-耗散替代），有限场响应朝转变方向增长多个数量级。这些测量解释了经验调优的阶梯：砰-砰脉冲，在产量上失速触发并释放，其迭代可能构建了它所利用的响应。该脉冲转移到稀疏奇偶校验和变压器；持续的权重空间损失失败。算法估计器的独特贡献是时机（何时触发和何时释放），而非归因。

英文摘要

Algorithmic Information Dynamics (AID) studies systems by perturbing them and measuring changes in algorithmic complexity, but its usual estimator, the Block Decomposition Method, is piecewise constant, restricting the calculus to finite differences. We use $K^{\mathrm{CDM}}_{\mathrm{s}F}$, a certified, differentiable estimator, to bring the calculus into learning dynamics: grokking, where a complexity order parameter is known but has not been made to act. As a transient loss kick, the estimator becomes a controller that accelerates grokking in Levin's description-length--versus-time sense, within a data-dependent Occam boundary whose finite-size trend, $f_c\sim\ln p/p$, is consistent with a coupon-collector interpretation. Ablations show that a complexity gate matches a train-loss gate in rescuing failing seeds with $27\%$ less intervention; among the tested signals, only map complexity marks the transition's completion; the certified prior and the per-parameter $\nabla K$ attribution are both fungible (a uniform-prior sensor makes bit-identical gate decisions, and random supports match $\nabla K$-selected ones above a sparsity threshold); and direct field perturbation shows a nucleation-like response to the Occam field (no linear regime is resolved over the probed amplitudes, so these measurements do not justify a fluctuation--dissipation surrogate), with a finite-field response growing by orders of magnitude toward the phase-transition. These measurements account for the empirically tuned staircase: bang--bang pulses, stall-fired and released on yield, whose iteration plausibly builds the response it exploits. The kick transfers to sparse parity and to a transformer; a sustained weight-space loss fails. The algorithmic estimator's distinct contribution is timing (when to fire and when to release), not attribution.

发表机构

• Oxford Immune Algorithmics（牛津免疫算法公司）
• Oxford University Innovation（牛津大学创新公司）
• London Institute for Healthcare Engineering（伦敦医疗工程研究所）

机构由 AI 辅助整理，请以论文原文为准。