发表机构
Texas State University（德克萨斯州立大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出分块稳定自适应三次正则化优化器ARC-φ₁，将参数按张量分块求解，在大参数INR任务上收敛速度与性能显著优于Adam。
AI 中文摘要
三次正则化牛顿方法具有最优的O(ε^(-3/2))全局收敛速率和自动逃离鞍点的机制，但其子问题通常需通过完整的特征分解求解，限制了可行的模型规模。本文提出一种分块优化器，按张量划分参数，对每个块使用自适应三次常数M_b最小化独立的三次模型，并基于全损失的单调约束接受或拒绝每个块的步长。子求解器按块大小选择：小块使用由显式构造的块内海森矩阵得到的惰性精确三次步；任意大的张量使用通过Lanczos过程构建的无矩阵Chebyshev约束Krylov子空间。当梯度驱动的偏移主导负曲率时，三次偏移约束所需的多项式阶数，使偏移算子在应用任何多项式前保持半正定性，并在子问题不精确求解的情况下保留O(ε^(-3/2))的收敛速率。我们证明了这些结论，且分块方案具有每个块单调下降的保证。实验覆盖FINER隐式神经表示（INRs，约19.9万个参数）和9140万参数的ViSIR INR，其中分块三次步在每个块上保持三次模型意义下的精确性，包括8850万参数的解码器张量（占模型的97%）。在FINER上完全收敛时，ARC-φ₁优化器达到133.5 dB的峰值信噪比（PSNR），而调优后的Adam在相同的扩展预算下稳定在78.2 dB；在Adam达到峰值所需的约70分钟内，ARC-φ₁达到95.6 dB。配套报告分离出导致Adam表现的损失景观特征。
英文摘要
Cubic regularized Newton methods have the optimal $\mathcal{O}(ε^{-3/2})$ global rate, but a dense subproblem solve limits the feasible block size. Scalable Cubic Newton variants replace the true block curvature with a diagonal, low-rank, Kronecker-factored, or sketched surrogate and, most often, give up the exact cubic step. We introduce a blockwise optimizer that minimizes an independent cubic model per parameter tensor over the true block Hessian, under a per-block adaptive cubic constant and a monotone guard on the full loss. Arbitrarily large tensors are handled matrix-free in a Lanczos-built Krylov subspace, where we prove that the step minimizes the cubic model. The theory also supplies the $\mathcal{O}(ε^{-3/2})$ iteration complexity bound, a second-order guarantee, and monotone per-block descent. Four variants of this outer scheme are evaluated against the original adaptive regularization with cubics (ARC) optimizer, some other recent cubic Newton variants, Adam, SOAP, and L-BFGS. On a 91.4M-parameter implicit neural representation (INR), the variants introduced in this work are the only evaluated here cubic Newton methods whose steps stay exact on every block. Run to full convergence on FINER 2D image fitting, one of the ARC variants introduced here, ARC-$φ_1$, reaches 133.5 dB peak signal-to-noise ratio, while tuned Adam plateaus at 78.2 dB after about 70 minutes. In that time ARC-$φ_1$ reaches 95.6 dB.
