p-adic 模型的连续优化

中文摘要 AI 辅助

我们提出了第一种针对具有 $p$-adic 参数的机器学习模型的原生连续梯度下降方法。现有的原生优化器是离散的，大多是组合搜索，因为 $p$-adic 数 $\mathbb{Q}_p$ 是完全不连通的，且标准损失函数在其最小值之外是平坦的。为了实现连续优化，我们建议通过其 Berkovich 仿射线来处理 $\mathbb{Q}_p$：这是 $\mathbb{Q}_p$ 的一个典范的、路径连通的扩张，它保持了 $\mathbb{Q}_p$ 的等距性，并唯一地扩展了其解析映射。这个包络是一个度量树，具有可解释的点和局部导数，我们证明这能够实现有效的优化器和反向传播。我们制定了梯度下降，并表明其近似能够高效地学习系数在 $\mathbb{Q}_p$ 中的线性模型，以执行模运算，这是一个无法用 $\mathbb{R}$ 中的线性模型表达的类似 XOR 的任务。我们还展示了动量（momentum）和 Adam 变体、线性回归以及二进制编码层次结构（Quillian 语义网络）上的分类，解决了 Martins (2025) 提出的开放问题。库位于此 https URL

英文摘要

We present the first method for native, continuous gradient descent for machine learning models with $p$-adic parameters. Existing native optimizers are discrete, mostly combinatorial searches, as the $p$-adic numbers $\mathbb{Q}_p$ are totally disconnected, with standard losses that are flat away from their minima. To enable continuous optimization, we propose working with $\mathbb{Q}_p$ via its Berkovich affine line: a canonical, path-connected expansion of $\mathbb{Q}_p$ that preserves its isometries and uniquely extends its analytic maps. This hull is a metric tree with interpretable points and local derivatives, which we show enables effective optimizers and backpropagation. We formulate gradient descent and show that its approximations efficiently learn linear models with coefficients in $\mathbb{Q}_p$ to do modular arithmetic, an XOR-like task not expressible by linear models in $\mathbb{R}$. We also demonstrate momentum and Adam variants, linear regression, and classification on binary-encoded hierarchies (Quillian semantic networks), addressing open problems posed by Martins (2025). Library at https://github.com/google-deepmind/padic-ml

发表机构

• Google DeepMind（谷歌DeepMind）

机构由 AI 辅助整理，请以论文原文为准。