arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.09768cs.LGcs.AIphysics.comp-ph
收藏

缩放定律与对称性:来自神经力场的证据

Scaling Laws and Symmetry, Evidence from Neural Force Fields

Mila - Quebec AI Institute(魁北克AI研究所);School of Computer Science, McGill University(麦吉尔大学计算机科学学院)

Khang Ngo, Siamak Ravanbakhsh

AI总结:

本文通过神经力场实证研究学习原子间势的几何任务,发现等变架构利用对称性在数据、参数和计算上比非等变模型具有更好的幂律缩放行为,且高阶表示能进一步提升缩放指数,表明对称性等基本归纳偏置不应留给模型自行发现。

摘要:

我们在学习原子间势的几何任务中进行了实证研究,表明等变性在更大规模下更为重要;我们展示了关于数据、参数和计算的清晰幂律缩放行为,且具有“依赖架构的指数”。特别地,我们观察到利用任务对称性的等变架构比非等变模型缩放得更好。此外,在等变架构中,高阶表示带来更好的缩放指数。我们的分析还表明,对于计算最优训练,无论架构如何,数据和模型规模应同步缩放。总体而言,这些结果表明,与普遍看法相反,我们不应将发现对称性等基本归纳偏置留给模型,特别是在扩大规模时,因为它们会改变任务及其缩放定律的固有难度。

英文摘要:

We present an empirical study in the geometric task of learning interatomic potentials, which shows equivariance matters even more at larger scales; we show a clear power-law scaling behaviour with respect to data, parameters and compute with ``architecture-dependent exponents''. In particular, we observe that equivariant architectures, which leverage task symmetry, scale better than non-equivariant models. Moreover, among equivariant architectures, higher-order representations translate to better scaling exponents. Our analysis also suggests that for compute-optimal training, the data and model sizes should scale in tandem regardless of the architecture. At a high level, these results suggest that, contrary to common belief, we should not leave it to the model to discover fundamental inductive biases such as symmetry, especially as we scale, because they change the inherent difficulty of the task and its scaling laws.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑