arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.02259cs.LGcond-mat.mtrl-sciphysics.chem-phq-bio.BM
收藏

Transformers在无图先验的情况下发现分子结构

Transformers Discover Molecular Structure Without Graph Priors

UC Berkeley(伯克利大学);LBNL(劳伦斯伯克利国家实验室)

Tobias Kreiman, Yutong Bai, Fadi Atieh, Elizabeth Weaver, Eric Qu, Aditi S. Krishnapriyan

AI总结:

本研究验证了无图先验、直接基于笛卡尔坐标训练的纯Transformer可实现与最先进等变GNN相当的分子能量和力预测精度,其能自适应学习物理规律且符合缩放定律,挑战了图归纳偏置的必要性。

摘要:

图神经网络(GNNs)是分子机器学习领域的主流架构,尤其适用于分子性质预测和机器学习原子间势(MLIPs)任务。GNNs在通常由固定半径截断或k近邻方案生成的预定义图上执行消息传递。尽管这种设计契合许多分子任务中存在的局部性特征,但硬编码的图会因感受野固定而限制表达能力,且稀疏图操作会拖慢推理速度。本研究中,我们探究直接在笛卡尔坐标上训练的纯的、未修改的Transformers——无需预定义图或物理先验——能否近似分子能量和力。作为分析的起点,我们展示了在匹配的训练计算预算下,如何在OMol25数据集上训练出与最先进等变GNN相比具备竞争力的能量和力平均绝对误差的Transformer。我们发现Transformer能学习到物理上一致的模式——例如注意力权重随原子间距离成反比衰减——并且由于没有硬编码偏差,它能在不同分子环境中灵活调整这些模式。标准Transformer的使用还解锁了训练资源扩展带来的可预测性能提升,这与其他领域观察到的经验缩放定律一致。我们的结果表明,GNNs的许多优良特性可以在Transformers中自适应涌现,这对硬编码图归纳偏置的必要性提出了挑战,并为分子建模指明了标准化、可扩展的架构方向。

英文摘要:

Computational simulations play a central role in scientific discovery, and machine learning (ML) has emerged as a promising alternative to traditional physics-based modeling. However, scientific modeling requires physically meaningful predictions, raising a fundamental question for data-driven methods: to what extent can physical inductive biases-that is, prior assumptions about the structure of the physical world-emerge by learning from data alone? In this work, we study atomistic modeling, a representative field in the computational sciences where ML architectures have historically embedded strong physical inductive biases-such as geometric locality and graph structure. We develop a systematic understanding of how physical patterns can alternatively be discovered directly from data by training a model without domain-specific priors, including any manually defined atomistic pairwise interactions. We find that the model autonomously recovers key physical structure, such as learned interatomic interaction strengths that mirror classical electrostatics and interaction cutoffs consistent with traditional physical models. We further demonstrate predictable neural scaling law behavior with increased data and compute, and find accuracy on certain metrics competitive with physics-informed architectures. Our results clarify the boundary between engineered inductive biases and learnable physical structure, suggesting that general-purpose architectures can serve as principled baselines for scientific modeling by learning fundamental physical structure directly from data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑