arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.11686cs.LGcond-mat.mtrl-sciphysics.comp-ph
收藏

Crystalformer:用于周期结构编码的无限连接注意力

Crystalformer: Infinitely Connected Attention for Periodic Structure Encoding

OMRON SINIC X Corp.(欧姆龙SINIC X公司);Toyota Motor Corp.(丰田汽车公司);Tohoku University(东北大学);Randeft Inc.(兰代夫公司);Osaka University(大阪大学)

Tatsunori Taniai, Ryo Igarashi, Yuta Suzuki, Naoya Chiba, Kotaro Saito, Yoshitaka Ushiku, Kanta Ono

AI总结:

针对晶体结构周期性的特点,提出无限连接注意力机制,构建轻量级Transformer编码器Crystalformer,以29.4%的参数在多个性质回归任务上超越现有最优方法。

摘要:

根据晶体结构预测材料的物理性质是材料科学中的一个基本问题。在分子性质预测等周边领域,全连接注意力网络已被证明是成功的。然而,与这些有限的原子排列不同,晶体结构是无限重复的、周期性的原子排列,其全连接注意力导致无限连接的注意力。在这项工作中,我们表明这种无限连接的注意力可以导致一种计算上易于处理的公式,可解释为神经势求和,在深度学习的特征空间中进行无限的原子间势求和。然后,我们提出了一种简单而有效的基于Transformer的晶体结构编码器架构,称为Crystalformer。与现有的基于Transformer的模型相比,所提出的模型仅需要29.4%的参数数量,且对原始Transformer架构的修改最小。尽管架构简单,所提出的方法在Materials Project和JARVIS-DFT数据集上的各种性质回归任务中优于最先进的方法。

英文摘要:

Predicting physical properties of materials from their crystal structures is a fundamental problem in materials science. In peripheral areas such as the prediction of molecular properties, fully connected attention networks have been shown to be successful. However, unlike these finite atom arrangements, crystal structures are infinitely repeating, periodic arrangements of atoms, whose fully connected attention results in infinitely connected attention. In this work, we show that this infinitely connected attention can lead to a computationally tractable formulation, interpreted as neural potential summation, that performs infinite interatomic potential summations in a deeply learned feature space. We then propose a simple yet effective Transformer-based encoder architecture for crystal structures called Crystalformer. Compared to an existing Transformer-based model, the proposed model requires only 29.4% of the number of parameters, with minimal modifications to the original Transformer architecture. Despite the architectural simplicity, the proposed method outperforms state-of-the-art methods for various property regression tasks on the Materials Project and JARVIS-DFT datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑