arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2309.00563cs.CEphysics.chem-ph
收藏

CatBERTa 催化剂性质预测:通过大型语言模型揭示特征探索策略

Catalyst Property Prediction with CatBERTa: Unveiling Feature Exploration Strategies through Large Language Models

Janghoon Ock, Chakradhar Guntuboina, Amir Barati Farimani

AI总结:

本研究提出 CatBERTa,一种基于大型语言模型的文本输入 Transformer,用于预测催化剂吸附能,无需图表示,实现与 GNN 相当的精度并显著降低系统误差。

摘要:

高效的催化剂筛选需要能够预测吸附能的模型,吸附能是反应活性的关键性质。然而,现有方法,尤其是图神经网络(GNNs),需要精确的原子坐标来构建图表示,同时整合可观测属性仍然具有挑战性。本研究引入了 CatBERTa,一种使用文本输入的能源预测 Transformer 模型。CatBERTa 基于预训练的 Transformer 编码器构建,处理人类可解释的文本,并融入目标特征。注意力分数分析揭示了 CatBERTa 对与吸附物、体相组成及其相互作用原子相关的标记的关注。此外,相互作用原子作为吸附构型的有效描述符出现,而键长和这些原子的原子性质等因素对预测的贡献有限。通过从初始结构的文本表示预测吸附能,CatBERTa 实现了 0.75 eV 的平均绝对误差(MAE),与普通图神经网络(GNNs)相当。此外,CatBERTa 预测能量的相减有效地抵消了其系统误差,对于化学相似系统最多可减少 19.3%,超过了 GNNs 中观察到的误差减少。这一结果突显了其提高能量差预测准确性的潜力。这项研究为基于文本的催化剂性质预测建立了基础框架,无需依赖图表示,同时揭示了复杂的特征-性质关系。

英文摘要:

Efficient catalyst screening necessitates predictive models for adsorption energy, a key property of reactivity. However, prevailing methods, notably graph neural networks (GNNs), demand precise atomic coordinates for constructing graph representations, while integrating observable attributes remains challenging. This research introduces CatBERTa, an energy prediction Transformer model using textual inputs. Built on a pretrained Transformer encoder, CatBERTa processes human-interpretable text, incorporating target features. Attention score analysis reveals CatBERTa's focus on tokens related to adsorbates, bulk composition, and their interacting atoms. Moreover, interacting atoms emerge as effective descriptors for adsorption configurations, while factors such as bond length and atomic properties of these atoms offer limited predictive contributions. By predicting adsorption energy from the textual representation of initial structures, CatBERTa achieves a mean absolute error (MAE) of 0.75 eV-comparable to vanilla Graph Neural Networks (GNNs). Furthermore, the subtraction of the CatBERTa-predicted energies effectively cancels out their systematic errors by as much as 19.3% for chemically similar systems, surpassing the error reduction observed in GNNs. This outcome highlights its potential to enhance the accuracy of energy difference predictions. This research establishes a fundamental framework for text-based catalyst property prediction, without relying on graph representations, while also unveiling intricate feature-property relationships.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑