arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.04379cs.LGcond-mat.mtrl-sci
收藏

微调语言模型以文本形式生成稳定无机材料

Fine-Tuned Language Models Generate Stable Inorganic Materials as Text

NYU(纽约大学);Meta FAIR

Nate Gruver, Anuroop Sriram, Andrea Madotto, Andrew Gordon Wilson, C. Lawrence Zitnick, Zachary Ulissi

AI总结:

本文提出在文本编码原子数据上微调LLM以生成稳定无机材料,LLaMA-2 70B的亚稳态生成率达49%,约为CDVAE两倍,并支持无条件生成、结构补全和文本条件生成。

摘要:

我们提出微调大型语言模型以生成稳定材料。尽管看似非传统,但在文本编码的原子数据上微调大型语言模型不仅实现简单,而且可靠:约90%采样得到的结构都遵守关于原子位置和电荷的物理约束。利用学习得到的ML势函数以及金标准DFT计算分别进行的能垒以上能量计算,我们表明最强模型(微调后的LLaMA-2 70B)生成被预测为亚稳态材料的比率约为竞争扩散模型CDVAE的两倍(49%对28%)。由于文本提示固有的灵活性,我们的模型可同时用于稳定材料的无条件生成、部分结构的补全以及文本条件生成。最后,我们表明语言模型捕捉晶体结构关键对称性的能力随模型规模提升而增强,说明预训练LLM的偏置出奇地适合原子数据。

英文摘要:

We propose fine-tuning large language models for generation of stable materials. While unorthodox, fine-tuning large language models on text-encoded atomistic data is simple to implement yet reliable, with around 90% of sampled structures obeying physical constraints on atom positions and charges. Using energy above hull calculations from both learned ML potentials and gold-standard DFT calculations, we show that our strongest model (fine-tuned LLaMA-2 70B) can generate materials predicted to be metastable at about twice the rate (49% vs 28%) of CDVAE, a competing diffusion model. Because of text prompting's inherent flexibility, our models can simultaneously be used for unconditional generation of stable material, infilling of partial structures and text-conditional generation. Finally, we show that language models' ability to capture key symmetries of crystal structures improves with model scale, suggesting that the biases of pretrained LLMs are surprisingly well-suited for atomistic data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑