arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.19783cond-mat.mtrl-scics.LG
收藏

AlloyBERT:基于大语言模型的合金性能预测

AlloyBERT: Alloy Property Prediction with Large Language Models

Carnegie Mellon University(卡内基梅隆大学)

Akshat Chaudhari, Chakradhar Guntuboina, Hongshuo Huang, Amir Barati Farimani

AI总结:

本文提出基于RoBERTa编码器的AlloyBERT模型,通过文本输入预测合金弹性模量和屈服强度,在MPEA和RAYS数据集上均优于浅层模型,为不依赖复杂底层计算的合金性能文本预测建立了基础框架。

摘要:

针对特定需求研发新型合金给该领域的研究人员带来了重大挑战,这凸显了基于合金化学成分和加工参数开发关键物理性能预测技术的重要性。本研究提出了AlloyBERT,一种基于Transformer编码器的模型,旨在利用文本输入预测合金的弹性模量和屈服强度等性能。AlloyBERT以预训练的RoBERTa编码器模型为基础,采用自注意力机制建立词与词之间的有意义关系,使其能够解读人类可读的输入并预测目标合金性能。通过将在本文本数据上训练的tokenizer与针对该特定任务进行预训练和微调的RoBERTa编码器相结合,我们在多主元合金(MPEA)数据集上实现了0.00015的均方误差(MSE),在难熔合金屈服强度(RAYS)数据集上实现了0.00611的均方误差。这一结果优于浅层模型的表现,浅层模型在MPEA和RAYS数据集上的最佳MSE分别为0.00025和0.0076。我们的结果凸显了语言模型在材料科学中的潜力,并为基于文本的合金性能预测建立了一个基础框架,该框架不依赖于复杂的底层表示、计算或模拟。

英文摘要:

The pursuit of novel alloys tailored to specific requirements poses significant challenges for researchers in the field. This underscores the importance of developing predictive techniques for essential physical properties of alloys based on their chemical composition and processing parameters. This study introduces AlloyBERT, a transformer encoder-based model designed to predict properties such as elastic modulus and yield strength of alloys using textual inputs. Leveraging the pre-trained RoBERTa encoder model as its foundation, AlloyBERT employs self-attention mechanisms to establish meaningful relationships between words, enabling it to interpret human-readable input and predict target alloy properties. By combining a tokenizer trained on our textual data and a RoBERTa encoder pre-trained and fine-tuned for this specific task, we achieved a mean squared error (MSE) of 0.00015 on the Multi Principal Elemental Alloys (MPEA) data set and 0.00611 on the Refractory Alloy Yield Strength (RAYS) dataset. This surpasses the performance of shallow models, which achieved a best-case MSE of 0.00025 and 0.0076 on the MPEA and RAYS datasets respectively. Our results highlight the potential of language models in material science and establish a foundational framework for text-based prediction of alloy properties that does not rely on complex underlying representations, calculations, or simulations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑