GPT-MolBERTa：用于分子性质预测的GPT分子特征语言模型
GPT-MolBERTa: GPT Molecular Features Language Model for molecular property prediction
AI总结：
针对SMILES分子表示的局限性，研究提出基于分子文本描述的自监督大语言模型GPT-MolBERTa，用ChatGPT生成的32万分子文本训练，微调结合BERT与RoBERTa，在分子性质基准上表现优异且具备可解释性。
摘要：
随着Transformer架构的出现及其对文本数据的强大理解能力，基于文本描述预测分子性质开辟了新的方向。SMILES是最常见的表示形式，但它缺乏鲁棒性、丰富信息和规范性，限制了其成为可泛化表示的有效性。本文提出GPT-MolBERTa，这是一种自监督大语言模型（LLM），利用分子的详细文本描述预测其性质。我们使用ChatGPT收集了326000个分子的文本描述，用于训练LLM学习分子表示。为预测下游任务的性质，在微调阶段同时使用了BERT和RoBERTa模型。实验表明，GPT-MolBERTa在多种分子性质基准上表现良好，在回归任务中接近当前最优性能。此外，对注意力机制的进一步分析显示，GPT-MolBERTa能够从输入文本数据中提取重要信息，体现了模型的可解释性。
英文摘要：
With the emergence of Transformer architectures and their powerful understanding of textual data, a new horizon has opened up to predict the molecular properties based on text description. While SMILES are the most common form of representation, they are lacking robustness, rich information and canonicity, which limit their effectiveness in becoming generalizable representations. Here, we present GPT-MolBERTa, a self-supervised large language model (LLM) which uses detailed textual descriptions of molecules to predict their properties. A text based description of 326000 molecules were collected using ChatGPT and used to train LLM to learn the representation of molecules. To predict the properties for the downstream tasks, both BERT and RoBERTa models were used in the finetuning stage. Experiments show that GPT-MolBERTa performs well on various molecule property benchmarks, and approaching state of the art performance in regression tasks. Additionally, further analysis of the attention mechanisms show that GPT-MolBERTa is able to pick up important information from the input textual data, displaying the interpretability of the model.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
