arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.14818cs.CLcs.DL
收藏

开发 ChemDFM 作为化学领域的大型语言基础模型

Developing ChemDFM as a large language foundation model for chemistry

X-LANCE Lab, Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence, SJTU AI Institute Shanghai Jiao Tong University, Shanghai, China(X-LANCE实验室,计算机科学与工程系,人工智能关键实验室,上海交通大学人工智能研究院,上海,中国);Suzhou Laboratory, Suzhou, China(苏州实验室,苏州,中国);Shanghai Key Laboratory for Molecular Engineering of Chiral Drugs School of Chemistry and Chemical Engineering Shanghai Jiao Tong University, Shanghai, China(上海分子工程学药物重点实验室,化学与化工学院,上海交通大学,上海,中国);AI Speech Co, .Ltd., Suzhou, China(AI Speech公司,苏州,中国)

Zihan Zhao, Da Ma, Lu Chen, Liangtai Sun, Zihao Li, Yi Xia, Bo Chen, Hongshen Xu, Zichen Zhu, Su Zhu, Shuai Fan, Guodong Shen, Kai Yu, Xin Chen

AI总结:

本文提出 ChemDFM,一个基于 340 亿化学语料训练、270 万指令微调的化学大语言模型,在自由对话中展现化学推理能力,并在多数化学任务上超越 GPT-4 及主流开源模型。

摘要:

人工智能(AI)在化学研究中发挥着日益重要的作用。然而,当前化学领域使用的大多数模型都是专用模型,需要针对特定任务进行训练和调优。一种更通用、更高效的解决方案是构建一个能够处理多种任务并在广泛的化学领域支持自由形式对话的 AI 模型。在其最高形态下,这种通用型 AI 化学家可被称为化学通用智能。大型语言模型(LLMs)最近在自然语言处理的通用领域取得了巨大成功,展现出新兴的任务泛化能力和自由形式对话能力。然而,在训练通用领域 LLMs 时,化学领域知识在很大程度上是缺失的。这种知识的缺乏极大地阻碍了通用型 LLMs 在化学领域的表现。为此,我们开发了 ChemDFM,这是一个开创性的化学 LLM,基于来自化学文献和教科书的 340 亿 token 进行训练,并使用 270 万条指令进行微调。因此,它能够在自由形式对话中理解和推理化学知识。定量评估表明,ChemDFM 显著超越了大多数具有代表性的开源 LLMs。尽管模型规模存在巨大差异,它在大部分化学任务上的表现仍优于 GPT-4。我们已在 Huggingface(https://huggingface.co/OpenDFM/ChemDFM-v1.0-13B)上开源了 ChemDFM 的推理代码、评估数据集和模型权重。

英文摘要:

Artificial intelligence (AI) has played an increasingly important role in chemical research. However, most models currently used in chemistry are specialist models that require training and tuning for specific tasks. A more generic and efficient solution would be an AI model that could address many tasks and support free-form dialogue in the broad field of chemistry. In its utmost form, such a generalist AI chemist could be referred to as Chemical General Intelligence. Large language models (LLMs) have recently logged tremendous success in the general domain of natural language processing, showing emerging task generalization and free-form dialogue capabilities. However, domain knowledge of chemistry is largely missing when training general-domain LLMs. The lack of such knowledge greatly hinders the performance of generalist LLMs in the field of chemistry. To this end, we develop ChemDFM, a pioneering LLM for chemistry trained on 34B tokens from chemical literature and textbooks, and fine-tuned using 2.7M instructions. As a result, it can understand and reason with chemical knowledge in free-form dialogue. Quantitative evaluations show that ChemDFM significantly surpasses most representative open-source LLMs. It outperforms GPT-4 on a great portion of chemical tasks, despite the substantial size difference. We have open-sourced the inference codes, evaluation datasets, and model weights of ChemDFM on Huggingface (https://huggingface.co/OpenDFM/ChemDFM-v1.0-13B).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑