arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.18045cs.CLcs.AI
收藏

PharmaGPT:面向生物制药与化学领域的专用大语言模型

PharmaGPT: Domain-Specific Large Language Models for Bio-Pharmaceutical and Chemistry

PatSnap Co., LTD.(慧算(PatSnap))
Linqing Chen, Weilei Wang, Zilong Bai, Peng Xu, Yan Fang, Jie Fang, Wentao Wu, Lizhi Zhou, Ruiji Zhang, Yubin Xia, Chaobo Xu, Ran Hu, Licong Xu, Qijun Cai, Haor… 展开作者

Linqing Chen, Weilei Wang, Zilong Bai, Peng Xu, Yan Fang, Jie Fang, Wentao Wu, Lizhi Zhou, Ruiji Zhang, Yubin Xia, Chaobo Xu, Ran Hu, Licong Xu, Qijun Cai, Haoran Hua, Jing Sun, Jin Liu, Tian Qiu, Haowen Liu, Meng Hu, Xiuwen Li, Fei Gao, Yufu Wang, Lin Tie, Chaochao Wang, Jianping Lu, Cheng Sun, Yixin Wang, Shengjie Yang, Yuancheng Li, Lu Jin, Lisha Zhang, Fu Bian, Zhongkai Ye, Lidong Pei, Changyang Tu

AI总结:

针对通用大语言模型难以满足生物制药与化学领域专业精度需求的问题,研究推出基于领域定制语料训练的130亿、700亿参数PharmaGPT模型,在NAPLEX等基准上超越参数量远大于它的通用模型,为该领域NLP应用提供新基准。

摘要:

大语言模型(LLMs)通过大幅降低对复杂特征工程的需求,革新了自然语言处理(NLP)领域。然而,LLMs在生物制药、化学等专业领域的应用仍有待充分探索。这些领域术语复杂、知识专业性强且对精度要求极高,通用大语言模型往往难以满足需求。本研究推出PharmaGPT,这是一套包含130亿参数和700亿参数版本的领域专用大语言模型,专门基于为生物制药与化学领域定制的综合语料库训练而成。评估结果显示,PharmaGPT在NAPLEX等领域专用基准测试上的表现优于现有通用模型,展现出在领域特定任务中的卓越能力。值得注意的是,取得这一性能的模型参数量仅为通用大模型的几分之一,有时甚至仅为十分之一。这一进展为生物制药与化学领域的大语言模型树立了新基准,填补了专用语言建模领域的现有空白,也为提升研发效率提供了可行方向,为这些领域更精准、高效的NLP应用铺平了道路。

英文摘要:

Large language models (LLMs) have revolutionized Natural Language Processing (NLP) by minimizing the need for complex feature engineering. However, the application of LLMs in specialized domains like biopharmaceuticals and chemistry remains largely unexplored. These fields are characterized by intricate terminologies, specialized knowledge, and a high demand for precision areas where general purpose LLMs often fall short. In this study, we introduce PharmaGPT, a suite of domain specilized LLMs with 13 billion and 70 billion parameters, specifically trained on a comprehensive corpus tailored to the Bio-Pharmaceutical and Chemical domains. Our evaluation shows that PharmaGPT surpasses existing general models on specific-domain benchmarks such as NAPLEX, demonstrating its exceptional capability in domain-specific tasks. Remarkably, this performance is achieved with a model that has only a fraction, sometimes just one-tenth-of the parameters of general-purpose large models. This advancement establishes a new benchmark for LLMs in the bio-pharmaceutical and chemical fields, addressing the existing gap in specialized language modeling. It also suggests a promising path for enhanced research and development, paving the way for more precise and effective NLP applications in these areas.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑