Chemistry plays a crucial role in many domains, such as drug discovery and material science. While large language models (LLMs) such as GPT-4 exhibit remarkable capabilities on natural language processing tasks, existing research indicates that their performance on chemistry tasks is discouragingly low. In this paper, however, we demonstrate that our developed LLMs can achieve very strong results on a comprehensive set of chemistry tasks, outperforming the most advanced GPT-4 and Claude 3 Opus by a substantial margin. To accomplish this, we propose SMolInstruct, a large-scale, comprehensive, and high-quality dataset for instruction tuning. It contains 14 selected chemistry tasks and over three million samples, laying a solid foundation for training and evaluating LLMs for chemistry. Using SMolInstruct, we fine-tune a set of open-source LLMs, among which, we find that Mistral serves as the best base model for chemistry tasks. Our analysis further demonstrates the critical role of the proposed dataset in driving the performance improvements.
MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypotheses
Nanyang Technological University(南洋理工大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);University of Science and Technology of China(中国科学技术大学);Wuhan University(武汉大学);University of New South Wales(新南威尔士大学);GreenDynamics;Singapore University of Technology and Design(新加坡科技设计大学)
ChemVTS-Bench: Evaluating Visual-Textual-Symbolic Reasoning of Multimodal Large Language Models in Chemistry
Renmin University of China(中国人民大学);Beijing University of Posts and Telecommunications(北京邮电大学);South China University of Technology(华南理工大学);Gaotu Techedu Inc(高图科技公司)
Advancing the Scientific Method with Large Language Models: From Hypothesis to Discovery
Tufts University(塔夫茨大学);KAUST(阿卜杜拉国王科技大学);Network Rail(英国铁路网公司);Zuse Institute Berlin(柏林祖泽研究所);NVIDIA Research(英伟达研究院);King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);Pasteur Labs(巴斯德实验室);Harvard University(哈佛大学);University of Southampton(南安普顿大学);Stanford University(斯坦福大学);University of Chicago(芝加哥大学);Santa Fe Institute(圣塔菲研究所);The University of Edinburgh(爱丁堡大学);Jožef Stefan Institute(约热夫·斯特凡研究所);Karolinska Institutet(卡罗林斯卡学院);The Alan Turing Institute(艾伦·图灵研究所);Oxford Immune Algorithmics(牛津免疫算法公司);Francis Crick Institute(弗朗西斯·克里克研究所)
SciKnowEval: Evaluating Multi-level Scientific Knowledge of Large Language Models
College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University(浙江大学Hangzhou全球科技创新中心);ZJU-UIUC Institute, Zhejiang University(浙江大学UIUC研究院)
RefTool: Reference-Guided Tool Creation for Knowledge-Intensive Reasoning
Wangxuan Institute of Computer Technology, Peking University(北京大学王萱计算机技术研究所);University of Chicago(芝加哥大学);University of California, Los Angeles(加州大学洛杉矶分校)
FGBench: A Dataset and Benchmark for Molecular Property Reasoning at Functional Group-Level in Large Language Models
Department of Chemical and Biomolecular Engineering, University of Illinois Urbana-Champaign(化学与生物分子工程系,伊利诺伊大学厄巴纳-香槟分校);Department of Computer Science, University of Illinois Urbana-Champaign(计算机科学系,伊利诺伊大学厄巴纳-香槟分校)
Beyond Chemical QA: Evaluating LLM's Chemical Reasoning with Modular Chemical Operations
Pengcheng Laboratory(鹏城实验室);International Digital Economy Academy(国际数字经济学院);School of Electronic and Computer Engineering, Peking University(北京大学电子与计算机工程学院);School of AI for Science, Peking University(北京大学科学人工智能学院);Yale University(耶鲁大学)