arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2311.09656cs.CLcs.AI
收藏

大型语言模型的结构化化学推理

Structured Chemistry Reasoning with Large Language Models

University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Shanghai Jiao Tong University(上海交通大学);New York University(纽约大学);Allen Institute for Artificial Intelligence(艾伦人工智能研究所);University of California San-Diego(加利福尼亚大学圣迭戈分校);University of Washington(华盛顿大学)

Siru Ouyang, Zhuosheng Zhang, Bing Yan, Xuan Liu, Yejin Choi, Jiawei Han, Lianhui Qin

AI总结:

针对LLMs在复杂化学推理中的不足,提出StructChem提示策略,通过结构化引导提升GPT-4在四个化学领域的性能,峰值提升达30%。

摘要:

大型语言模型(LLMs)在多个领域表现出色,但在复杂的科学推理方面,尤其是化学领域,仍面临挑战。与以往研究中处理的简单化学任务(如分子分类)不同,复杂化学问题不仅需要广泛的知识和精确的计算,还需要对不同概念(如温度变化)之间丰富的动态交互进行组合推理。我们的研究表明,即使是先进的LLMs,如GPT-4,也可能以不同的方式轻易失败。有趣的是,错误往往并非源于LLMs缺乏领域知识,而是缺乏有效的推理结构来引导LLMs引出正确的知识,将知识融入逐步推理中,并迭代优化结果以进一步提高质量。基于此,我们引入了StructChem,一种简单而有效的提示策略,提供所需的指导,并显著提升LLMs的化学推理能力。在四个化学领域——量子化学、力学、物理化学和动力学——的测试中,StructChem显著提升了GPT-4的性能,峰值提升高达30%。我们的分析还强调了LLMs在科学中进行精确基础推理的独特困难,凸显了在该领域进行更多研究的必要性。代码可在\url{https://github.com/ozyyshr/StructChem}获取。

英文摘要:

Large Language Models (LLMs) excel in diverse areas, yet struggle with complex scientific reasoning, especially in the field of chemistry. Different from the simple chemistry tasks (e.g., molecule classification) addressed in previous studies, complex chemistry problems require not only vast knowledge and precise calculation, but also compositional reasoning about rich dynamic interactions of different concepts (e.g., temperature changes). Our study shows that even advanced LLMs, like GPT-4, can fail easily in different ways. Interestingly, the errors often stem not from a lack of domain knowledge within the LLMs, but rather from the absence of an effective reasoning structure that guides the LLMs to elicit the right knowledge, incorporate the knowledge in step-by-step reasoning, and iteratively refine results for further improved quality. On this basis, we introduce StructChem, a simple yet effective prompting strategy that offers the desired guidance and substantially boosts the LLMs' chemical reasoning capability. Testing across four chemistry areas -- quantum chemistry, mechanics, physical chemistry, and kinetics -- StructChem substantially enhances GPT-4's performance, with up to 30\% peak improvement. Our analysis also underscores the unique difficulties of precise grounded reasoning in science with LLMs, highlighting a need for more research in this area. Code is available at \url{https://github.com/ozyyshr/StructChem}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑