Structured Chemistry Reasoning with Large Language Models
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Shanghai Jiao Tong University(上海交通大学);New York University(纽约大学);Allen Institute for Artificial Intelligence(艾伦人工智能研究所);University of California San-Diego(加利福尼亚大学圣迭戈分校);University of Washington(华盛顿大学)
Large Language Models (LLMs) excel in diverse areas, yet struggle with complex scientific reasoning, especially in the field of chemistry. Different from the simple chemistry tasks (e.g., molecule classification) addressed in previous studies, complex chemistry problems require not only vast knowledge and precise calculation, but also compositional reasoning about rich dynamic interactions of different concepts (e.g., temperature changes). Our study shows that even advanced LLMs, like GPT-4, can fail easily in different ways. Interestingly, the errors often stem not from a lack of domain knowledge within the LLMs, but rather from the absence of an effective reasoning structure that guides the LLMs to elicit the right knowledge, incorporate the knowledge in step-by-step reasoning, and iteratively refine results for further improved quality. On this basis, we introduce StructChem, a simple yet effective prompting strategy that offers the desired guidance and substantially boosts the LLMs' chemical reasoning capability. Testing across four chemistry areas -- quantum chemistry, mechanics, physical chemistry, and kinetics -- StructChem substantially enhances GPT-4's performance, with up to 30\% peak improvement. Our analysis also underscores the unique difficulties of precise grounded reasoning in science with LLMs, highlighting a need for more research in this area. Code is available at \url{https://github.com/ozyyshr/StructChem}.
G-SHARE: A Guideline-Based Structured Reasoning Framework for Human-Factor Event Diagnosis
Institute of Nuclear and New Energy Technology, Tsinghua University(清华大学核能与新能源技术研究院);National Key Laboratory of Human Factors Engineering(人因工程重点实验室);Fujian Fuqing Nuclear Power Co., Ltd.(福建福清核电有限公司)
ChemVTS-Bench: Evaluating Visual-Textual-Symbolic Reasoning of Multimodal Large Language Models in Chemistry
Renmin University of China(中国人民大学);Beijing University of Posts and Telecommunications(北京邮电大学);South China University of Technology(华南理工大学);Gaotu Techedu Inc(高图科技公司)
Reasoning Factual Knowledge in Structured Data with Large Language Models
Hong Kong Polytechnic University(香港理工大学);University Technology Sydney(悉尼科技大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Soochow University(苏州大学);The Education University of Hong Kong(香港教育大学)
Focused Chain-of-Thought: Efficient LLM Reasoning via Structured Input Information
German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);Technical University of Darmstadt(德累斯顿技术大学);University of Kassel(卡塞尔大学);TU Dortmund University(Dortmund技术大学);TU Center for Trustworthy Data Science and Security, University Alliance Ruhr(TU可信数据科学与安全中心,鲁尔大学联盟);Hessian Center for AI (Hessian.AI)(黑森人工智能中心(黑森.AI));Centre for Cognitive Science, Technical University of Darmstadt(认知科学中心,德累斯顿技术大学)
Cognitive Chain-of-Thought (CoCoT): Structured Multimodal Reasoning about Social Situations
Seoul National University(首尔国立大学);Human-Computer Interaction Institute, Carnegie Mellon University(人机交互研究所,卡内基梅隆大学);Language Technologies Institute, Carnegie Mellon University(语言技术研究所,卡内基梅隆大学)
Graph Reasoning Paradigm: Structured and Symbolic Reasoning with Topology-Aware Reinforcement Learning for Large Language Models
Harbin Institute of Technology(哈尔滨工业大学);State Key Laboratory of Cognitive Intelligence(认知智能国家重点实验室);Tianjin Normal University(天津师范大学);Pengcheng Laboratory(鹏城实验室)
SoT: Structured-of-Thought Prompting Guides Multilingual Reasoning in Large Language Models
Key Laboratory of Big Data & Artificial Intelligence in Transportation (Beijing Jiaotong University), Ministry of Education(大数据与人工智能交通 key laboratory(北京交通大学));School of Computer Science and Technology, Beijing Jiaotong University(计算机科学与技术学院(北京交通大学));University of Montreal(蒙特利尔大学)