Reasoning about real-life events is a unifying challenge in AI and NLP that has profound utility in a variety of domains, while fallacy in high-stake applications could be catastrophic. Able to work with diverse text in these domains, large language models (LLMs) have proven capable of answering questions and solving problems. However, I show that end-to-end LLMs still systematically fail to reason about complex events, and they lack interpretability due to their black-box nature. To address these issues, I propose three general approaches to use LLMs in conjunction with a structured representation of events. The first is a language-based representation involving relations of sub-events that can be learned by LLMs via fine-tuning. The second is a semi-symbolic representation involving states of entities that can be predicted and leveraged by LLMs via few-shot prompting. The third is a fully symbolic representation that can be predicted by LLMs trained with structured data and be executed by symbolic solvers. On a suite of event reasoning tasks spanning common-sense inference and planning, I show that each approach greatly outperforms end-to-end LLMs with more interpretability. These results suggest manners of synergy between LLMs and structured representations for event reasoning and beyond.
Graph Reasoning Paradigm: Structured and Symbolic Reasoning with Topology-Aware Reinforcement Learning for Large Language Models
Harbin Institute of Technology(哈尔滨工业大学);State Key Laboratory of Cognitive Intelligence(认知智能国家重点实验室);Tianjin Normal University(天津师范大学);Pengcheng Laboratory(鹏城实验室)
SoT: Structured-of-Thought Prompting Guides Multilingual Reasoning in Large Language Models
Key Laboratory of Big Data & Artificial Intelligence in Transportation (Beijing Jiaotong University), Ministry of Education(大数据与人工智能交通 key laboratory(北京交通大学));School of Computer Science and Technology, Beijing Jiaotong University(计算机科学与技术学院(北京交通大学));University of Montreal(蒙特利尔大学)
Pandora: Leveraging Code-driven Knowledge Transfer for Unified Structured Knowledge Reasoning
Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education(新一代人工智能技术及交叉应用重点实验室(东南大学));China Mobile Research(中国移动研究院);Monash University(莫纳什大学)
JPO: Juris Policy Optimization for Structured Legal Reasoning in Criminal Judgment Prediction
Tencent(腾讯);Peking University(北京大学);Tsinghua University(清华大学);City University of Hong Kong(香港城市大学);University of California(加利福尼亚大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Zhejiang University(浙江大学);University of Hong Kong(香港大学)
K-DeCore: Facilitating Knowledge Transfer in Continual Structured Knowledge Reasoning via Knowledge Decoupling
School of Computer Science and Engineering, Southeast University, China(东南大学计算机科学与工程学院);Key Laboratory of New Generation Artificial Intelligence Technology and its Interdisciplinary Applications (Southeast University), Ministry of Education, China(新一代人工智能技术及 interdisciplinary 应用关键实验室(东南大学));China Mobile Research Institute(中国移动研究院);Department of Data Science & AI, Monash University, Australia(莫纳什大学数据科学与人工智能系)
STRICTA: Structured Reasoning in Critical Text Assessment for Peer Review and Beyond
UKP Lab, ATHENE National Research Center for Applied Cybersecurity, Technical University of Darmstadt(UKP实验室,ATHENE国家应用网络安全研究中心,德累斯顿技术大学);AIML Lab, Hessian Center for AI, Technical University of Darmstadt(AIML实验室,黑森人工智能中心,德累斯顿技术大学);Synthetic RNA Biology, Technical University of Darmstadt(合成RNA生物学,德累斯顿技术大学)
G-SHARE: A Guideline-Based Structured Reasoning Framework for Human-Factor Event Diagnosis
Institute of Nuclear and New Energy Technology, Tsinghua University(清华大学核能与新能源技术研究院);National Key Laboratory of Human Factors Engineering(人因工程重点实验室);Fujian Fuqing Nuclear Power Co., Ltd.(福建福清核电有限公司)
Cognitive Chain-of-Thought (CoCoT): Structured Multimodal Reasoning about Social Situations
Seoul National University(首尔国立大学);Human-Computer Interaction Institute, Carnegie Mellon University(人机交互研究所,卡内基梅隆大学);Language Technologies Institute, Carnegie Mellon University(语言技术研究所,卡内基梅隆大学)
Reasoning Factual Knowledge in Structured Data with Large Language Models
Hong Kong Polytechnic University(香港理工大学);University Technology Sydney(悉尼科技大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Soochow University(苏州大学);The Education University of Hong Kong(香港教育大学)
Focused Chain-of-Thought: Efficient LLM Reasoning via Structured Input Information
German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);Technical University of Darmstadt(德累斯顿技术大学);University of Kassel(卡塞尔大学);TU Dortmund University(Dortmund技术大学);TU Center for Trustworthy Data Science and Security, University Alliance Ruhr(TU可信数据科学与安全中心,鲁尔大学联盟);Hessian Center for AI (Hessian.AI)(黑森人工智能中心(黑森.AI));Centre for Cognitive Science, Technical University of Darmstadt(认知科学中心,德累斯顿技术大学)
Matrix as Plan: Structured Logical Reasoning with Feedback-Driven Replanning
Faculty of Arts and Sciences(艺术与科学学院);Beijing Normal University(北京师范大学);Institute of Artificial Intelligence and Future Networks(人工智能与未来网络研究所);Engineering Research Center of Cloud-Edge Intelligent Collaboration on Big Data, Ministry of Education(教育部云-边智能协同大数据工程研究中心)
Logical Structure as Knowledge: Enhancing LLM Reasoning via Structured Logical Knowledge Density Estimation
Huzhou University(湖州大学);Zhejiang Key Laboratory of Intelligent Education Technology and Application(浙江智能教育技术与应用重点实验室);Banbu AI Foundation(Banbu AI基金会);Baichuan Inc(白川公司);University of Edinburgh(爱丁堡大学);Zhejiang University(浙江大学)
BeamPERL: Parameter-Efficient RL with Verifiable Rewards Specializes Compact LLMs for Structured Beam Mechanics Reasoning
Department of Mechanical Engineering(机械工程系);Massachusetts Institute of Technology(麻省理工学院);Department of Civil and Environmental Engineering(土木与环境工程系);Schwarzman College of Computing(施瓦茨曼计算学院)
Structured, flexible, and robust: benchmarking and improving large language models towards more human-like behavior in out-of-distribution reasoning tasks