arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.16098cs.CL
收藏

基于大型语言模型的结构化事件推理

Structured Event Reasoning with Large Language Models

University of Pennsylvania(宾夕法尼亚大学);University of Michigan(密歇根大学);Carnegie Mellon University(卡内基梅隆大学)

Li Zhang

AI总结:

针对端到端大型语言模型在复杂事件推理中系统性失败且缺乏可解释性的问题,提出三种将LLMs与事件结构化表示相结合的通用方法,在常识推理和规划任务上大幅优于端到端LLMs并提升可解释性。

摘要:

对现实生活事件进行推理是人工智能(AI)和自然语言处理(NLP)中的一项统一挑战,在多种领域具有深远的实用性,而在高风险应用中的推理谬误可能带来灾难性后果。大型语言模型(LLMs)能够处理这些领域中的多样化文本,已被证明具备回答问题和解决问题的能力。然而,我表明,端到端 LLMs 在复杂事件推理上仍然存在系统性失败,并且由于其黑盒性质而缺乏可解释性。为了解决这些问题,我提出了三种将 LLMs 与事件结构化表示相结合的通用方法。第一种是基于语言的表示,涉及子事件之间的关系,LLMs 可以通过微调来学习。第二种是半符号表示,涉及实体状态,LLMs 可以通过少样本提示进行预测并加以利用。第三种是全符号表示,可以由使用结构化数据训练的 LLMs 进行预测,并由符号求解器执行。在一套涵盖常识推理和规划的事件推理任务中,我表明每种方法都大幅优于端到端 LLMs,并具有更强的可解释性。这些结果为 LLMs 与结构化表示在事件推理及其他领域的协同方式提供了启示。

英文摘要:

Reasoning about real-life events is a unifying challenge in AI and NLP that has profound utility in a variety of domains, while fallacy in high-stake applications could be catastrophic. Able to work with diverse text in these domains, large language models (LLMs) have proven capable of answering questions and solving problems. However, I show that end-to-end LLMs still systematically fail to reason about complex events, and they lack interpretability due to their black-box nature. To address these issues, I propose three general approaches to use LLMs in conjunction with a structured representation of events. The first is a language-based representation involving relations of sub-events that can be learned by LLMs via fine-tuning. The second is a semi-symbolic representation involving states of entities that can be predicted and leveraged by LLMs via few-shot prompting. The third is a fully symbolic representation that can be predicted by LLMs trained with structured data and be executed by symbolic solvers. On a suite of event reasoning tasks spanning common-sense inference and planning, I show that each approach greatly outperforms end-to-end LLMs with more interpretability. These results suggest manners of synergy between LLMs and structured representations for event reasoning and beyond.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑