arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2305.09645cs.CL
收藏

StructGPT:大型语言模型在结构化数据上进行推理的通用框架

StructGPT: A General Framework for Large Language Model to Reason over Structured Data

Renmin University of China(中国人民大学);University of Electronic Science and Technology of China(电子科技大学)

Jinhao Jiang, Kun Zhou, Zican Dong, Keming Ye, Wayne Xin Zhao, Ji-Rong Wen

AI总结:

本文提出StructGPT框架,通过迭代阅读后推理(IRR)方法和调用-线性化-生成流程,提升大型语言模型在结构化数据上的零样本推理能力,显著提升ChatGPT性能并媲美全数据监督微调基线。

摘要:

在本文中,我们研究如何以统一的方式提升大型语言模型(LLMs)在结构化数据上的零样本推理能力。受LLMs工具增强研究的启发,我们开发了一种基于结构化数据解决问答任务的迭代阅读后推理(IRR)方法,称为StructGPT。在我们的方法中,我们构建专门的函数从结构化数据中收集相关证据(即阅读),并让LLMs基于收集到的信息集中进行推理任务(即推理)。特别地,我们提出了一种调用-线性化-生成流程,借助外部接口支持LLMs在结构化数据上进行推理。通过使用提供的接口迭代此流程,我们的方法可以逐步逼近给定查询的目标答案。在三种类型的结构化数据上进行的广泛实验证明了我们方法的有效性,该方法能够显著提升ChatGPT的性能,并取得与全数据监督微调基线相当的性能。我们的代码和数据已在https://github.com/RUCAIBox/StructGPT公开发布。

英文摘要:

In this paper, we study how to improve the zero-shot reasoning ability of large language models~(LLMs) over structured data in a unified way. Inspired by the study on tool augmentation for LLMs, we develop an \emph{Iterative Reading-then-Reasoning~(IRR)} approach for solving question answering tasks based on structured data, called \textbf{StructGPT}. In our approach, we construct the specialized function to collect relevant evidence from structured data (\ie \emph{reading}), and let LLMs concentrate the reasoning task based on the collected information (\ie \emph{reasoning}). Specially, we propose an \emph{invoking-linearization-generation} procedure to support LLMs in reasoning on the structured data with the help of the external interfaces. By iterating this procedures with provided interfaces, our approach can gradually approach the target answer to a given query. Extensive experiments conducted on three types of structured data demonstrate the effectiveness of our approach, which can significantly boost the performance of ChatGPT and achieve comparable performance against the full-data supervised-tuning baselines. Our codes and data are publicly available at~\url{https://github.com/RUCAIBox/StructGPT}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑