arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.11490cs.CL
收藏

DocTabQA:使用表格回答长文档中的问题

DocTabQA: Answering Questions from Long Documents Using Tables

Peking University(北京大学);University of Science and Technology of China(中国科学技术大学);Microsoft Corporation(微软公司)

Haochen Wang, Kai Hu, Haoyu Dong, Liangcai Gao

AI总结:

本文提出DocTabQA问答设置,要求从长文档中提取信息生成结构化表格作为答案,并构建QTabA数据集,提出两阶段框架DocTabTalk,通过检索相关句子并生成层次化表格,显著提升GPT-4在表格生成任务中的性能。

摘要:

我们研究了一种新的问答(QA)问题设置,称为DocTabQA。在该设置中,给定一篇长文档,目标是通过将答案组织成直接源自文档内容的结构化表格来回答问题。与主要依赖非结构化文本来构建响应的传统QA方法不同,DocTabQA旨在利用结构化表格作为答案,清晰系统地传达信息,从而增强用户理解并突出数据点之间的关系。据我们所知,这一问题此前尚未被探索过。在本文中,我们引入了QTabA数据集,包含300份财务文档,并附有手动标注的1.5k个问题-表格对。最初,我们利用大型语言模型(LLMs)如GPT-4来建立基线。然而,众所周知,LLMs在从长输入序列生成复杂结构化输出时面临困难。为克服这些挑战,我们提出了一个两阶段框架,称为DocTabTalk,它首先从大量文档中检索相关句子,随后基于这些识别出的句子生成层次化表格。DocTabTalk包含两项关键技术创新:AlignLLaMA和TabTalk,专门为协助GPT-4处理DocTabQA而设计,使其能够生成组织更完善、更清晰的层次化表格。在QTabA和RotoWire数据集上进行的全面实验评估表明,我们的DocTabTalk显著提升了GPT-4在我们提出的DocTabQA任务和表格生成任务中的性能。代码和数据集可在https://github.com/SmileWHC/DocTabQA获取以供进一步研究。

英文摘要:

We study a new problem setting of question answering (QA), referred to as DocTabQA. Within this setting, given a long document, the goal is to respond to questions by organizing the answers into structured tables derived directly from the document's content. Unlike traditional QA approaches which predominantly rely on unstructured text to formulate responses, DocTabQA aims to leverage structured tables as answers to convey information clearly and systematically, thereby enhancing user comprehension and highlighting relationships between data points. To the best of our knowledge, this problem has not been previously explored. In this paper, we introduce the QTabA dataset, encompassing 300 financial documents, accompanied by manually annotated 1.5k question-table pairs. Initially, we leverage Large Language Models (LLMs) such as GPT-4 to establish a baseline. However, it is widely acknowledged that LLMs encounter difficulties when tasked with generating intricate, structured outputs from long input sequences. To overcome these challenges, we present a two-stage framework, called DocTabTalk, which initially retrieves relevant sentences from extensive documents and subsequently generates hierarchical tables based on these identified sentences. DocTabTalk incorporates two key technological innovations: AlignLLaMA and TabTalk, which are specifically tailored to assist GPT-4 in tackling DocTabQA, enabling it to generate well-structured, hierarchical tables with improved organization and clarity. Comprehensive experimental evaluations conducted on both QTabA and RotoWire datasets demonstrate that our DocTabTalk significantly enhances the performances of the GPT-4 in our proposed DocTabQA task and the table generation task. The code and dataset are available at https://github.com/SmileWHC/DocTabQA for further research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑