arXiv 2305.149872023-10-30cs.CL

探究真实世界信息寻求场景中LLM的表格到文本生成能力

Investigating Table-to-Text Generation Capabilities of LLMs in Real-World Information Seeking Scenarios

Yale University（耶鲁大学）；Allen Institute for AI（艾伦人工智能研究所）；Technical University of Munich（慕尼黑工业大学）

Yilun Zhao, Haowei Zhang, Shengyun Si, Linyong Nan, Xiangru Tang, Arman Cohan

备注：: Camera-ready version for EMNLP 2023 industry track

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文通过四个数据集探究LLM在真实信息寻求场景中的表格到文本生成、自动评估与反馈生成能力，发现GPT-4表现优异，而开源模型与GPT-4仍有显著差距。

摘要：

表格数据在各行各业中普遍存在，用户为了信息寻求目的需要花费大量时间和精力来理解和处理这些数据。大型语言模型（LLMs）的进步显示出提升用户效率的巨大潜力。然而，在真实世界应用中采用LLM进行表格信息寻求的研究仍不充分。本文利用两个真实世界信息寻求场景中的四个数据集，探究不同LLM的表格到文本能力。这些数据集包括用于数据洞察生成的LogicNLG和我们新构建的LoTNLG数据集，以及用于基于查询生成的FeTaQA和我们新构建的F2WTQ数据集。我们围绕三个研究问题组织研究，分别评估LLM在表格到文本生成、自动评估和反馈生成方面的表现。实验结果表明，当前高性能的LLM，特别是GPT-4，能够有效充当表格到文本生成器、评估器和反馈生成器，促进用户在真实世界场景中的信息寻求目的。然而，其他开源LLM（如Tulu和LLaMA-2）与GPT-4模型之间仍存在显著的性能差距。我们的数据和代码已在https://github.com/yale-nlp/LLM-T2T公开。

英文摘要：

Tabular data is prevalent across various industries, necessitating significant time and effort for users to understand and manipulate for their information-seeking purposes. The advancements in large language models (LLMs) have shown enormous potential to improve user efficiency. However, the adoption of LLMs in real-world applications for table information seeking remains underexplored. In this paper, we investigate the table-to-text capabilities of different LLMs using four datasets within two real-world information seeking scenarios. These include the LogicNLG and our newly-constructed LoTNLG datasets for data insight generation, along with the FeTaQA and our newly-constructed F2WTQ datasets for query-based generation. We structure our investigation around three research questions, evaluating the performance of LLMs in table-to-text generation, automated evaluation, and feedback generation, respectively. Experimental results indicate that the current high-performing LLM, specifically GPT-4, can effectively serve as a table-to-text generator, evaluator, and feedback generator, facilitating users' information seeking purposes in real-world scenarios. However, a significant performance gap still exists between other open-sourced LLMs (e.g., Tulu and LLaMA-2) and GPT-4 models. Our data and code are publicly available at https://github.com/yale-nlp/LLM-T2T.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。