arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.06945cs.CL
收藏

以知识为中心的文档模板化视图

Knowledge-Centric Templatic Views of Documents

Johns Hopkins University(约翰斯·霍普金斯大学);Microsoft(微软公司)

Isabel Cachola, Silviu Cucerzan, Allen Herring, Vuksan Mijovic, Erik Oveson, Sujay Kumar Jauhar

AI总结:

本文提出将多种文档格式视为同一底层知识的模板化视图,通过结构化中间表示增强LLM生成性能,并设计统一评估框架,在人工评估中82%的生成文档更受青睐。

摘要:

希望与更广泛受众沟通的作者通常会以多种文档格式分享其想法,例如幻灯片、简报、报告和海报。先前关于文档生成的工作通常将每种独立格式的创建视为不同任务,导致学习过程碎片化、模型和方法冗余以及评估彼此割裂。我们将这些文档中的每一种都视为同一底层知识/内容的模板化视图,并旨在统一这些模板化视图的生成与评估。我们首先表明,当前的大语言模型(LLMs)能够在几乎没有监督或完全没有监督的情况下生成各种文档格式。此外,一种涉及结构化中间表示的简单增强方法可以提升性能,尤其是对于较小的模型。随后,我们引入了一个新颖的统一评估框架,该框架可适配于衡量面向异构下游应用的文档生成器的质量。该评估方法能够适配一系列用户定义的标准和应用场景,从而无需针对特定任务设计评估指标。最后,我们进行了人工评估,结果表明人们更喜欢使用我们方法生成的文档中的82%,并且与文献中先前指标相比,该结果与我们的统一评估框架具有更高的相关性。

英文摘要:

Authors seeking to communicate with broader audiences often share their ideas in various document formats, such as slide decks, newsletters, reports, and posters. Prior work on document generation has generally tackled the creation of each separate format to be a different task, leading to fragmented learning processes, redundancy in models and methods, and disjointed evaluation. We consider each of these documents as templatic views of the same underlying knowledge/content, and we aim to unify the generation and evaluation of these templatic views. We begin by showing that current LLMs are capable of generating various document formats with little to no supervision. Further, a simple augmentation involving a structured intermediate representation can improve performance, especially for smaller models. We then introduce a novel unified evaluation framework that can be adapted to measuring the quality of document generators for heterogeneous downstream applications. This evaluation is adaptable to a range of user defined criteria and application scenarios, obviating the need for task specific evaluation metrics. Finally, we conduct a human evaluation, which shows that people prefer 82% of the documents generated with our method, while correlating more highly with our unified evaluation framework than prior metrics in the literature.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑