利用大型语言模型协助从零撰写维基百科风格文章
Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models
AI总结:
本文提出 STORM 写作系统,通过多视角研究、基于可信来源的模拟对话和信息整理生成提纲,提升从零撰写维基百科风格长文的组织性与覆盖广度。
摘要:
我们研究如何应用大型语言模型从零开始撰写有依据、有组织的长篇文章,使其在广度和深度上可与维基百科页面相媲美。这一尚未得到充分探索的问题在写作前阶段带来了新的挑战,包括如何研究主题以及在写作前准备提纲。我们提出了 STORM,这是一个通过检索和多视角提问来综合主题提纲的写作系统。STORM 对写作前阶段进行建模:(1)发现研究给定主题时的多元视角;(2)模拟对话,让持不同视角的写作者基于可信互联网来源向主题专家提问;(3)整理收集到的信息以创建提纲。在评估方面,我们构建了 FreshWiki,这是一个由近期高质量维基百科文章组成的数据集,并制定提纲评估方案以评价写作前阶段。我们还进一步收集了经验丰富的维基百科编辑的反馈。与由提纲驱动、检索增强的基线方法生成的文章相比,STORM 的文章有更多被认为组织良好(绝对提升25%)且覆盖广泛(绝对提升10%)。专家反馈还有助于识别生成有依据长文时的新挑战,例如来源偏见迁移以及对不相关事实的过度关联。
英文摘要:
We study how to apply large language models to write grounded and organized long-form articles from scratch, with comparable breadth and depth to Wikipedia pages. This underexplored problem poses new challenges at the pre-writing stage, including how to research the topic and prepare an outline prior to writing. We propose STORM, a writing system for the Synthesis of Topic Outlines through Retrieval and Multi-perspective Question Asking. STORM models the pre-writing stage by (1) discovering diverse perspectives in researching the given topic, (2) simulating conversations where writers carrying different perspectives pose questions to a topic expert grounded on trusted Internet sources, (3) curating the collected information to create an outline. For evaluation, we curate FreshWiki, a dataset of recent high-quality Wikipedia articles, and formulate outline assessments to evaluate the pre-writing stage. We further gather feedback from experienced Wikipedia editors. Compared to articles generated by an outline-driven retrieval-augmented baseline, more of STORM's articles are deemed to be organized (by a 25% absolute increase) and broad in coverage (by 10%). The expert feedback also helps identify new challenges for generating grounded long articles, such as source bias transfer and over-association of unrelated facts.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。