多轮、思维链后编辑用于不忠实摘要
Multi-round, Chain-of-thought Post-editing for Unfaithful Summaries
AI总结：
本研究探索了使用大型语言模型进行新闻摘要的忠实性评估与后编辑，通过思维链提示和多轮编辑策略，有效纠正事实不一致，提升摘要忠实性。
摘要：
近期的大型语言模型（LLMs）在自然语言理解和生成任务上展现出了非凡的能力。在本工作中，我们研究了使用LLMs评估新闻摘要忠实性的方法，发现其与人类判断具有强相关性。我们进一步探究了LLMs作为忠实性后编辑器的能力，通过实验不同的思维链提示来定位并纠正生成摘要与源新闻文档之间的事实不一致，取得了比先前工作更高的编辑成功率。我们对后编辑后的摘要进行了自动和人工评估，发现使用关于事实错误类型的思维链推理来提示LLMs是一种有效的忠实性后编辑策略，其表现可与微调的后编辑模型相媲美。我们还证明了多轮后编辑（此前未被探索）可用于逐步提高那些单轮无法完全纠正错误的摘要的忠实性。
英文摘要：
Recent large language models (LLMs) have demonstrated a remarkable ability to perform natural language understanding and generation tasks. In this work, we investigate the use of LLMs for evaluating faithfulness in news summarization, finding that it achieves a strong correlation with human judgments. We further investigate LLMs' capabilities as a faithfulness post-editor, experimenting with different chain-of-thought prompts for locating and correcting factual inconsistencies between a generated summary and the source news document and are able to achieve a higher editing success rate than was reported in prior work. We perform both automated and human evaluations of the post-edited summaries, finding that prompting LLMs using chain-of-thought reasoning about factual error types is an effective faithfulness post-editing strategy, performing comparably to fine-tuned post-editing models. We also demonstrate that multiple rounds of post-editing, which has not previously been explored, can be used to gradually improve the faithfulness of summaries whose errors cannot be fully corrected in a single round.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
