AMRFact：通过AMR驱动的负样本生成增强摘要事实性评估
AMRFact: Enhancing Summarization Factuality Evaluation with AMR-Driven Negative Samples Generation
AI总结：
本文提出AMRFact框架，通过将摘要解析为AMR图并注入受控错误生成高质量负样本，结合NegFilter数据选择模块，在AggreFact-SOTA基准上显著提升了抽象式摘要事实一致性评估的性能。
摘要：
确保事实一致性对于自然语言生成任务至关重要，尤其是在抽象式摘要中，保持信息的完整性是首要任务。先前关于摘要事实一致性评估的工作通常采用基于蕴含的方法：首先生成扰动（事实不一致）的摘要，然后在生成的数据上训练分类器，以在测试阶段检测事实不一致。然而，先前生成扰动摘要的方法要么连贯性较低，要么缺乏错误类型覆盖。为了解决这些问题，我们提出了AMRFact，这是一个利用抽象意义表示（AMR）生成扰动摘要的框架。我们的方法将事实一致的摘要解析为AMR图，并注入受控的事实不一致以创建负例，从而能够生成具有高错误类型覆盖率的连贯事实不一致摘要。此外，我们提出了一个基于自然语言推理和BARTScore的数据选择模块NegFilter，以确保生成的负样本的质量。实验结果表明，我们的方法在AggreFact-SOTA基准上显著优于先前系统，展示了其在评估抽象式摘要事实性方面的有效性。
英文摘要：
Ensuring factual consistency is crucial for natural language generation tasks, particularly in abstractive summarization, where preserving the integrity of information is paramount. Prior works on evaluating factual consistency of summarization often take the entailment-based approaches that first generate perturbed (factual inconsistent) summaries and then train a classifier on the generated data to detect the factually inconsistencies during testing time. However, previous approaches generating perturbed summaries are either of low coherence or lack error-type coverage. To address these issues, we propose AMRFact, a framework that generates perturbed summaries using Abstract Meaning Representations (AMRs). Our approach parses factually consistent summaries into AMR graphs and injects controlled factual inconsistencies to create negative examples, allowing for coherent factually inconsistent summaries to be generated with high error-type coverage. Additionally, we present a data selection module NegFilter based on natural language inference and BARTScore to ensure the quality of the generated negative samples. Experimental results demonstrate our approach significantly outperforms previous systems on the AggreFact-SOTA benchmark, showcasing its efficacy in evaluating factuality of abstractive summarization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
