arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.04114cs.CLcs.LG
收藏

跨域零样本事实一致性评估

Zero-shot Factual Consistency Evaluation Across Domains

University of Potsdam(波茨坦大学)

Raunak Agarwal

AI总结:

本文统一多种事实一致性相关任务训练跨域评估模型,在涵盖22个数据集的基准上超越八个基线,实现最先进性能与跨域泛化。

摘要:

本工作针对文本生成系统中的事实一致性问题。我们统一了自然语言推理、摘要评估、事实性验证和事实一致性评估等任务,以训练能够在多种领域上评估源-目标对事实一致性的模型。我们在一个包含22个数据集的综合基准套件上,将这些模型与八个基线进行了严格评估,该基准套件涵盖了各种任务、领域和文档长度。结果表明,我们的方法在这一异构基准上取得了最先进的性能,同时解决了效率问题并实现了跨域泛化。

英文摘要:

This work addresses the challenge of factual consistency in text generation systems. We unify the tasks of Natural Language Inference, Summarization Evaluation, Factuality Verification and Factual Consistency Evaluation to train models capable of evaluating the factual consistency of source-target pairs across diverse domains. We rigorously evaluate these against eight baselines on a comprehensive benchmark suite comprising 22 datasets that span various tasks, domains, and document lengths. Results demonstrate that our method achieves state-of-the-art performance on this heterogeneous benchmark while addressing efficiency concerns and attaining cross-domain generalization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑