arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2204.07447cs.CLcs.LG
收藏

扩展语句对NLI模型以推理长文档和聚类

Stretching Sentence-pair NLI Models to Reason over Long Documents and Clusters

Google Research(谷歌研究院);University of Pennsylvania(宾夕法尼亚大学)

Tal Schuster, Sihao Chen, Senaka Buthpitiya, Alex Fabrikant, Donald Metzler

摘要:

自然语言推理(NLI)作为估计句子对之间的语义关系的框架已经被NLP社区广泛研究。虽然早期的工作发现了NLI模型中的某些偏差,但最近在建模和数据集方面的进展显示出了良好的性能。在这项工作中,我们进一步探索了NLI模型对实际应用的直接zero-shot适用性,超越了它们所训练的双对设置。首先,我们分析了这些模型对较长和域外输入的鲁棒性。然后,我们开发了新的聚合方法,以允许对完整文档进行操作,从而在ContractNLI数据集上达到最先进的性能。有趣的是,我们发现NLI分数提供了强大的检索信号,与常见的基于相似性的方法相比,可以提取更多相关的证据。最后,我们进一步调查整个文件集群,以确定来源之间的差异和共识。在一个测试案例中,我们发现不同语言的维基百科页面之间关于同一主题的真实不一致。

英文摘要:

Natural Language Inference (NLI) has been extensively studied by the NLP community as a framework for estimating the semantic relation between sentence pairs. While early work identified certain biases in NLI models, recent advancements in modeling and datasets demonstrated promising performance. In this work, we further explore the direct zero-shot applicability of NLI models to real applications, beyond the sentence-pair setting they were trained on. First, we analyze the robustness of these models to longer and out-of-domain inputs. Then, we develop new aggregation methods to allow operating over full documents, reaching state-of-the-art performance on the ContractNLI dataset. Interestingly, we find NLI scores to provide strong retrieval signals, leading to more relevant evidence extractions compared to common similarity-based methods. Finally, we go further and investigate whole document clusters to identify both discrepancies and consensus among sources. In a test case, we find real inconsistencies between Wikipedia pages in different languages about the same topic.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑