arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2503.03417cs.CLcs.AI
收藏

当声明演变时:评估并增强嵌入模型对错误信息编辑的鲁棒性

When Claims Evolve: Evaluating and Enhancing the Robustness of Embedding Models Against Misinformation Edits

University of Oxford(牛津大学);Alan Turing Institute(艾伦·图灵研究所);World Bank(世界银行);New York University(纽约大学);Meedan

Jabez Magomere, Emanuele La Malfa, Manuel Tonneau, Ashkan Kazemi, Scott Hale

AI总结:

本文提出扰动框架评估嵌入模型对声明编辑的鲁棒性,发现LLM蒸馏模型更稳健但成本更高,并通过训练与推理时缓解方法显著提升声明匹配系统的域内外性能。

摘要:

在线错误信息仍然是一个关键挑战,事实核查人员越来越依赖使用句子嵌入模型来检索相关事实核查结果的声明匹配系统。然而,当用户在线与声明互动时,他们往往会引入编辑,目前尚不清楚检索中使用的当前嵌入模型是否对此类编辑具有鲁棒性。为了研究这一问题,我们引入了一个扰动框架,该框架生成有效且自然的声明变体,使我们能够在多阶段检索流程中评估各种句子嵌入模型的鲁棒性,并评估不同缓解方法的有效性。我们的评估显示,标准嵌入模型在编辑后的声明上表现出明显的性能下降,而LLM蒸馏的嵌入模型以更高的计算成本提供了更好的鲁棒性。尽管强重排序器有助于减少性能下降,但它无法完全弥补第一阶段检索的差距。为了解决这些检索差距,我们评估了训练时和推理时的缓解方法,证明它们可以将域内鲁棒性提高多达17个百分点,并将域外泛化能力提升10个百分点。总体而言,我们的研究结果为声明匹配系统提供了实用的改进,使对不断演变的错误信息进行更可靠的事实核查成为可能。代码和数据可在https://github.com/JabezNzomo99/claim-matching-robustness获取。

英文摘要:

Online misinformation remains a critical challenge, and fact-checkers increasingly rely on claim matching systems that use sentence embedding models to retrieve relevant fact-checks. However, as users interact with claims online, they often introduce edits, and it remains unclear whether current embedding models used in retrieval are robust to such edits. To investigate this, we introduce a perturbation framework that generates valid and natural claim variations, enabling us to assess the robustness of a wide-range of sentence embedding models in a multi-stage retrieval pipeline and evaluate the effectiveness of various mitigation approaches. Our evaluation reveals that standard embedding models exhibit notable performance drops on edited claims, while LLM-distilled embedding models offer improved robustness at a higher computational cost. Although a strong reranker helps to reduce the performance drop, it cannot fully compensate for first-stage retrieval gaps. To address these retrieval gaps, we evaluate train- and inference-time mitigation approaches, demonstrating that they can improve in-domain robustness by up to 17 percentage points and boost out-of-domain generalization by 10 percentage points. Overall, our findings provide practical improvements to claim-matching systems, enabling more reliable fact-checking of evolving misinformation. Code and data are available at https://github.com/JabezNzomo99/claim-matching-robustness.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑