缓解生成式人工智能（GenAI）驱动的证据污染以检测脱离上下文的虚假信息
Mitigating GenAI-Powered Evidence Pollution for Out-Of-Context Misinformation Detection
AI总结：
该研究针对GenAI污染证据削弱OOC多模态虚假信息检测的问题，提出跨模态证据重排序与声明-证据推理策略，在基准数据集上验证了其对现有检测器鲁棒性的提升效果。
摘要：
尽管生成式人工智能（GenAI）模型已取得显著成功，但它们被滥用以生成欺骗性内容的情况引发了对网络信息安全日益增长的担忧。脱离上下文（OOC）多模态虚假信息检测系统通常依赖网络检索到的证据来识别被重新用于虚假上下文的图像，但它们越来越受到GenAI污染证据的挑战。现有工作主要集中在验证在声明层面经历了风格改写的声明，并假设证据语料库是干净的。在这项工作中，我们摒弃了这一假设，系统研究了GenAI驱动的证据污染威胁对OOC检测的影响。我们表明，污染的证据可使最先进的检测器的性能降低超过9个百分点。我们提出了两种缓解策略，即跨模态证据重新排序和跨模态声明-证据推理，以应对污染证据带来的挑战。在两个基准数据集上进行的大量实验表明，我们的方法在污染证据存在的情况下有效增强了现有OOC检测器的鲁棒性。源代码和数据可在该httpsURL公开获取。
英文摘要：
While generative artificial intelligence (GenAI) models have achieved significant success, their misuse for generating deceptive content raises growing concerns about online information security. Out-of-context (OOC) multimodal misinformation detection systems typically rely on Web-retrieved evidence to identify images repurposed in false contexts, but they are increasingly challenged by the presence of GenAI-polluted evidence. Existing work mainly focuses on verifying claims that have undergone stylistic rewriting at the claim level and assume a clean evidence corpus. In this work, we remove this assumption and systematically study the impact of GenAI-driven evidence pollution threat on OOC detection. We show that polluted evidence can degrade the performance of state-of-the-art detectors by more than 9 percentage points. We propose two mitigating strategies, cross-modal evidence reranking and cross-modal claim-evidence reasoning, to address the challenge posed by polluted evidence. Extensive experiments on two benchmark datasets demonstrate that our approaches effectively enhance the robustness of existing OOC detectors amidst polluted evidence. The source code and data are publicly available at https://github.com/YanZehong/GenAI-Evidence-Pollution.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
