arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.05330cs.IRcs.AI
收藏

神经机器去排序

Neural Machine Unranking

School of Information Management, Wuhan University(武汉大学信息管理学院);Department of Computer Science, Loughborough University(洛桑大学计算机科学系)

Jingrui Hou, Axel Finke, Georgina Cosma

AI总结:

本文提出神经机器去排序任务NuMuR,并设计双目标CoCoL损失,在MS MARCO与TREC CAR上实现有效遗忘且最小化保留和泛化性能损失。

摘要:

我们研究神经信息检索(IR)中的机器遗忘问题,并提出一个称为神经机器去排序(Neural Machine UnRanking,NuMuR)的新任务。该问题源于神经IR系统中对数据隐私合规和选择性信息删除日益增长的需求。现有的任务无关或模型无关遗忘方法主要为分类任务设计,对NuMuR并不理想,原因在于两个核心挑战:第一,神经排序器输出的是未归一化的相关性分数,而非概率分布,这限制了传统师生蒸馏框架的有效性;第二,在纠缠数据场景中,查询和文档同时出现在遗忘集与保留集中,可能导致现有方法的保留性能下降。为解决这些问题,我们提出对比与一致损失(Contrastive and Consistent Loss,CoCoL),这是一个双目标框架。CoCoL包括:(1)对比损失,用于降低遗忘集上的相关性分数,同时维持纠缠样本上的性能;(2)一致损失,用于保持保留集上的准确性。在MS MARCO和TREC CAR数据集上、跨四种神经IR模型进行的大量实验表明,CoCoL实现了充分遗忘,同时使保留性能和泛化性能损失降至最低。与现有技术相比,我们的方法促进了更有效、更可控的数据删除。

英文摘要:

We address the problem of machine unlearning in neural information retrieval (IR), introducing a novel task termed Neural Machine UnRanking (NuMuR). This problem is motivated by growing demands for data privacy compliance and selective information removal in neural IR systems. Existing task- or model- agnostic unlearning approaches, primarily designed for classification tasks, are suboptimal for NuMuR due to two core challenges: (1) neural rankers output unnormalised relevance scores rather than probability distributions, limiting the effectiveness of traditional teacher-student distillation frameworks; and (2) entangled data scenarios, where queries and documents appear simultaneously across both forget and retain sets, may degrade retention performance in existing methods. To address these issues, we propose Contrastive and Consistent Loss (CoCoL), a dual-objective framework. CoCoL comprises (1) a contrastive loss that reduces relevance scores on forget sets while maintaining performance on entangled samples, and (2) a consistent loss that preserves accuracy on retain set. Extensive experiments on MS MARCO and TREC CAR datasets, across four neural IR models, demonstrate that CoCoL achieves substantial forgetting with minimal retain and generalisation performance loss. Our method facilitates more effective and controllable data removal than existing techniques.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑