神经纠正式机器去排序
Neural Corrective Machine Unranking
AI总结:
本文形式化神经信息检索中的纠正式去排序任务,并提出师生框架CuRD,通过分数模仿、替代文档纠正与性能保持,在多种模型和数据集上优于七个基线。
摘要:
神经信息检索(IR)系统中的机器遗忘要求在移除特定数据的同时保持模型性能。将现有机器遗忘方法应用于IR可能会损害检索效果,或者由于从呈现给用户的检索结果中移除特定项目而无意中暴露遗忘行为。我们形式化定义了纠正式去排序,该任务通过整合替代文档以保持排序完整性,扩展了(神经)IR场景下的机器遗忘;并为此任务提出了一种新颖的师生框架——纠正式去排序蒸馏(Corrective unRanking Distillation,CuRD)。CuRD:(1)通过调整(已训练的)神经IR模型促进遗忘,使其对待遗忘样本输出的相关性分数模仿低排名、不可检索样本的分数;(2)通过微调替代样本的相关性分数,使其紧密匹配相应待遗忘样本的分数,从而实现纠正;(3)力求保持模型在非遗忘目标样本上的性能。我们使用MS MARCO和TREC CAR数据集,在四种神经IR模型(BERTcat、BERTdot、ColBERT、PARADE)上评估CuRD。遗忘集规模为训练数据集1%至20%的实验表明,CuRD在遗忘和纠正方面优于七个最先进基线,同时保持了模型的保留能力和泛化能力。
英文摘要:
Machine unlearning in neural information retrieval (IR) systems requires removing specific data whilst maintaining model performance. Applying existing machine unlearning methods to IR may compromise retrieval effectiveness or inadvertently expose unlearning actions due to the removal of particular items from the retrieved results presented to users. We formalise corrective unranking, which extends machine unlearning in (neural) IR context by integrating substitute documents to preserve ranking integrity, and propose a novel teacher-student framework, Corrective unRanking Distillation (CuRD), for this task. CuRD (1) facilitates forgetting by adjusting the (trained) neural IR model such that its output relevance scores of to-be-forgotten samples mimic those of low-ranking, non-retrievable samples; (2) enables correction by fine-tuning the relevance scores for the substitute samples to match those of corresponding to-be-forgotten samples closely; (3) seeks to preserve performance on samples that are not targeted for forgetting. We evaluate CuRD on four neural IR models (BERTcat, BERTdot, ColBERT, PARADE) using MS MARCO and TREC CAR datasets. Experiments with forget set sizes from 1 % and 20 % of the training dataset demonstrate that CuRD outperforms seven state-of-the-art baselines in terms of forgetting and correction while maintaining model retention and generalisation capabilities.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。