SALMUBench：一种用于敏感关联级多模态反训练的基准
SALMUBench: A Benchmark for Sensitive Association-Level Multimodal Unlearning
AI总结：
本文提出SALMUBench基准，用于评估多模态反训练中敏感信息删除的效果，通过合成数据和基础模型测试发现现有方法存在遗忘不足或过度泛化问题。
摘要：
随着多模态模型如CLIP成为下游系统的重要组成部分，删除敏感信息的需求变得至关重要。然而，针对对比训练编码器的机器反训练仍处于探索阶段，现有评估未能诊断细粒度的关联级遗忘。我们引入SALMUBench（敏感关联级多模态反训练）基准，基于包含60,000人-属性关联的合成数据集和两个基础模型：一个受污染的模型和一个干净的模型。为了隔离反训练效果，两者均从头在相同的4亿对保留基础数据上训练，受污染的模型还额外训练了敏感数据集。我们提出了一种新的评估协议，包含结构化的抽样集（抽样身份、抽样关联），以精确测量反训练效果和副作用。该基准揭示，尽管实用高效的删除是可行的，但现有方法表现出不同的失败模式：要么无法有效遗忘，要么过度泛化，通过删除更多于预期的内容。SALMUBench为全面的反训练评估设定了新标准，我们公开发布我们的数据集、模型、评估脚本和排行榜以促进未来研究。
英文摘要：
As multimodal models like CLIP become integral to downstream systems, the need to remove sensitive information is critical. However, machine unlearning for contrastively-trained encoders remains underexplored, and existing evaluations fail to diagnose fine-grained, association-level forgetting. We introduce SALMUBench (Sensitive Association-Level Multimodal Unlearning), a benchmark built upon a synthetic dataset of 60K persona-attribute associations and two foundational models: a Compromised model polluted with this data, and a Clean model without it. To isolate unlearning effects, both are trained from scratch on the same 400M-pair retain base, with the Compromised model additionally trained on the sensitive set. We propose a novel evaluation protocol with structured holdout sets (holdout identity, holdout association) to precisely measure unlearning efficacy and collateral damage. Our benchmark reveals that while utility-efficient deletion is feasible, current methods exhibit distinct failure modes: they either fail to forget effectively or over-generalize by erasing more than intended. SALMUBench sets a new standard for comprehensive unlearning evaluation, and we publicly release our dataset, models, evaluation scripts, and leaderboards to foster future research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
