RePAIR：通过提示感知模型修复实现交互式机器去学习
RePAIR: Interactive Machine Unlearning through Prompt-Aware Model Repair
AI总结：
本文提出RePAIR框架，通过提示感知模型修复实现用户交互式机器去学习，解决大语言模型在预训练中吸收有害知识、虚假信息和隐私数据的问题，实现用户自主控制数据。
摘要：
大型语言模型（LLMs）在预训练过程中会固有地吸收有害知识、虚假信息和个人数据，缺乏选择性移除的原生机制。尽管机器去学习提供了系统性的解决方案，但现有方法以提供者为中心，需要重新训练管道、定制保留数据集以及直接干预模型服务提供商（MSPs），从而排除了终端用户对自身数据的控制。我们引入交互式机器去学习（IMU），这是一种新的范式，用户可以在推理时间通过自然语言指令让LLMs忘记特定知识。为了实现IMU，我们提出了RePAIR，一个提示感知的模型修复框架，包括（i）一个用于去学习意图检测的watchdog模型，（ii）一个用于生成修复程序的surgeon模型，以及（iii）一个参数可自主更新的patient模型。RePAIR的核心是开发Steering Through Activation Manipulation with PseudoInverse（STAMP），一种无需训练的单样本去学习方法，通过闭式伪逆更新将MLP激活转向拒绝子空间。其低秩变体将计算复杂度从O(d^3)降低到O(r^3 + r^2 * d)，使在设备上高效去学习成为可能，比基于训练的基线快约3倍。在有害知识抑制、虚假信息纠正和个人数据擦除的广泛实验中，RePAIR实现了接近零的遗忘分数（Acc_f = 0.00，F-RL = 0.00），同时保持模型效用（Acc_r最高达84.47，R-RL最高达0.88），优于六个最先进的基线。这些结果确立了RePAIR作为用户驱动模型编辑的有效且实用框架，推动透明和在设备上的学习知识控制，具有向多模态基础模型扩展的潜力。
英文摘要：
Large language models (LLMs) inherently absorb harmful knowledge, misinformation, and personal data during pretraining on large-scale web corpora, with no native mechanism for selective removal. While machine unlearning offers a principled solution, existing approaches are provider-centric, requiring retraining pipelines, curated retain datasets, and direct intervention by model service providers (MSPs), thereby excluding end users from controlling their own data. We introduce Interactive Machine Unlearning (IMU), a new paradigm in which users can instruct LLMs to forget targeted knowledge through natural language at inference time. To realize IMU, we propose RePAIR, a prompt-aware model repair framework comprising (i) a watchdog model for unlearning intent detection, (ii) a surgeon model for generating repair procedures, and (iii) a patient model whose parameters are updated autonomously. At the core of RePAIR, we develop Steering Through Activation Manipulation with PseudoInverse (STAMP), a training-free, single-sample unlearning method that redirects MLP activations toward a refusal subspace via closed-form pseudoinverse updates. Its low-rank variant reduces computational complexity from O(d^3) to O(r^3 + r^2 * d), enabling efficient on-device unlearning with up to ~3x speedup over training-based baselines. Extensive experiments across harmful knowledge suppression, misinformation correction, and personal data erasure demonstrate that RePAIR achieves near-zero forget scores (Acc_f = 0.00, F-RL = 0.00) while preserving model utility (Acc_r up to 84.47, R-RL up to 0.88), outperforming six state-of-the-art baselines. These results establish RePAIR as an effective and practical framework for user-driven model editing, advancing transparent and on-device control over learned knowledge, with potential extensions to multimodal foundation models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
