AI 中文总结
本文提出QR-Erase框架，结合层定位技术实现高效机器遗忘，在多类遗忘任务中取得优于优化方法的权衡效果，为基础模型提供了SVD的高效替代方案。
AI 中文摘要
机器遗忘旨在从已训练模型中移除指定信息，无需代价高昂的重新训练。现有基于优化的方法常会降低模型无关能力，而基于子空间的方法则依赖计算成本高昂的奇异值分解（SVD）。本文提出QR-Erase，这是一种基于子空间的框架，采用选主元QR分解直接从模型参数中识别并移除任务特定表示；还进一步提出层定位QR-Erase，将更新限制在包含任务特定信息最集中的层。研究表明，选主元QR可实现带界误差的准确子空间恢复，且在温和的谱间隙条件下，恢复的子空间接近最优SVD解。在任务级、跨语言及语音遗忘任务中，QR-Erase相比基于优化的方法实现了更优的遗忘-保留权衡，同时所有指标与SVD的差距均在5%以内；利用低秩和层定位结构还进一步提升了遗忘效果，例如将语音遗忘集准确率从53.1%降至15.7%。这些结果表明，准确的子空间恢复（而非最优重构）足以实现有效遗忘，为现代基础模型提供了一种高效且通用的SVD替代方案。
英文摘要
Machine unlearning seeks to remove targeted information from trained models without requiring costly retraining. Existing optimization-based methods often degrade unrelated capabilities, while subspace-based approaches rely on computationally expensive singular value decompositions (SVD). We introduce QR-Erase, a subspace-based framework that uses Pivoted QR decomposition to identify and remove task-specific representations directly from model parameters. We further propose Layer-Localized QR-Erase, which restricts updates to layers containing the highest concentration of task-specific information. We show that Pivoted QR provides accurate subspace recovery with bounded error, and that under a mild spectral gap condition, the recovered subspace approaches the optimal SVD solution. Across task-level, cross-lingual, and speech unlearning, QR-Erase achieves a stronger forgetting-retention tradeoff than optimization-based methods while remaining within 5% of SVD across all metrics. Exploiting low-rank and layer-localized structure further improves forgetting (for example, reducing speech forget-set accuracy from 53.1% to 15.7%). These results demonstrate that accurate subspace recovery, rather than optimal reconstruction, is sufficient for effective unlearning and provides an efficient and general alternative to SVD-based methods for modern foundation models.
