GRIP：通过几何路由约束实现混合专家的算法无关机器反学习
GRIP: Algorithm-Agnostic Machine Unlearning for Mixture-of-Experts via Geometric Router Constraints
AI总结：
GRIP通过几何路由约束实现混合专家的算法无关机器反学习，有效消除专家选择偏移并保持模型效用。
摘要：
对于大型语言模型而言，机器反学习（MU）已成为人工智能安全的关键问题，但现有方法无法推广到混合专家（MoE）架构。我们发现传统反学习方法利用了MoE架构的脆弱性：它们操纵路由器将查询导向非专业知识专家，而不是删除知识，导致模型效用损失和表面遗忘。我们提出了几何路由不变性保护（GRIP），一种用于MoE的反学习算法无关框架。我们的核心贡献是一个几何约束，通过将路由梯度更新投影到专家特定的零空间中实现。关键在于将路由稳定性与参数刚性解耦：在保留知识的情况下，离散专家选择保持稳定，而连续路由参数在零空间内保持可塑性，使模型能够经历必要的内部重新配置以满足反学习目标。这迫使反学习优化直接从专家参数中删除知识，而不是利用表面路由操纵的捷径。GRIP作为适配器，限制路由参数更新而不修改底层反学习算法。在大规模MoE模型上的大量实验表明，我们的适配器在所有测试的反学习方法中消除了专家选择偏移（实现了超过95%的路由稳定性）的同时保持其效用。通过防止现有算法利用MoE模型的路由器脆弱性，GRIP将现有反学习研究从密集架构适应到MoEs。
英文摘要：
Machine unlearning in Mixture-of-Experts (MoE) large language models presents a critical yet under-explored challenge. Current unlearning methods applied to MoE architectures often exploit dynamic routing as an optimization shortcut: rather than genuinely erasing knowledge from expert parameters, they manipulate routers to redirect queries away from the originally assigned experts. This not only causes severe utility degradation but also leaves hazardous knowledge intact. Consequently, adversaries can bypass the router to recover sensitive information directly from dormant experts. In this study, we propose Geometric Routing Invariance Preservation (GRIP), an algorithm-agnostic framework that resolves these failure modes by enforcing hard geometric constraints on router updates. By projecting router gradient updates into the null space of the retain set's routing matrix, GRIP suppresses routing manipulation without freezing the router entirely, thereby directing the unlearning pressure into the expert parameters themselves across all relevant experts. GRIP offers two complementary variants: training-time stochastic projection and a post-training closed-form analytical correction. Extensive experiments on two MoE models across hazardous knowledge removal and copyright unlearning benchmarks demonstrate that GRIP restores routing stability from 0.21 to >0.94, improves retain accuracy by up to 89%, and reduces white-box adversarial knowledge recovery from 11% to just 3% while in line with dense-architecture unlearning under black-box prompt attack, establishing geometric constraints as a principled solution for genuine unlearning in sparse MoE architectures.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
