FALCON：面向大语言模型的对比正交失配细粒度激活操纵
FALCON: Fine-grained Activation Manipulation by Contrastive Orthogonal Unalignment for Large Language Model
AI总结：
针对大语言模型敏感知识遗忘中移除效果与模型效用难以平衡的问题，提出FALCON，通过信息论参数选择、对比表示分离和正交梯度投影实现细粒度激活操纵，实验证明其遗忘更有效、效用保持良好且抗知识恢复。
摘要：
大语言模型已被广泛应用，但可能在不经意间编码敏感或有害信息，引发严重的安全担忧。机器遗忘已被提出以缓解这一问题；然而，现有训练阶段遗忘方法依赖粗粒度损失组合，在精确分离知识以及平衡移除效果与模型效用方面存在局限。相比之下，我们提出通过对比正交失配实现细粒度激活操纵（FALCON），这是一种新颖的表示引导遗忘方法：它利用信息论指导进行高效参数选择，采用对比机制增强表示分离，并将冲突梯度投影到正交子空间，以解决遗忘目标与保留目标之间的冲突。大量实验表明，FALCON在保持模型效用的同时实现了优异的遗忘效果，并对知识恢复尝试表现出强大的抵抗能力。
英文摘要：
Large language models have been widely applied, but can inadvertently encode sensitive or harmful information, raising significant safety concerns. Machine unlearning has emerged to alleviate this concern; however, existing training-time unlearning approaches, relying on coarse-grained loss combinations, have limitations in precisely separating knowledge and balancing removal effectiveness with model utility. In contrast, we propose Fine-grained Activation manipuLation by Contrastive Orthogonal uNalignment (FALCON), a novel representation-guided unlearning approach that leverages information-theoretic guidance for efficient parameter selection, employs contrastive mechanisms to enhance representation separation, and projects conflict gradients onto orthogonal subspaces to resolve conflicts between forgetting and retention objectives. Extensive experiments demonstrate that FALCON achieves superior unlearning effectiveness while maintaining model utility, exhibiting robust resistance against knowledge recovery attempts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
