RefusalGuard：面向大语言模型安全的保几何微调方法
RefusalGuard: Geometry-Preserving Fine-Tuning for Safety in LLMs
AI总结：
本研究提出保几何微调框架RefusalGuard，解决大语言模型微调时安全对齐退化问题，在多模型及安全、下游任务基准上表现优于基线，平衡安全与任务性能。
摘要：
对安全对齐的语言模型进行下游任务微调，常导致其弃权（不执行）行为大幅退化，使模型易受对抗性滥用。已有研究表明，安全相关特征编码在模型激活空间的结构化表示中，但微调过程中这些表示如何变化、对齐为何退化仍未被充分理解。本研究探究对齐退化的表示层面机制，分析发现标准微调会引发安全相关表示的系统性漂移、扭曲其几何结构，并在任务优化与安全特征间引入干扰，共同导致有害合规性提升。基于此，我们提出REFUSALGUARD，一种在模型适配过程中保留安全相关结构的表示层面微调框架，该方法约束隐藏表示空间的更新，确保安全介导组件稳定，同时允许在互补方向进行任务特定学习。我们在LLaMA、Gemma、Qwen等多个模型家族，以及AdvBench、DirectHarm4、JailbreakBench等对抗性安全基准和下游实用任务上评估REFUSALGUARD，其攻击成功率与基础安全对齐模型相当，同时保持有竞争力的任务性能，显著优于基线方法。
英文摘要：
Fine-tuning safety-aligned language models for downstream tasks often leads to substantial degradation of refusal behavior, making models vulnerable to adversarial misuse. While prior work has shown that safety-relevant features are encoded in structured representations within the model's activation space, how these representations change during fine-tuning and why alignment degrades remains poorly understood. In this work, we investigate the representation-level mechanisms underlying alignment degradation. Our analysis shows that standard fine-tuning induces systematic drift in safety-relevant representations, distorts their geometric structure, and introduces interference between task optimization and safety features. These effects collectively lead to increased harmful compliance. Motivated by these findings, we introduce REFUSALGUARD, a representation-level fine-tuning framework that preserves safety-relevant structure during model adaptation. Our approach constrains updates in hidden representation space, ensuring that safety-mediating components remain stable while allowing task-specific learning in complementary directions. We evaluate REFUSALGUARD across multiple model families, including LLaMA, Gemma, and Qwen, on adversarial safety benchmarks such as AdvBench, DirectHarm4, and JailbreakBench, as well as downstream utility tasks. Our approach achieves attack success rates comparable to base safety-aligned models while maintaining competitive task performance, significantly outperforming baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
