Q-realign: 量化驱动的对齐以实现安全高效的LLM部署
Q-realign: Piggybacking Realignment on Quantization for Safe and Efficient LLM Deployment
AI总结：
Q-realign通过量化驱动的对齐方法，在部署流程中实现安全高效的LLM部署，有效减少不安全行为并降低计算开销。
摘要：
公开的大型语言模型（LLMs）通常在预训练过程中进行安全对齐，但用于部署的任务特定微调往往削弱这种对齐并引入安全风险。现有的防御措施要么将安全恢复嵌入微调过程，要么依赖微调衍生的先验知识进行事后修正，导致安全恢复与训练紧密耦合，并带来高计算开销和复杂的流程。为了解决这些挑战，我们提出了Q-realign，一种基于训练后量化的后处理防御方法，通过分析表示结构进行指导。通过将量化重新框架为压缩和安全的双重目标过程，Q-realign将安全对齐与微调解耦，并自然地融入现代部署流程。在多个模型和数据集上的实验表明，我们的方法显著减少了不安全行为，同时保持任务性能，同时在内存使用和GPU小时上实现了显著减少。值得注意的是，我们的方法可以在40分钟内使用单块RTX 4090恢复微调7B LLM的安全对齐。总体而言，我们的工作提供了一种实用的、即开即用的解决方案，用于安全意识的部署。
英文摘要：
Public large language models (LLMs) are typically safety-aligned during pretraining, yet task-specific fine-tuning required for deployment often erodes this alignment and introduces safety risks. Existing defenses either embed safety recovery into fine-tuning or rely on fine-tuning-derived priors for post-hoc correction, leaving safety recovery tightly coupled with training and incurring high computational overhead and a complex workflow. To address these challenges, we propose \texttt{Q-realign}, a post-hoc defense method based on post-training quantization, guided by an analysis of representational structure. By reframing quantization as a dual-objective procedure for compression and safety, \texttt{Q-realign} decouples safety alignment from fine-tuning and naturally piggybacks into modern deployment pipelines. Experiments across multiple models and datasets demonstrate that our method substantially reduces unsafe behaviors while preserving task performance, with significant reductions in memory usage and GPU hours. Notably, our approach can recover the safety alignment of a fine-tuned 7B LLM on a single RTX 4090 within 40 minutes. Overall, our work provides a practical, turnkey solution for safety-aware deployment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
