KV缓存量化下的对齐崩溃：诊断与缓解
Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation
AI总结：
研究发现低比特KV缓存量化会无声破坏大模型的安全对齐，根源在于安全特征位于低维激活子空间，易受量化噪声影响；提出逐通道缩减（PCR）诊断方法，分类三种失效模式并指导缓解，无需训练即可恢复高达97%的对齐损失。
摘要：
键值（KV）缓存量化被广泛用于减少大语言模型（LLM）推理内存，然而现有评估仅关注困惑度和准确率，未评估安全性影响。在本研究中，我们探索了KV缓存量化下的对齐保持。在11个指令微调模型（3.8B-72B）和5个基准（1,894个提示）上，我们发现低比特量化可以无声地破坏安全对齐：Mistral-7B在仅1.03倍困惑度下丢失了15.2%的拒绝能力，且不存在通用的安全位宽，标准指标无法察觉的尖锐模型特定相变普遍存在。我们识别出根本原因是几何性的：安全特征占据一个低维激活子空间，其对量化噪声的脆弱性比困惑度平均的全表示空间高10^2-10^3倍。受此观察启发，我们提出逐通道缩减（PCR），一种诊断方法，将每个模型分类为三种机制性失效模式之一：异常值压碎安全（安全位于非异常值通道，被异常值驱动的缩放因子连带损害）；异常值即安全（安全与异常值通道重叠，更细粒度无法挽救）；多层稀释（安全分布在许多层，逐层修复失败）。PCR在全部9个主要模型和来自独立家族的1个保留模型上，使用20个校准提示预测了正确的缓解方向。PCR泛化到未见过的提示、模型和生产量化器，包括KIVI，恢复率高达97.2%，而基于注意力的分配方法失败。由此产生的免训练协议，大约需要35 GPU分钟，以最小的内存开销恢复高达97%的丢失对齐，解决了在NVIDIA GPU上使用FP8 KV缓存的生产vLLM服务中确认的漏洞。
英文摘要：
Key-value (KV) cache quantization is widely used to reduce Large Language Model (LLM) inference memory, yet existing evaluations solely focus on measuring perplexity and accuracy without assessing the safety impact. In this study, we explore alignment preservation under KV cache quantization. Across eleven instruction-tuned models (3.8B-72B) and five benchmarks (1,894 prompts), we find that low-bit quantization can silently destroy safety alignment: Mistral-7B loses 15.2% of its refusals at only 1.03x perplexity, and no universal safe bit-width exists, with sharp model-specific phase transitions invisible to standard metrics. We identify that the root cause is geometric: safety features occupy a low-dimensional activation subspace 10^2-10^3x more vulnerable to quantization noise than the full representation space perplexity averages over. Inspired by this observation, we propose Per-Channel Reduction (PCR), a diagnostic that classifies each model into one of three mechanistic failure modes: outlier-crushes-safety, where safety lives in non-outlier channels collaterally damaged by outlier-driven scale factors; outlier-as-safety, where safety overlaps outlier channels and finer granularity cannot rescue it; and multi-layer dilution, where safety is distributed across many layers and per-layer fixes fail. PCR predicts the correct mitigation direction on all nine primary models and one held-out model from an independent family using 20 calibration prompts. PCR generalizes across unseen prompts, models, and production quantizers, including KIVI with up to 97.2% recovery, succeeding where attention-based allocation methods fail. The resulting training-free protocol, requiring approximately 35 GPU-minutes, recovers up to 97% of lost alignment at minimal memory overhead, addressing vulnerabilities confirmed in production vLLM serving with FP8 KV cache on NVIDIA GPUs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
