残差流分析过拟合与结构破坏
Residual Stream Analysis of Overfitting And Structural Disruptions
AI总结：
研究通过分析残差流几何揭示安全数据导致的虚假拒绝问题，提出VCL正则化方法降低虚假拒绝率并保持性能。
摘要：
确保大型语言模型（LLMs）既有用又无害是一个重大挑战：在重复安全数据集上微调，其中不安全提示与标准拒绝模板配对，常导致虚假拒绝，即良性查询被拒绝。我们首先量化这一效应，显示安全数据的token熵和2-gram多样性（0.048）显著低于通用指令数据。为揭示根本原因，我们引入FlowLens，一种基于稳定PCA的残差流几何分析工具，发现安全示例的更高比例集中在少数组件上，减少表示平滑性并驱动虚假拒绝（当安全数据从0%增加到40%时，虚假拒绝率从63%上升到84%）。基于这些见解，我们提出Variance Concentration Loss（VCL），一种辅助正则化器，惩罚中间层残差中的过度方差集中。实证结果表明，VCL通过超过35个百分点降低虚假拒绝率，同时在MMLU和GSM8K等通用基准上维持或提高性能。
英文摘要：
Ensuring that large language models (LLMs) remain both helpful and harmless poses a significant challenge: fine-tuning on repetitive safety datasets, where unsafe prompts are paired with standard refusal templates, often leads to false refusals, in which benign queries are declined. We first quantify this effect, showing that safety data exhibits substantially lower token entropy and 2-gram diversity (0.048) compared to general instruction data. To uncover the root cause, we introduce FlowLens, a stable PCA-based tool for residual-stream geometry analysis, and reveal that higher proportions of safety examples concentrate variance along a few components, reducing representational smoothness and driving false refusals (false refusal rate rises from 63 percent to 84 percent as safety data increases from 0 percent to 40 percent). Guided by these insights, we propose Variance Concentration Loss (VCL), an auxiliary regularizer that penalizes excessive variance concentration in mid-layer residuals. Empirical results demonstrate that VCL reduces false refusals by over 35 percentage points while maintaining or improving performance on general benchmarks such as MMLU and GSM8K.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
