AI 中文总结
本文针对差分隐私零阶微调大语言模型的噪声问题，提出SAGE噪声感知收缩方法，经理论分析和多模型实验验证，该方法在相同隐私预算下多数场景优于基线，且保留内存效率。
AI 中文摘要
差分隐私零阶优化（DP-ZO）仅使用前向评估即可实现大语言模型的内存高效私有微调。现有基于聚合的DP-ZO方法以固定尺度重构模型更新，忽略了训练过程中有用信号的强度会发生变化，导致噪声主导的更新可能获得过大权重，降低模型效用。为解决该问题，本文提出SAGE，一种噪声感知收缩方法，可根据私有化估计的估计信号质量自适应衰减其值。SAGE从观测到的二阶矩中减去已知的高斯噪声方差以估计底层信号能量，通过时间跟踪稳定该估计，并将当前信噪比与预热参考值比较以得到有界收缩因子。作为纯后处理方法，SAGE无需额外隐私预算或模型查询，仅引入恒定额外状态。理论分析表明，收缩可更快降低二次更新风险项，同时保留有用下降作用并限制噪声主导更新的影响。在RoBERTa-large、OPT-1.3B和OPT-6.7B上的实验显示，在相同隐私预算下，SAGE在多数设置中优于现有基线，同时保留了DP-ZO仅前向评估的内存效率。
英文摘要
Differentially private zeroth-order optimization (DP-ZO) enables memory-efficient private fine-tuning of large language models using only forward evaluations. Existing aggregation-based DP-ZO methods reconstruct model updates at a fixed scale, ignoring that the strength of useful signals varies throughout training. Consequently, noise-dominated updates may receive excessive weight and degrade model utility. To address this issue, we propose SAGE, a noise-aware shrinkage method that adaptively attenuates privatized estimates according to their estimated signal quality. SAGE subtracts the known Gaussian noise variance from the observed second moment to estimate the underlying signal energy, stabilizes this estimate through temporal tracking, and compares its current signal-to-noise level with a warm-up reference to derive a bounded shrinkage factor. As pure post-processing, SAGE requires neither additional privacy budget nor model queries and introduces only constant additional state. Our theoretical analysis shows that shrinkage reduces the quadratic update-risk term faster than the linear descent term, preserving useful descent while limiting the influence of noise-dominated updates. Experiments on RoBERTa-large, OPT-1.3B, and OPT-6.7B demonstrate that SAGE outperforms existing baselines in most settings under the same privacy budgets while preserving the forward-only memory efficiency of DP-ZO.
