面向可扩展大语言模型微调的差分隐私零阶方法
Differentially Private Zeroth-Order Methods for Scalable Large Language Model Finetuning
AI总结：
本文提出阶段式差分隐私零阶方法DP-ZOSO，通过动态调度超参数克服DP-SGD的可扩展性瓶颈，并在多种语言模型和任务上显著提升了隐私微调的效用与可扩展性。
摘要：
在任务特定数据集上进行微调是利用预训练大语言模型（LLM）强大能力以应对各种下游任务的广泛采用范式。由于LLM微调的流行及其伴随的隐私担忧，预训练LLM的差分隐私（DP）微调已被广泛用于保护任务特定数据集的隐私。DP LLM微调方法的设计核心在于隐私、效用和可扩展性之间的满意权衡。现有大多数方法建立在DP-SGD这一开创性工作的基础上。尽管基于DP-SGD的微调方法已将DP-SGD的可扩展性推向极限，但不幸的是，它们受限于SGD固有的低效性。本文研究了DP零阶方法用于LLM预训练的潜力，该方法通过用更高效的零阶梯度近似梯度，避免了SGD的可扩展性瓶颈。本文并未将零阶方法视为SGD的直接替代品，而是从理论和实证两方面进行了全面研究。首先，我们提出了阶段式DP零阶方法（DP-ZOSO），该方法动态调度关键超参数。这一设计基于DP随机扰动与零阶方法梯度近似误差之间的协同作用，及其对微调轨迹的影响。我们为所提出的两种方法提供了理论分析。我们在仅编码器的掩码语言模型和仅解码器的自回归语言模型上进行了广泛的实证分析，无论任务类别如何，在可扩展性和效用方面都取得了令人印象深刻的结果（与DPZero相比，当ε=4时，DP-ZOPO在SST-5上提高了4.5%，在MNLI上使用RoBERTa-Large提高了5.5%，在CB上使用OPT-2.7b提高了9.2%，在BoolQ上提高了3.9%，在更复杂的任务上表现出更显著的性能提升）。
英文摘要：
Fine-tuning on task-specific datasets is a widely-embraced paradigm of harnessing the powerful capability of pretrained LLMs for various downstream tasks. Due to the popularity of LLMs fine-tuning and its accompanying privacy concerns, differentially private (DP) fine-tuning of pretrained LLMs has been widely used to safeguarding the privacy of task-specific datasets. Lying at the design core of DP LLM fine-tuning methods is the satisfactory tradeoff among privacy, utility, and scalability. Most existing methods build upon the seminal work of DP-SGD. Despite pushing the scalability of DP-SGD to its limit, DP-SGD-based fine-tuning methods are unfortunately limited by the inherent inefficiency of SGD. In this paper, we investigate the potential of DP zeroth-order methods for LLM pretraining, which avoids the scalability bottleneck of SGD by approximating the gradient with the more efficient zeroth-order gradient. Rather than treating the zeroth-order method as a drop-in replacement for SGD, this paper presents a comprehensive study both theoretically and empirically. First, we propose the stagewise DP zeroth-order method (DP-ZOSO) that dynamically schedules key hyperparameters. This design is grounded on the synergy between DP random perturbation and the gradient approximation error of the zeroth-order method, and its effect on fine-tuning trajectory. We provide theoretical analysis for both proposed methods. We conduct extensive empirical analysis on both encoder-only masked language model and decoder-only autoregressive language model, achieving impressive results in terms of scalability and utility regardless of the class of tasks (compared with DPZero, DP-ZOPO improves $4.5\%$ on SST-5, $5.5\%$ on MNLI with RoBERTa-Large and 9.2\% on CB, 3.9\% on BoolQ with OPT-2.7b when $ε=4$, demonstrates more significant enhancement in performance on more complicated tasks).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
