通过自适应分层采样实现通用的零阶优化
Universally Empowering Zeroth-Order Optimization via Adaptive Layer-wise Sampling
AI总结:
本文提出AdaLeZO框架,通过自适应分层采样和逆概率加权机制,提升零阶优化效率,实验显示在LLaMA和OPT模型上加速1.7至3倍,且不增加内存开销。
摘要:
零阶优化通过仅依赖前向传递提供了一种内存高效的微调大型语言模型的范式。然而,其实际应用受到收敛速度慢和估计方差高的限制。本文分析了零阶算法的运行特性,发现生成扰动和参数更新占训练延迟的40%以上。我们提出AdaLeZO框架,将分层选择过程建模为非平稳多臂老虎机问题,动态分配扰动预算至最敏感参数。引入基于替换采样的逆概率加权机制,保证梯度估计无偏并减少方差。在LLaMA和OPT模型上实验表明,AdaLeZO比现有方法加速1.7至3倍,且作为通用模块提升现有零阶优化器效率,不增加内存开销。
英文摘要:
Zeroth-Order optimization presents a promising memory-efficient paradigm for fine-tuning Large Language Models by relying solely on forward passes. However, its practical adoption is severely constrained by slow wall-clock convergence and high estimation variance. In this work, we dissect the runtime characteristics of ZO algorithms and identify a critical system bottleneck where the generation of perturbations and parameter updates accounts for over 40% of the training latency. We argue that the standard uniform exploration strategy is fundamentally flawed as it fails to account for the heterogeneous sensitivity of layers in deep networks, resulting in computationally wasteful blind searches. To address this structural mismatch, we propose AdaLeZO, an Adaptive Layer-wise ZO optimization framework. By formulating the layer selection process as a non-stationary Multi-Armed Bandit problem, AdaLeZO dynamically allocates the limited perturbation budget to the most sensitive parameters. We further introduce an Inverse Probability Weighting mechanism based on sampling with replacement, which guarantees unbiased gradient estimation while effectively acting as a temporal denoiser to reduce variance. Extensive experiments on LLaMA and OPT models ranging from 6.7B to 30B parameters demonstrate that AdaLeZO achieves 1.7x to 3.0x wall-clock acceleration compared to state-of-the-art methods. Crucially, AdaLeZO functions as a universal plug-and-play module that seamlessly enhances the efficiency of existing ZO optimizers without incurring additional memory overhead.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。