Fine-tuning large pretrained language models (LLMs) is a cornerstone of modern NLP, yet its growing memory demands (driven by backpropagation and large optimizer States) limit deployment in resource-constrained settings. Zero-order (ZO) methods bypass backpropagation by estimating directional derivatives from forward evaluations, offering substantial memory savings. However, classical ZO estimators suffer from high variance and an adverse dependence on the parameter dimensionality $d$, which has constrained their use to low-dimensional problems. In this work, we propose a policy-driven ZO framework that treats the sampling distribution over perturbation directions as a learnable policy and updates it to reduce the variance of directional estimates. We develop a practical algorithm implementing this idea and provide a theoretical analysis, showing that learned sampling distributions improve the quality of gradient information and relax the explicit dependence on $d$ in convergence bounds. Empirically, we validate the approach on challenging LLM fine-tuning benchmarks, demonstrating substantially improved performance compared to standard ZO baselines. Our results suggest that adaptive direction sampling is a promising route to make ZO fine-tuning viable at scale. The source code is available at https://github.com/brain-lab-research/zo_ldsd
GTS: Inference-Time Scaling of Latent Reasoning with a Learnable Gaussian Thought Sampler
Department of Data Science & AI, Monash University(墨尔本大学数据科学与人工智能系);Faculty of Medicine Dentistry and Health Sciences, University of Melbourne(墨尔本大学医学牙科与健康科学学院);Department of Computer Science, University College London(伦敦大学学院计算机科学系)
LEAP: Learnable Pruning for Transformer-based Models
Microsoft(微软);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of California, Berkeley(加州大学伯克利分校);International Computer Science Institute(国际计算机科学研究所)
The Easy, the Hard, and the Learnable: Confidence and Difficulty-Adaptive Policy Optimization for LLM Reasoning
TMLR Group, Department of Computer Science, Hong Kong Baptist University(香港 Baptist 大学计算机科学系 TMLR 组);Stanford University(斯坦福大学);Sydney AI Centre, The University of Sydney(悉尼大学人工智能中心)
LoCA: Location-Aware Cosine Adaptation for Parameter-Efficient Fine-Tuning
University of Electronic Science and Technology of China(电子科技大学);The University of Melbourne(墨尔本大学);Nankai University(南开大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
Key Lab of Intell. Info. Process., Inst. of Comput. Tech., CAS(中国科学院计算技术研究所智能信息处理重点实验室);University of Chinese Academy of Sciences(中国科学院大学);Tsinghua University(清华大学);Peng Cheng Laboratory(鹏城实验室)
Initialization using Update Approximation is a Silver Bullet for Extremely Efficient Low-Rank Fine-Tuning
Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Georgia Institute of Technology(佐治亚理工学院);Massachusetts Institute of Technology(麻省理工学院)
Outlier-weighed Layerwise Sampling for LLM Fine-tuning
Dalian University of Technology(大连理工大学);University of Surrey(Surrey大学);Eindhoven University of Technology(埃因霍温理工大学);University College London(伦敦大学学院);University of Oxford(牛津大学)
Robust Langevin Sampling under State-Dependent Heavy-Tailed Noise with Applications to LLM Fine-Tuning
Elmore Family School of Electrical and Computer Engineering, Purdue University(普渡大学埃尔莫尔家族电气与计算机工程学院);Weldon School of Biomedical Engineering, Purdue University(普渡大学韦尔顿生物医学工程学院)
Parameter Tuning of the Firefly Algorithm by Three Tuning Methods: Standard Monte Carlo, Quasi-Monte Carlo and Latin Hypercube Sampling Methods
School of Science and Technology, Middlesex University(科学与技术学院,中塞克斯大学);Computer Engineering and Informatics, Middlesex University Dubai(计算机工程与信息学,中塞克斯大学迪拜)
The Surprising Effectiveness of Skip-Tuning in Diffusion Sampling
The Hong Kong University of Science and Technology(香港科技大学);University of Chinese Academy of Sciences(中国科学院大学);Academy of Mathematics and Systems Science(数学与系统科学研究院);Huawei Noah’s Ark Lab(华为诺亚方舟实验室);University of Electronic Science and Technology of China(电子科技大学);National University of Singapore(新加坡国立大学)
Fine-Tuning Diffusion Models for Molecular Generation via Reinforcement Learning and Fast Sampling
Department of Computer Science and Engineering, Shanghai Jiao Tong University(上海交通大学计算机科学与工程系);Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)(广东人工智能与数字经济实验室(深圳))
CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuning
King Abdullah University of Science and Technology (KAUST)(阿卜杜拉国王科技大学);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳));University of Sydney(悉尼大学)
Learning Evolution via Optimization Knowledge Adaptation
Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education(教育部智能感知与图像理解重点实验室);International Research Center for Intelligent Perception and Computation(智能感知与计算国际研究中心);Xidian University(西安电子科技大学)