Must the dimension and accuracy costs of zeroth-order optimization multiply? Global upper bounds for smooth stochastic nonconvex optimization scale as $O(dε^{-4})$, with both factors separately necessary. We show that their product is unavoidable. We establish an $Ω(dε^{-4})$ lower bound for smooth stochastic nonconvex optimization, yielding an $Ω(dδ^{-1}ε^{-3})$ bound for nonsmooth Goldstein stationarity. Both match the best-known full-space upper bounds, establishing joint minimax optimality in dimension and accuracy. These bounds hold against fully adaptive randomized algorithms with unrestricted queries and outputs, even under arbitrary sample reuse. As a byproduct, we establish noiseless exact-value lower bounds for both smooth and Goldstein stationarity, even against randomized algorithms.
Zeroth-Order primal-dual Alternating Projection Gradient Algorithms for Nonconvex Minimax Problems with Coupled linear Constraints
Department of Mathematics, College of Sciences, Shanghai University（上海大学数学系）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；Newtouch Center for Mathematics of Shanghai University（上海大学数学中心）；State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院数学科学国家重点实验室）；School of Mathematical Sciences, University of Chinese Academy of Sciences（中国科学院大学数学科学学院）
Zeroth-Order Alternating Gradient Descent Ascent Algorithms for a Class of Nonconvex-Nonconcave Minimax Problems
Shanghai University（上海大学）；Newtouch Center for Mathematics of Shanghai University（上海大学纽崔克数学中心）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）
Zeroth-Order Stochastic Variance Reduction for Nonconvex Optimization
MIT-IBM Watson AI Lab（麻省理工学院-IBM Watson人工智能实验室）；IBM Research（IBM研究）；Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）；University of Michigan, Ann Arbor（密歇根大学安娜堡分校）
Zeroth-Order Methods for Stochastic Nonconvex Nonsmooth Composite Optimization
Department of Computer Science University of Maryland College Park（计算机科学系大学马里兰大学学院公园分校）；Department of Computer Science and Engineering University of Texas Arlington（计算机科学与工程系德克萨斯大学阿灵顿分校）
Nonconvex Zeroth-Order Stochastic ADMM Methods with Lower Function Query Complexity
University of Pittsburgh（匹兹堡大学）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Duke University（杜克大学）；University of Maryland College Park（马里兰大学帕克分校）
Improved Zeroth-Order Variance Reduced Algorithms and Analysis for Nonconvex Optimization
The Ohio State University（俄亥俄州立大学）；Duke University（杜克大学）
基于一阶技术SVRG和SARAH/SPIDER，最近分别设计了两类求解非凸优化问题的零阶随机算法。本文讨论了几个重要的问题，这些方法仍然是开放的。首先，所有现有的SVRG型零阶算法遭受更差的函数查询复杂度比零阶梯度下降（ZO-GD）或随机梯度下降（ZO-SGD）。在本文中，我们提出了一种新算法ZO-SVRG-Coord-Rand，并对Liu et al. 2018 b中提出的现有ZO-SVRG-Coord算法进行了新的分析，并表明ZO-SVRG-Coord-Rand和ZO-SVRG-Coord（在我们的新分析下）优于其他现有的SVRG型零阶方法以及ZO-GD和ZO-SGD。其次，现有的SPIDER型算法SPIDER-SZO（Fang et al. 2018）具有优越的理论性能，但在实践中会产生大量高斯随机变量以及$\sqrtε$级步长。在本文中，我们开发了一个新的算法ZO-SPIDER-Coord，这是免费的高斯变量的生成，并允许一个大的常数步长，同时保持相同的收敛速度和查询复杂度，我们进一步表明，ZO-SPIDER-Coord自动实现线性收敛速度，因为搜索器进入一个本地PL区域没有重新启动和算法修改。
Zeroth-order Nonconvex Stochastic Optimization: Handling Constraints, High-Dimensionality and Saddle-Points
Department of Statistics, University of California, Davis（加利福尼亚大学戴维斯分校统计系）；Department of Operations Research and Financial Engineering, Princeton University（普林斯顿大学运筹学与金融工程系）
Derivative-free Alternating Projection Algorithms for General Nonconvex-Concave Minimax Problems
Shanghai University（上海大学）；Newtouch Center for Mathematics of Shanghai University（上海大学新触数学中心）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）
Zeroth-Order Methods for Nonconvex Stochastic Problems with Decision-Dependent Distributions
Graduate School of Information Science and Technology, University of Tokyo（东京大学信息科学与技术研究生院）；Center for Advanced Intelligence Project, RIKEN（理化学研究所先进智能项目中心）
Lower Bounds for Smooth Nonconvex Zeroth-Order Optimization: The Price of Determinism under Low Curvature Trace
Faculty of Arts and Sciences, Beijing Normal University（北京师范大学艺术与科学学部）；Institute of Artificial Intelligence and Future Networks, Faculty of Arts and Sciences, Beijing Normal University（北京师范大学艺术与科学学部人工智能与未来网络研究所）；Guangdong Key Lab of AI and Multi-Modal Data Processing, Beijing Normal-Hong Kong Baptist University（京港人工智能与多模态数据处理广东省重点实验室）
Min-Max Optimisation for Nonconvex-Nonconcave Functions Using a Random Zeroth-Order Extragradient Algorithm
School of Engineering Australian National University（工程学院澳大利亚国立大学）；CIICADA Lab（CIICADA实验室）；Department of Electrical Engineering Polytechnique Montréal（电气工程系蒙特利尔理工学院）；GERAD & Mila（GERAD与Mila）；Department of Mathematics and Industrial Engineering Polytechnique Montréal（数学与工业工程系蒙特利尔理工学院）；GERAD；Department of Electrical and Electronic Engineering University of Melbourne（电气与电子工程系墨尔本大学）
ZO-AdaMM: Zeroth-Order Adaptive Momentum Method for Black-Box Optimization
University of Minnesota（明尼苏达大学）；MIT-IBM Watson AI Lab, IBM Research（麻省理工学院-IBM沃森人工智能实验室，IBM研究院）；Northeastern University（东北大学）；Princeton University（普林斯顿大学）
Zeroth-Order Negative Curvature Finding: Escaping Saddle Points without Gradients
Nanjing University of Information Science & Technology（南京信息工程大学）；Harbin Institute of Technology（哈尔滨工业大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
A knowledge-driven nonconvex square-root regularization framework for joint group sparsity and structured smoothness
College of Mathematics and Statistics, Hunan University of Technology and Business（湖南工业大学数学与统计学院）；College of Mathematics and Statistics, Chongqing University（重庆大学数学与统计学院）
Efficient Exact Unlearning for Federated Zeroth-Order Optimization
School of Cyber Science and Technology, Beihang University（北京航空航天大学网络空间科学与技术学院）；Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University（北京航空航天大学未来区块链与隐私计算北京高精尖创新中心）；School of Electrical Engineering and Computer Science, TU Berlin（柏林工业大学电气工程与计算机科学学院）；Division of Computer, Electrical and Mathematical Sciences and Engineering, King Abdullah University of Science and Technology（阿卜杜拉国王科技大学计算机、电气与数学科学与工程学部）
ZorBA: Zeroth-order Federated Fine-tuning of LLMs with Heterogeneous Block Activation
Department of Electrical and Computer Engineering, The University of British Columbia, Vancouver, Canada（电气与计算机工程系，不列颠哥伦比亚大学）；Department of Computer Science and Engineering, Southern University of Science and Technology, China（计算机科学与工程系，南方科技大学）
On the Complexity of Finite-Sum Nonconvex-Strongly-Concave Minimax Optimization with Exact Proximal Oracles
Beijing Normal University（北京师范大学）；Institute of Artificial Intelligence and Future Networks（人工智能与未来网络研究院）；Faculty of Arts and Sciences（文理学院）；Beijing Normal-Hong Kong Baptist University（北京师范大学-香港浸会大学联合国际学院）；Guangdong Key Lab of AI and Multi-Modal Data Processing（广东省人工智能与多模态数据处理重点实验室）
Accelerated Stochastic Gradient-free and Projection-free Methods
College of Computer Science & Technology, Nanjing University of Aeronautics and Astronautics（南京航空航天大学计算机科学与技术学院）；MIIT Key Laboratory of Pattern Analysis & Machine Intelligence（工信部模式分析与机器智能重点实验室）
