In this paper, we consider comparison-based adaptive stochastic algorithms for solving numerical optimisation problems. We consider a specific subclass of algorithms that we call comparison-based step-size adaptive randomized search (CB-SARS), where the state variables at a given iteration are a vector of the search space and a positive parameter, the step-size, typically controlling the overall standard deviation of the underlying search distribution.We investigate the linear convergence of CB-SARS on\emph{scaling-invariant} objective functions. Scaling-invariantfunctions preserve the ordering of points with respect to their functionvalue when the points are scaled with the same positive parameter (thescaling is done w.r.t. a fixed reference point). This class offunctions includes norms composed with strictly increasing functions aswell as many non quasi-convex and non-continuousfunctions. On scaling-invariant functions, we show the existence of ahomogeneous Markov chain, as a consequence of natural invarianceproperties of CB-SARS (essentially scale-invariance and invariance tostrictly increasing transformation of the objective function). We thenderive sufficient conditions for \emph{global linear convergence} ofCB-SARS, expressed in terms of different stability conditions of thenormalised homogeneous Markov chain (irreducibility, positivity, Harrisrecurrence, geometric ergodicity) and thus define a general methodologyfor proving global linear convergence of CB-SARS algorithms onscaling-invariant functions. As a by-product we provide aconnexion between comparison-based adaptive stochasticalgorithms and Markov chain Monte Carlo algorithms.
Enhanced randomized Douglas-Rachford method for linear systems: Improved probabilities and adaptive momentum
School of Mathematical Sciences, Beihang University（北京航空航天大学数学科学学院）；LMIB of the Ministry of Education, School of Mathematical Sciences, Beihang University（教育部重点实验室，北京航空航天大学数学科学学院）
Adaptive Randomized Extended Bregman-Kaczmarz Method for Combined Optimization Problems
Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University（同济大学上海智能自主系统研究院）；School of Mathematical Sciences, Tongji University（同济大学数学科学学院）；The Institute for Advanced Study, Shenzhen University（深圳大学高等研究院）
Adaptive-Growth Randomized Neural Networks for PDEs: Algorithms and Numerical Analysis
School of Mathematics and Statistics, Xi’an Jiaotong University（西安交通大学数学与统计学院）；School of Mathematics and Statistics & State Key Laboratory of Multiphase Flow in Power Engineering, Xi’an Jiaotong University（西安交通大学数学与统计学院及动力工程多相流国家重点实验室）；Institute of Applied Physics and Computational Mathematics（应用物理与计算数学研究所）
Adaptive Randomized Tensor Train Rounding using Khatri-Rao Products
Scientific Computing Department, STFC（STFC 科学计算部）；Rutherford Appleton Laboratory（鲁瑟福·阿普尔顿实验室）；Department of Computer Science, Wake Forest University（维克森林大学计算机科学系）；Institute of Mathematics, EPFL（洛桑联邦理工学院数学研究所）；PSI Center for Scientific Computing, Theory and Data（瑞士保罗谢勒研究所科学计算、理论与数据中心）；Department of Mathematics, North Carolina State University（北卡罗来纳州立大学数学系）
Precision-induced Adaptive Randomized Low-Rank Approximation for SVD and Matrix Inversion
Nanjing University of Information Science and Technology（南京信息工程大学）；Xi’an Jiaotong University（西安交通大学）；Xiamen University（厦门大学）；Pengcheng Laboratory（鹏城实验室）
Matrix-free construction of HSS representation using adaptive randomized sampling
University of California, Santa Barbara（加州大学圣塔芭芭拉分校）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）；University of Manchester（曼彻斯特大学）；Livermore Software Technology Corporation（利弗莫尔软件技术公司）
Adaptive-Growth Randomized Neural Networks for Level-Set Computation of Multivalued Nonlinear First-Order PDEs with Hyperbolic Characteristics
School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China（数学与统计学学院，西安交通大学，西安，陕西 710049，中国）；School of Mathematical Sciences & Institute of Natural Sciences, Shanghai Jiao Tong University, Shanghai 200240, China（数学科学学院及自然科学研究所，上海交通大学，上海 200240，中国）；School of Mathematics and Statistics & State Key Laboratory of Multiphase Flow in Power Engineering, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China（数学与统计学学院及电力工程多相流国家重点实验室，西安交通大学，西安，陕西 710049，中国）
Adaptive Randomized Neural Networks for PDEs: Structure-Aware Feature Redistribution and Residual-Driven Block Enrichment
School of Mathematics and Statistics, Xi’an Jiaotong University（西安交通大学数学与统计学院）；School of Mathematics and Statistics & State Key Laboratory of Multiphase Flow in Power Engineering, Xi’an Jiaotong University（西安交通大学数学与统计学院）
Constrained Optimization via Exact Augmented Lagrangian and Randomized Iterative Sketching
University of Chicago（芝加哥大学）；University of California, Berkeley（加州大学伯克利分校）；International Computer Science Institute（国际计算机科学研究所）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）
