The best classical optimizer for a continuous black-box problem depends on its budget and landscape. Existing routers decide from the dimension and budget, past improvement or features fitted to one benchmark, and few test an optimizer's implicit assumptions online. AMM (Adaptive Multi Mode) routes a known budget among quasi-Newton, trust-region, coordinate, multistart and evolution-strategy optimization engines with online probes of curvature, plateaus, separability, kinks, basins and fractal structure. Each probe has a default, a dimension-by-budget validity range and a cost charged to the budget. In its pre-registered development loop on bbob, 32 of 36 predictions held over the last 16 confirmations. AMM was then tested once on held-out bbob instances 101-110 (D=2-40, 10*D-1000*D) against up to seven competitors. It has the highest target share in 382 of 600 function-dimension-budget cells (236 outright, 146 tied) and, by a descriptive posterior rule, is credibly behind in 35, mostly at 10*D and 30*D. At every budget it also leads the controls built from its own engines (the best engine alone, the best engine per dimension and budget, and a greedy schedule), narrowly at 10*D and 30*D, and nevergrad's NgIohTuned. In pre-registered runs its lead over IPOP-CMA-ES holds under affine and box transformations, on box-constrained bbob and, pooled, on rotated f2-f4, but rotating the separable f3 and f4 removes most of the lead there. On held-out MA-BBOB its final share at 2000*D is level with IPOP-CMA-ES's and its anytime area lower. We published AMM as an open-source Python package (pip install amm-optimizer; https://github.com/BjBodner/amm-optimizer).
Exploring Landscapes for Better Minima along Valleys
State Key Lab of Processors, Institute of Computing Technology, Chinese Academy of Sciences(中国科学院过程工程研究所状态关键实验室);University of Chinese Academy of Sciences(中国科学院大学)
How does the optimizer implicitly bias the model merging loss landscape?
University of Luxembourg(卢森堡大学);ETH Zürich, MPI for Intelligent Systems(苏黎世联邦理工学院、智能系统研究所);Idiap Research Institute(Idiap研究机构);ELLIS Institute Tübingen, MPI for Intelligent Systems, Tübingen AI Center(图宾根ELLIS研究所、智能系统研究所、图宾根人工智能中心)
OPT-GAN: A Broad-Spectrum Global Optimizer for Black-box Problems by Learning Distribution
University of Jinan(济南大学);Quan Cheng Laboratory(泉城实验室);The University of Suwon(水原大学);Victoria University of Wellington(惠灵顿维多利亚大学);Cainiao Network(菜鸟网络)
Schattor: Schatten-family methods for deep learning optimization
School of Data Science, The Chinese University of Hong Kong (Shenzhen)(香港中文大学(深圳)数据科学学院);Department of Industrial Systems Engineering and Management, National University of Singapore(新加坡国立大学工业系统工程与管理系);Department of Mathematics, Linköping University(利乌波德大学数学系)
EM-Aware Physical Synthesis: Neural Inductor Modeling and Intelligent Placement & Routing for RF Circuits
Electrical Engineering, University of Southern California(电气工程,南加州大学);Electrical Engineering and Computer Science, University of California Irvine(电气工程与计算机科学,加州大学伊维特分校)
Chandar Research Lab(钱达尔研究实验室);Mila - Québec AI Institute(米拉 - 魁北克人工智能研究所);Polytechnique Montréal(蒙特利尔理工学院);Samsung SAIT AI Lab Montréal(三星SAIT人工智能实验室蒙特利尔分部);Google DeepMind(谷歌DeepMind)