We provide a novel transcription of monotone operator theory to the non-Euclidean finite-dimensional spaces $\ell_1$ and $\ell_{\infty}$. We first establish properties of mappings which are monotone with respect to the non-Euclidean norms $\ell_1$ or $\ell_{\infty}$. In analogy with their Euclidean counterparts, mappings which are monotone with respect to a non-Euclidean norm are amenable to numerous algorithms for computing their zeros. We demonstrate that several classic iterative methods for computing zeros of monotone operators are directly applicable in the non-Euclidean framework. We present a case-study in the equilibrium computation of recurrent neural networks and demonstrate that casting the computation as a suitable operator splitting problem improves convergence rates.
Contracting Neural Networks: Sharp LMI Conditions with Applications to Integral Control and Deep Learning
Center for Control, Dynamical Systems, and Computation, UC Santa Barbara(加州大学圣塔芭芭拉分校控制、动力系统与计算中心);Department of Electronics and Telecommunications, Politecnico di Torino(都灵理工大学电子与电信系);Graduate School of Advanced Science and Engineering, Hiroshima University(广岛大学高级科学与工程学院)
Convergence Rates for Variational Inequality Projection Neural Networks with a State-Dependent Metric
King Fahd University of Petroleum & Minerals(阿卜杜勒阿齐兹国王石油矿物大学);Interdisciplinary Research Center for Smart Mobility and Logistics, King Fahd University of Petroleum & Minerals(阿卜杜勒阿齐兹国王石油矿物大学智能交通与物流交叉研究中心)
Sydney Institute for Robotics and Intelligent Systems (SIRIS)(悉尼机器人与智能系统研究所);Australian Centre for Field Robotics (ACFR)(澳大利亚野外机器人中心);University of Sydney(悉尼大学)