Least squares kernel based methods have been widely used in regression problems due to the simple implementation and good generalization performance. Among them, least squares support vector regression (LS-SVR) and extreme learning machine (ELM) are popular techniques. However, the noise sensitivity is a major bottleneck. To address this issue, a generalized loss function, called $\ell_s$-loss, is proposed in this paper. With the support of novel loss function, two kernel based regressors are constructed by replacing the $\ell_2$-loss in LS-SVR and ELM with the proposed $\ell_s$-loss for better noise robustness. Important properties of $\ell_s$-loss, including robustness, asymmetry and asymptotic approximation behaviors, are verified theoretically. Moreover, iteratively reweighted least squares (IRLS) is utilized to optimize and interpret the proposed methods from a weighted viewpoint. The convergence of the proposal are proved, and detailed analyses of robustness are given. Experiments on both artificial and benchmark datasets confirm the validity of the proposed methods.
Deterministic Online Classification: Non-iteratively Reweighted Recursive Least-Squares for Binary Class Rebalancing
Gordon Center for Medical Imaging（戈登医学影像中心）；Center for Advanced Medical Computing and Analysis（高级医学计算与分析中心）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
Efficient Low-rank Identification via Accelerated Iteratively Reweighted Nuclear Norm Minimization
School of Information Science and Technology, ShanghaiTech University（上海科技大学信息科学与技术学院）；Center for Applied Mathematics of Henan Province, Henan University（河南大学河南省应用数学中心）
Alternating Iteratively Reweighted $\ell_1$ and Subspace Newton Algorithms for Nonconvex Sparse Optimization
School of Information Science and Technology, ShanghaiTech University（上海科技大学信息科学与技术学院）；School of Mathematics and Statistics, Henan University（河南大学数学与统计学院）；Center for Applied Mathematics of Henan Province, Henan University（河南大学河南省应用数学中心）
Recovering Simultaneously Structured Data via Non-Convex Iteratively Reweighted Least Squares
University of North Carolina at Charlotte（北卡罗来纳大学夏洛特分校）；Ludwig-Maximilians-Universität München（慕尼黑大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）
Precise asymptotics of reweighted least-squares algorithms for linear diagonal networks
School of Electrical and Computer Engineering, Georgia Institute of Technology（佐治亚理工学院电气与计算机工程学院）；School of Industrial & Systems Engineering, Georgia Institute of Technology（佐治亚理工学院工业与系统工程学院）
Avoiding strict saddle points of nonconvex regularized problems
School of Information Science and Technology, ShanghaiTech University（上海科技大学信息科学与技术学院）；School of Mathematical Sciences, Shenzhen University（深圳大学数学科学学院）；Department of Applied Mathematics, The Hong Kong Polytechnic University（香港理工大学应用数学系）
Iterative Reweighted Framework Based Algorithms for Sparse Linear Regression with Generalized Elastic Net Penalty
Institute of Applied Mathematics, Shenzhen Polytechnic University（深圳职业技术学院应用数学研究所）；School of Mathematics and Statistics, Henan University（河南大学数学与统计学院）；Center for Applied Mathematics of Henan Province, Henan University（河南大学河南省应用数学中心）
Energy-Efficient Processing and Robust Wireless Cooperative Transmission for Edge Inference
ShanghaiTech University（上海科技大学）；Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences（中国科学院上海微系统与信息技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；University of Toronto（多伦多大学）；University of California at Davis（加州大学戴维斯分校）
