A new concentration result for regularized risk minimizers

论经验风险最小化子的集中性

On the Concentration of the Minimizers of Empirical Risks

本文研究经验风险最小化子向真实风险最小化子的集中问题，提出适用于度量空间、无界及非凸情形的稳定性假设，并给出依概率和期望的参数化集中速率及多类应用验证。

On concentration for (regularized) empirical risk minimization

Rates of convergence for empirical risk minimizers have been well studied in the literature. In this paper, we aim to provide a complementary set of results, in particular by showing that after normalization, the risk of the empirical minimizer concentrates on a single point. Such results have been established by~\cite{chatterjee2014new} for constrained estimators in the normal sequence model. We first generalize and sharpen this result to regularized least squares with convex penalties, making use of a "direct" argument based on Borell's theorem. We then study generalizations to other loss fu…

On the robustness of minimum norm interpolators and regularized empirical risk minimizers

This article develops a general theory for minimum norm interpolating estimators and regularized empirical risk minimizers (RERM) in linear models in the presence of additive, potentially adversarial, errors. In particular, no conditions on the errors are imposed. A quantitative bound for the prediction error is given, relating it to the Rademacher complexity of the covariates, the norm of the minimum norm interpolator of the errors and the size of the subdifferential around the true parameter. The general theory is illustrated for Gaussian features and several norms: The $\ell_1$, $\ell_2$, g…

Robust learning and complexity dependent bounds for regularized problems

We study Regularized Empirical Risk Minimizers (RERM) and minmax Median-Of-Means (MOM) estimators where the regularization function $ϕ(\cdot)$ is an even convex function. We obtain bounds on the $L_2$-estimation error and the excess risk that depend on $ϕ(f^*)$, where $f^*$ is the minimizer of the risk over a class $F$. The estimators are based on loss functions that are both Lipschitz and convex. Results for the RERM are derived under weak assumptions on the outputs and a sub-Gaussian assumption on the class $\{ (f-f^*)(X), f \in F \}$. Similar results are shown for minmax MOM estimators in a…

ERM and RERM are optimal estimators for regression problems when malicious outliers corrupt the labels

We study Empirical Risk Minimizers (ERM) and Regularized Empirical Risk Minimizers (RERM) for regression problems with convex and $L$-Lipschitz loss functions. We consider a setting where $|\cO|$ malicious outliers contaminate the labels. In that case, under a local Bernstein condition, we show that the $L_2$-error rate is bounded by $ r_N + AL |\cO|/N$, where $N$ is the total number of observations, $r_N$ is the $L_2$-error rate in the non-contaminated setting and $A$ is a parameter coming from the local Bernstein condition. When $r_N$ is minimax-rate-optimal in a non-contaminated setting, th…

正则化M估计量的中心极限定理

A Central Limit Theorem for Regularized M-Estimators

该研究在比例维 regime 下，针对满足特定条件的正则化M估计量，证明了其线性泛函的定量中心极限定理，确定了波动尺度并给出了收敛速率。