A concentration inequality for the excess risk in least-squares regression with random design and heteroscedastic noise

Slope heuristics and V-Fold model selection in heteroscedastic regression using strongly localized bases

We investigate the optimality for model selection of the so-called slope heuristics, $V$-fold cross-validation and $V$-fold penalization in a heteroscedastic with random design regression context. We consider a new class of linear models that we call strongly localized bases and that generalize histograms, piecewise polynomials and compactly supported wavelets. We derive sharp oracle inequalities that prove the asymptotic optimality of the slope heuristics---when the optimal penalty shape is known---and $V$ -fold penalization. Furthermore, $V$-fold cross-validation seems to be suboptimal for a…

多元异方差经验贝叶斯方法：基于非参数最大似然估计

Multivariate, Heteroscedastic Empirical Bayes via Nonparametric Maximum Likelihood

本研究将非参数最大似然估计扩展到多元异方差误差，提出一种经验贝叶斯去噪方法，具有近最优性能，并应用于天文学和层次线性模型，展示了其自适应性和优势。

广义线性模型中指数加权聚合的Kullback-Leibler超额风险界

Kullback-Leibler excess risk bounds for exponential weighted aggregation in Generalized linear models

本文研究广义线性模型（GLMs）中的稀疏聚合问题，提出指数加权聚合方案，证明其对Kullback-Leibler风险产生首项常数为1的精确oracle不等式且达到极小极大最优速率，并建立超额风险的高概率界。

Proofs of non-optimality of the standard least-squares method for track reconstructions

It is a standard criterium in statistics to define an optimal estimator the one with the minimum variance. Thus, the optimality is proved with inequality among variances of competing estimators. The inequalities, demonstrated here, disfavor the standard least squares estimators. Inequalities among estimators are connected to names of Cramer, Rao and Frechet. The standard demonstrations of these inequalities require very special analytical properties of the probability functions, globally indicated as regular models. These limiting conditions are too restrictive to handle realistic problems in …

用Oracle估计的极值统计

Statistics of extremes by oracle estimation

我们使用拟合的帕累托定律来构造伴随的过剩分布函数的近似。提出了一种基于分段失拟检验方法的超额分布函数位置的选择准则。我们的主要结果是一个预言型不等式的Kullback-Leibler损失。

可交换随机变量加权和的更紧Hoeffding界

A Sharper Hoeffding Bound for Weighted Sums of Exchangeable Random Variables

本文针对有界可交换随机变量加权和，证明了更紧的Hoeffding型矩生成函数界，改进了现有膨胀因子的阶并给出显式常数，还证明了逆总体大小膨胀的不可避免性。

The Rate-Distortion Function and Excess-Distortion Exponent of Sparse Regression Codes with Optimal Encoding

This paper studies the performance of sparse regression codes for lossy compression with the squared-error distortion criterion. In a sparse regression code, codewords are linear combinations of subsets of columns of a design matrix. It is shown that with minimum-distance encoding, sparse regression codes achieve the Shannon rate-distortion function for i.i.d. Gaussian sources $R^*(D)$ as well as the optimal excess-distortion exponent. This completes a previous result which showed that $R^*(D)$ and the optimal exponent were achievable for distortions below a certain threshold. The proof of the…

Robust Estimation of High-Dimensional Mean Regression

Data subject to heavy-tailed errors are commonly encountered in various scientific fields, especially in the modern era with explosion of massive data. To address this problem, procedures based on quantile regression and Least Absolute Deviation (LAD) regression have been devel- oped in recent years. These methods essentially estimate the conditional median (or quantile) function. They can be very different from the conditional mean functions when distributions are asymmetric and heteroscedastic. How can we efficiently estimate the mean regression functions in ultra-high dimensional setting wi…

Logistic regression with total variation regularization

We study logistic regression with total variation penalty on the canonical parameter and show that the resulting estimator satisfies a sharp oracle inequality: the excess risk of the estimator is adaptive to the number of jumps of the underlying signal or an approximation thereof. In particular when there are finitely many jumps, and jumps up are sufficiently separated from jumps down, then the estimator converges with a parametric rate up to a logarithmic term $\log n / n$, provided the tuning parameter is chosen appropriately of order $1/ \sqrt n$. Our results extend earlier results for quad…

重尾噪声下的正则化最小二乘学习是极小极大最优的

Regularized least squares learning with heavy-tailed noise is minimax optimal

本文证明在噪声仅具有有限高阶矩的重尾条件下，再生核希尔伯特空间中的岭回归仍能达到此前仅在次指数噪声假设下成立的极小极大最优收敛速率，从而揭示了正则化最小二乘对重尾噪声的渐近鲁棒性。

Support vector machines with a reject option

This paper studies $\ell_1$ regularization with high-dimensional features for support vector machines with a built-in reject option (meaning that the decision of classifying an observation can be withheld at a cost lower than that of misclassification). The procedure can be conveniently implemented as a linear program and computed using standard software. We prove that the minimizer of the penalized population risk favors sparse solutions and show that the behavior of the empirical risk minimizer mimics that of the population risk minimizer. We also introduce a notion of classification complex…