arXiv math/05074222005-07-21math.STstat.TH

Square Root Penalty: Adaptation to the Margin in Classification and in Edge Estimation

A. B. Tsybakov, S. A. van de Geer

备注：: Published at http://dx.doi.org/10.1214/009053604000001066 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)
发表信息：: Annals of Statistics 2005, Vol. 33, No. 3, 1203-1224
DOI：: 10.1214/009053604000001066

英文摘要：

We consider the problem of adaptation to the margin in binary classification. We suggest a penalized empirical risk minimization classifier that adaptively attains, up to a logarithmic factor, fast optimal rates of convergence for the excess risk, that is, rates that can be faster than n^{-1/2}, where n is the sample size. We show that our method also gives adaptive estimators for the problem of edge estimation.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

批量非参数老虎机中的适应性障碍：未知边际的代价的精确刻画

The Adaptivity Barrier in Batched Nonparametric Bandits: Sharp Characterization of the Price of Unknown Margin

Committee on Computational and Applied Mathematics, University of Chicago（芝加哥大学计算与应用数学委员会）；Department of Statistics, University of Chicago（芝加哥大学统计学系）

针对边际参数未知的批量非参数上下文老虎机，提出遗憾膨胀准则刻画未知性代价，证明最优遗憾膨胀随时间范围多项式增长，开发RoBIN算法实现最优膨胀，并揭示批次数超双对数阶时障碍消失的特性。

2025-11-10cs.LGmath.STstat.ML

Sharp Oracle Inequalities for Square Root Regularization

We study a set of regularization methods for high-dimensional linear regression models. These penalized estimators have the square root of the residual sum of squared errors as loss function, and any weakly decomposable norm as penalty function. This fit measure is chosen because of its property that the estimator does not depend on the unknown standard deviation of the noise. On the other hand, a generalized weakly decomposable norm penalty is very useful in being able to deal with different underlying sparsity structures. We can choose a different sparsity inducing norm depending on how we w…

2016-06-27math.STstat.TH

Root-n consistent estimation of the marginal density in some time series models

In this paper, we consider the problem of estimating the marginal density in some nonlinear autoregressive time series models for which the conditional mean and variance have a parametric specification. Under some regularity conditions, we show that a kernel type estimate based on the residuals can be root-n consistent even if the noise density is unknown. Our results, which are shown to be valid for classical time series models such as ARMA or GARCH processes, extend substantially the existing results obtained for some homoscedatic time series models. Asymptotic expansion of our estimator is …

2016-10-28math.STstat.TH

统一森林：改进随机森林中的交互建模和可解释性

Unity Forests: Improving Interaction Modelling and Interpretability in Random Forests

Institute for Medical Information Processing, Biometry and Epidemiology, Faculty of Medicine, Ludwig Maximilian University of Munich (LMU)（慕尼黑路德维希-马克西米利安大学医学信息处理与流行病学研究所）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Institute of General Practice and Health Services Research, Department Clinical Medicine, TUM School of Medicine and Health, Technical University of Munich (TUM)（慕尼黑技术大学医学与健康学院一般医学与健康服务研究研究所）；Institute of AI and Informatics in Medicine, TUM School of Medicine and Health, Technical University of Munich (TUM)（慕尼黑技术大学医学与健康学院医学人工智能与信息学研究所）

统一森林通过优化树根分割提升交互建模和可解释性，改进了随机森林的变量重要性度量和预测性能。

2026-01-11cs.LGstat.COstat.ME

The Group Square-Root Lasso: Theoretical Properties and Fast Algorithms

We introduce and study the Group Square-Root Lasso (GSRL) method for estimation in high dimensional sparse regression models with group structure. The new estimator minimizes the square root of the residual sum of squares plus a penalty term proportional to the sum of the Euclidean norms of groups of the regression parameter vector. The net advantage of the method over the existing Group Lasso (GL)-type procedures consists in the form of the proportionality factor used in the penalty term, which for GSRL is independent of the variance of the error terms. This is of crucial importance in models…

2013-07-31math.STstat.COstat.TH

Risk Bounds for CART Classifiers under a Margin Condition

Risk bounds for Classification and Regression Trees (CART, Breiman et. al. 1984) classifiers are obtained under a margin condition in the binary supervised classification framework. These risk bounds are obtained conditionally on the construction of the maximal deep binary tree and permit to prove that the linear penalty used in the CART pruning algorithm is valid under a margin condition. It is also shown that, conditionally on the construction of the maximal tree, the final selection by test sample does not alter dramatically the estimation accuracy of the Bayes classifier. In the two-class …

2012-03-01math.STstat.MLstat.TH

更多相关论文收起更多论文

$χ^2$-confidence sets in high-dimensional regression

We study a high-dimensional regression model. Aim is to construct a confidence set for a given group of regression coefficients, treating all other regression coefficients as nuisance parameters. We apply a one-step procedure with the square-root Lasso as initial estimator and a multivariate square-root Lasso for constructing a surrogate Fisher information matrix. The multivariate square-root Lasso is based on nuclear norm loss with $\ell_1$-penalty. We show that this procedure leads to an asymptotically $χ^2$-distributed pivot, with a remainder term depending only on the $\ell_1$-error of the…

2015-09-15math.STstat.TH

自适应条件森林采样用于决策依赖不确定性下的谱风险优化

Adaptive Conditional Residual Sampling for Spectral Risk Optimisation under Decision-Dependent Uncertainty

Institute for Global Prosperity, The Bartlett, University College London（全球繁荣研究所，巴特利特学院，伦敦大学学院）

提出ACFS框架，结合广义随机森林、CEM全局搜索和重加权聚焦增强，解决决策依赖分布下的谱风险最小化问题，在重尾和偏态基准上优于现有方法。

2026-10-19cs.LGmath.OCstat.CO

Variable selection and estimation for semi-parametric multiple-index models

In this paper, we propose a novel method to select significant variables and estimate the corresponding coefficients in multiple-index models with a group structure. All existing approaches for single-index models cannot be extended directly to handle this issue with several indices. This method integrates a popularly used shrinkage penalty such as LASSO with the group-wise minimum average variance estimation. It is capable of simultaneous dimension reduction and variable selection, while incorporating the group structure in predictors. Interestingly, the proposed estimator with the LASSO pena…

2015-04-10math.STstat.TH

硬间隔条件下神经网络分类器的超快收敛速率

Super-fast rates of convergence for Neural Networks Classifiers under the Hard Margin Condition

Department of Data Science（数据科学系）；City University of Hong Kong（香港城市大学）；Department of Mathematics（数学系）；School of Mathematics and Statistics（数学与统计学学院）；The University of Sydney（悉尼大学）

本文研究 ReLU 深度神经网络在 Tsybakov 低噪声及硬间隔条件下的二分类问题，证明带平方损失与 ℓ_p 惩罚的经验风险最小化器在回归函数足够光滑时可获得任意快的有限样本超额风险收敛速率。

2025-05-13cs.LGmath.STstat.TH

Risk Bounds for Embedded Variable Selection in Classification Trees

The problems of model and variable selections for classification trees are jointly considered. A penalized criterion is proposed which explicitly takes into account the number of variables, and a risk bound inequality is provided for the tree classifier minimizing this criterion. This penalized criterion is compared to the one used during the pruning step of the CART algorithm. It is shown that the two criteria are similar under some specific margin assumptions. In practice, the tuning parameter of the CART penalty has to be calibrated by hold-out. Simulation studies are performed which confir…

2012-06-23math.STstat.TH

Distributed Machine Learning with Sparse Heterogeneous Data

Motivated by distributed machine learning settings such as Federated Learning, we consider the problem of fitting a statistical model across a distributed collection of heterogeneous data sets whose similarity structure is encoded by a graph topology. Precisely, we analyse the case where each node is associated with fitting a sparse linear model, and edges join two nodes if the difference of their solutions is also sparse. We propose a method based on Basis Pursuit Denoising with a total variation penalty, and provide finite sample guarantees for sub-Gaussian design matrices. Taking the root o…

2021-11-27math.STstat.MLstat.TH

离散边缘Copula极大似然估计的新型Tau信息初始化方法

Novel Tau-Informed Initialization for Maximum Likelihood Estimation of Copulas with Discrete Margins

针对离散边缘（尤其是低计数泊松数据）高斯Copula模型的极大似然估计问题，提出三种基于Kendall tau的初始化方法，结合精确ML估计，提升了低计数场景下的计算效率、收敛性与估计精度。

2025-11-17math.STstat.MEstat.TH