arXiv 1106.24672011-06math.STstat.TH

Sharp oracle inequalities and slope heuristic for specification probabilities estimation in discrete random fields

Matthieu Lerasle, Daniel Y. Takahashi

备注：: Published at http://dx.doi.org/10.3150/14-BEJ660 in the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm)
发表信息：: Bernoulli 2016, Vol. 22, No. 1, 325-344
DOI：: 10.3150/14-BEJ660

英文摘要：

We study the problem of estimating the one-point specification probabilities in non-necessary finite discrete random fields from partially observed independent samples. Our procedures are based on model selection by minimization of a penalized empirical criterion. The selected estimators satisfy sharp oracle inequalities in $L_2$-risk. We also obtain theoretical results on the slope heuristic for this problem, justifying the slope algorithm to calibrate the leading constant in the penalty. The practical performances of our methods are investigated in two simulation studies. We illustrate the usefulness of our approach by applying the methods to a multi-unit neuronal data from a rat hippocampus.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

A note on sharp oracle bounds for Slope and Lasso

In this paper, we study the sharp oracle bounds for Slope and Lasso and generalize the results in Bellec et al. (2018) to allow the case that the parameter vector is not exactly sparse and obtain the optimal bounds for $\ell_q$ estimation errors with $1\leq q\leq \infty$ by using some extended Restricted Eigenvalue type conditions.

2021-07-23math.STstat.TH

Oracle approach and slope heuristic in context tree estimation

We introduce a general approach to prove oracle properties in context tree selection. The results derive from a concentration condition that is verified, for example, by mixing processes. Moreover, we show the superiority of the oracle approach from a non-asymptotic point of view in simulations where the classical BIC estimator has nice oracle properties even when it does not recover the source. Our second objective is to extend the slope algorithm of \cite{AM08} to context tree estimation. The algorithm gives a practical way to evaluate the leading constant in front of the penalties. We study…

2011-11-09math.STstat.TH

A sharp oracle inequality for Graph-Slope

Following recent success on the analysis of the Slope estimator, we provide a sharp oracle inequality in term of prediction error for Graph-Slope, a generalization of Slope to signals observed over a graph. In addition to improving upon best results obtained so far for the Total Variation denoiser (also referred to as Graph-Lasso or Generalized Lasso), we propose an efficient algorithm to compute Graph-Slope. The proposed algorithm is obtained by applying the forward-backward method to the dual formulation of the Graph-Slope optimization problem. We also provide experiments showing the interes…

2017-11-20math.STstat.TH

Slope meets Lasso: improved oracle bounds and optimality

We show that two polynomial time methods, a Lasso estimator with adaptively chosen tuning parameter and a Slope estimator, adaptively achieve the exact minimax prediction and $\ell_2$ estimation rate $(s/n)\log (p/s)$ in high-dimensional linear regression on the class of $s$-sparse target vectors in $\mathbb R^p$. This is done under the Restricted Eigenvalue (RE) condition for the Lasso and under a slightly more constraining assumption on the design for the Slope. The main results have the form of sharp oracle inequalities accounting for the model misspecification error. The minimax optimal bo…

2017-05-24math.STstat.TH

Sharp Oracle Inequalities for Square Root Regularization

We study a set of regularization methods for high-dimensional linear regression models. These penalized estimators have the square root of the residual sum of squared errors as loss function, and any weakly decomposable norm as penalty function. This fit measure is chosen because of its property that the estimator does not depend on the unknown standard deviation of the noise. On the other hand, a generalized weakly decomposable norm penalty is very useful in being able to deal with different underlying sparsity structures. We can choose a different sparsity inducing norm depending on how we w…

2016-06-27math.STstat.TH

Slope heuristics and V-Fold model selection in heteroscedastic regression using strongly localized bases

We investigate the optimality for model selection of the so-called slope heuristics, $V$-fold cross-validation and $V$-fold penalization in a heteroscedastic with random design regression context. We consider a new class of linear models that we call strongly localized bases and that generalize histograms, piecewise polynomials and compactly supported wavelets. We derive sharp oracle inequalities that prove the asymptotic optimality of the slope heuristics---when the optimal penalty shape is known---and $V$ -fold penalization. Furthermore, $V$-fold cross-validation seems to be suboptimal for a…

2017-03-15math.STstat.MEstat.TH

更多相关论文收起更多论文

The Slope Heuristics in Heteroscedastic Regression

We consider the estimation of a regression function with random design and heteroscedastic noise in a nonparametric setting. More precisely, we address the problem of characterizing the optimal penalty when the regression function is estimated by using a penalized least-squares model selection method. In this context, we show the existence of a minimal penalty, defined to be the maximum level of penalization under which the model selection procedure totally misbehaves. The optimal penalty is shown to be twice the minimal one and to satisfy a non-asymptotic pathwise oracle inequality with leadi…

2015-06-26math.STstat.TH

Optimal model selection in density estimation

We build penalized least-squares estimators using the slope heuristic and resampling penalties. We prove oracle inequalities for the selected estimator with leading constant asymptotically equal to 1. We compare the practical performances of these methods in a short simulation study.

2010-07-26math.STstat.TH

Bounds on the prediction error of penalized least squares estimators with convex penalty

This paper considers the penalized least squares estimator with arbitrary convex penalty. When the observation noise is Gaussian, we show that the prediction error is a subgaussian random variable concentrated around its median. We apply this concentration property to derive sharp oracle inequalities for the prediction error of the LASSO, the group LASSO and the SLOPE estimators, both in probability and in expectation. In contrast to the previous work on the LASSO type methods, our oracle inequalities in probability are obtained at any confidence level for estimators with tuning parameters tha…

2016-09-21math.STstat.TH

Diaconis-Ylvisaker惩罚逻辑回归（含拟合截距）的比例极限渐近理论

Proportional-limit asymptotics for Diaconis-Ylvisaker-penalised logistic regression with fitted intercept

本文为带拟合截距的Diaconis-Ylvisaker惩罚逻辑回归建立比例极限下的估计量渐近理论，给出斜率极限律、检验统计量分布及可行推断方法。

2026-09-10math.STstat.MEstat.TH

Oracle posterior contraction rates under hierarchical priors

We offer a general Bayes theoretic framework to derive posterior contraction rates under a hierarchical prior design: the first-step prior serves to assess the model selection uncertainty, and the second-step prior quantifies the prior belief on the strength of the signals within the model chosen from the first step. In particular, we establish non-asymptotic oracle posterior contraction rates under (i) a local Gaussianity condition on the log likelihood ratio of the statistical experiment, (ii) a local entropy condition on the dimensionality of the models, and (iii) a sufficient mass conditio…

2021-02-10math.STstat.TH

Estimation bounds and sharp oracle inequalities of regularized procedures with Lipschitz loss functions

We obtain estimation error rates and sharp oracle inequalities for regularization procedures of the form \begin{equation*} \hat f \in argmin_{f\in F}\left(\frac{1}{N}\sum_{i=1}^N\ell(f(X_i), Y_i)+λ\|f\|\right) \end{equation*} when $\|\cdot\|$ is any norm, $F$ is a convex class of functions and $\ell$ is a Lipschitz loss function satisfying a Bernstein condition over $F$. We explore both the bounded and subgaussian stochastic frameworks for the distribution of the $f(X_i)$'s, with no assumption on the distribution of the $Y_i$'s. The general results rely on two main objects: a complexity functi…

2017-02-07math.STstat.TH

从离散观测中对高维莱维驱动奥本海姆过程进行稀疏估计

Sparse Estimation for High-Dimensional Lévy-driven Ornstein--Uhlenbeck Processes from Discrete Observations

本文提出基于Lasso和Slope估计量对高维莱维驱动奥本海姆过程进行稀疏估计，并推导非渐近oracle不等式，建立最优收敛速率及样本复杂度。

2026-03-06math.STstat.TH

基于双框架采样框架的半参数高效数据整合

Semiparametric Efficient Data Integration Using the Dual-Frame Sampling Framework

Iowa State University（爱荷华州立大学）

本文提出基于双框架采样框架的半参数高效数据整合方法，通过两种互补估计器在概率与非概率样本整合中实现效率提升和稳健性。

2026-09-02math.STstat.MEstat.TH

Berry-Esseen bounds for Chernoff-type non-standard asymptotics in isotonic regression

A Chernoff-type distribution is a nonnormal distribution defined by the slope at zero of the greatest convex minorant of a two-sided Brownian motion with a polynomial drift. While a Chernoff-type distribution is known to appear as the distributional limit in many non-regular statistical estimation problems, the accuracy of Chernoff-type approximations has remained largely unknown. In the present paper, we tackle this problem and derive Berry-Esseen bounds for Chernoff-type limit distributions in the canonical non-regular statistical estimation problem of isotonic (or monotone) regression. The …

2021-06-22math.PRmath.STstat.TH