Yannick Baraud
Starting from the observation of an R^n-Gaussian vector of mean f and covariance matrix σ^2 I_n (I_n is the identity matrix), we propose a method for building a Euclidean confidence ball around f, with prescribed probability of coverage. For each n, we describe its nonasymptotic property and show its optimality with respect to some criteria.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
Adaptive confidence balls are constructed for individual resolution levels as well as the entire mean vector in a multiresolution framework. Finite sample lower bounds are given for the minimum expected squared radius for confidence balls with a prespecified confidence level. The confidence balls are centered on adaptive estimators based on special local block thresholding rules. The radius is derived from an analysis of the loss of this adaptive estimator. In addition adaptive honest confidence balls are constructed which have guaranteed coverage probability over all of $\mathbb{R}^N$ and exp…
We build confidence balls for the common density $s$ of a real valued sample $X_1,...,X_n$. We use resampling methods to estimate the projection of $s$ onto finite dimensional linear spaces and a model selection procedure to choose an optimal approximation space. The covering property is ensured for all $n\geq2$ and the balls are adaptive over a collection of linear spaces.
In the nonparametric Gaussian sequence space model an $\ell^2$-confidence ball $C_n$ is constructed that adapts to unknown smoothness and Sobolev-norm of the infinite-dimensional parameter to be estimated. The confidence ball has exact and honest asymptotic coverage over appropriately defined `self-similar' parameter spaces. It is shown by information-theoretic methods that this `self-similarity' condition is weakest possible.
We study minimax rates for high-dimensional linear regression with additive errors under the $\ell_p\ (1\leq p<\infty)$-losses, where the regression parameter is of weak sparsity. Our lower and upper bounds agree up to constant factors, implying that the proposed estimator is minimax optimal.
In this note, we consider the problem of existence of adaptive confidence bands in the fixed design regression model, adapting ideas in Hoffmann and Nickl (2011) to the present case. In the course of the proof, we show that sup-norm adaptive estimators exist as well in regression.
A simple construction of adaptive confidence sets is proposed in isotonic, convex and unimodal regression. In univariate isotonic regression, the proposed confidence set enjoys uniform coverage over all non-decreasing regression functions. Furthermore, the diameter of the proposed confidence set automatically adapts to the unknown number of pieces of the true parameter, in the sense that the diameter is bounded from above by the minimax risk over the class of $k$-piecewise constant functions. The diameter of the confidence set is a simple increasing function of the number of jumps of the isoto…
Consider a linear regression model with independent and identically normally distributed random errors. Suppose that the parameter of interest is a specified linear combination of the regression parameters. We prove that the usual confidence interval for this parameter is admissible within a broad class of confidence intervals.
Estimation procedures based on recursive algorithms are interesting and powerful techniques that are able to deal rapidly with (very) large samples of high dimensional data. The collected data may be contaminated by noise so that robust location indicators, such as the geometric median, may be preferred to the mean. In this context, an estimator of the geometric median based on a fast and efficient averaged non linear stochastic gradient algorithm has been developed by Cardot, Cénac and Zitt (2013). This work aims at studying more precisely the non asymptotic behavior of this algorithm by givi…
This paper develops a method to construct uniform confidence bands for a nonparametric regression function where a predictor variable is subject to a measurement error. We allow for the distribution of the measurement error to be unknown, but assume the availability of validation data or repeated measurements on the latent predictor variable. The proposed confidence band builds on the deconvolution kernel estimation and a novel application of the multiplier bootstrap method. We establish asymptotic validity of the proposed confidence band. To our knowledge, this is the first paper to derive as…
We construct nonparametric confidence sets for regression functions using wavelets that are uniform over Besov balls. We consider both thresholding and modulation estimators for the wavelet coefficients. The confidence set is obtained by showing that a pivot process, constructed from the loss function, converges uniformly to a mean zero Gaussian process. Inverting this pivot yields a confidence set for the wavelet coefficients, and from this we obtain confidence sets on functionals of the regression curve.
This article considers inference in linear instrumental variables models with many regressors, all of which could be endogenous. We propose the STIV estimator. Identification robust confidence sets are derived by solving linear programs. We present results on rates of convergence, variable selection, confidence sets which adapt to the sparsity, and analyze confidence bands for vectors of linear functions using bias correction. We also provide solutions to some instruments being endogenous. The application is to the EASI demand system.
Confidence sets play a fundamental role in statistical inference. In this paper, we consider confidence intervals for high dimensional linear regression with random design. We first establish the convergence rates of the minimax expected length for confidence intervals in the oracle setting where the sparsity parameter is given. The focus is then on the problem of adaptation to sparsity for the construction of confidence intervals. Ideally, an adaptive confidence interval should have its length automatically adjusted to the sparsity of the unknown regression vector, while maintaining a prespec…
Asymptotic confidence interval for R2 in multiple linear regression
本文在White稳健多元线性回归框架下，于最小矩条件中构造了多重相关系数R²的渐近置信区间，并给出各R²估计量的渐近联合分布，模拟验证了其稳健性及对计数回归的适用性。
We consider nonparametric estimation of mean regression and conditional variance (or volatility) functions in nonlinear stochastic regression models. Simultaneous confidence bands are constructed and the coverage probabilities are shown to be asymptotically correct. The imposed dependence structure allows applications in many linear and nonlinear auto-regressive processes. The results are applied to the S&P 500 Index data.
Credible intervals and bootstrap confidence intervals in monotone regression
本文从频率学派视角分析单调回归中基于可信区间的贝叶斯方法，证明其等价于百分位自助法，并提出表现更优、无需覆盖校正的平滑自助法，证明采用鞅方法。
Adaptive Confidence Sets for Binary Regression without Design Smoothness
该研究针对无设计光滑性的二元回归，构造了渐近诚实、速率最优的自适应置信集，回答了Mukherjee与Sen2018年提出的问题。
Adaptive confidence intervals for regression functions are constructed under shape constraints of monotonicity and convexity. A natural benchmark is established for the minimum expected length of confidence intervals at a given function in terms of an analytic quantity, the local modulus of continuity. This bound depends not only on the function but also the assumed function class. These benchmarks show that the constructed confidence intervals have near minimum expected length for each individual function, while maintaining a given coverage probability for functions within the class. Such ada…
Consider a linear regression model. Fan and Li (2001) describe the smoothly clipped absolute deviation (SCAD) point estimator of the regression parameter vector. To gain insight into the properties of this estimator, they consider an orthonormal design matrix and focus on the estimation of a specified component of this vector. They show that the SCAD point estimator has three attractive properties. We answer the question: To what extent can an interval estimator, centred on the SCAD estimator, have similar attractive properties?
We study a high-dimensional regression model. Aim is to construct a confidence set for a given group of regression coefficients, treating all other regression coefficients as nuisance parameters. We apply a one-step procedure with the square-root Lasso as initial estimator and a multivariate square-root Lasso for constructing a surrogate Fisher information matrix. The multivariate square-root Lasso is based on nuclear norm loss with $\ell_1$-penalty. We show that this procedure leads to an asymptotically $χ^2$-distributed pivot, with a remainder term depending only on the $\ell_1$-error of the…
This paper develops a simple method to construct confidence bands, centered at a principal component analysis (PCA) based estimator, for the slope function in a functional linear regression model with a scalar response variable and a functional predictor variable. The PCA-based estimator is a series estimator with estimated basis functions, and so construction of valid confidence bands for it is a non-trivial challenge. We propose a confidence band that aims at covering the slope function at "most" of points with a prespecified probability (level), and prove its asymptotic validity under suita…
A long-standing problem in the construction of asymptotically correct confidence bands for a regression function $m(x)=E[Y|X=x]$, where $Y$ is the response variable influenced by the covariate $X$, involves the situation where $Y$ values may be missing at random, and where the selection probability, the density function $f(x)$ of $X$, and the conditional variance of $Y$ given $X$ are all completely unknown. This can be particularly more complicated in nonparametric situations. In this paper we propose a new kernel-type regression estimator and study the limiting distribution of the properly no…
This was a revision of arXiv:1105.2454v1 from 2012. It considers a variation on the STIV estimator where, instead of one conic constraint, there are as many conic constraints as moments (instruments) allowing to use more directly moderate deviations for self-normalized sums. The idea first appeared in formula (6.5) in arXiv:1105.2454v1 when some instruments can be endogenous. For reference and to avoid confusion with the STIV estimator, this estimator should be called C-STIV.
Asymptotic confidence bands for the histogram regression estimator
本文提出了一种适用于异方差噪声下多变量非参数回归模型的直方图估计器渐近置信带构造方法，能够处理Hölder正则性低于一的不光滑回归函数。
We propose simultaneous confidence bands of the hyperbolic-type for the contrasts between several nonlinear (curvilinear) regression curves. The critical value of a confidence band is determined from the distribution of the maximum of a chi-square random process defined on the domain of explanatory variables. We use the volume-of-tube method to derive an upper tail probability formula of the maximum of a chi-square random process, which is asymptotically exact and sufficiently accurate in commonly used tail regions. Moreover, we prove that the formula obtained is equivalent to the expectation …
We construct uniform and point-wise asymptotic confidence sets for the single edge in an otherwise smooth image function which are based on rotated differences of two one-sided kernel estimators. Using methods from M-estimation, we show consistency of the estimators of location, slope and height of the edge function and develop a uniform linearization of the contrast process. The uniform confidence bands then rely on a Gaussian approximation of the score process together with anti-concentration results for suprema of Gaussian processes, while point-wise bands are based on asymptotic normality.…
The problem of constructing confidence sets in the high-dimensional linear model with $n$ response variables and $p$ parameters, possibly $p\ge n$, is considered. Full honest adaptive inference is possible if the rate of sparse estimation does not exceed $n^{-1/4}$, otherwise sparse adaptive confidence sets exist only over strict subsets of the parameter spaces for which sparse estimators exist. Necessary and sufficient conditions for the existence of confidence sets that adapt to a fixed sparsity level of the parameter vector are given in terms of minimal $\ell^2$-separation conditions on the…
In this paper we consider the problem of constructing confidence intervals for coefficients of martingale regression models (in particular, time series models) after variable selection. Although constructing confidence intervals are common practice in statistical analysis, it is challenging in our framework due to the data-dependence of the selected model and the correlation among the variables being selected and not selected. We first introduce estimators for the selected coefficients and show that it is consistent under martingale regression model, in which the observations can be dependent …
Fast Near-Optimal Estimation over Symmetric Norm Balls
提出一种多项式时间算法，用于在对称范数单位球约束下实现信号的近最优欧几里得估计，并扩展至随机设计的中等维度线性回归。
Confidence intervals in monotone regression
本文为单调回归函数构建了基于平滑最小二乘估计器（SLSE）的一致自助法置信区间，推导其渐近分布，比较多种估计器效果，提出自动带宽选择方法，并修正了先前研究的错误。
In this paper we offer a unified approach to the problem of nonparametric regression on the unit interval. It is based on a universal, honest and non-asymptotic confidence region which is defined by a set of linear inequalities involving the values of the functions at the design points. Interest will typically centre on certain simplest functions in that region where simplicity can be defined in terms of shape (number of local extremes, intervals of convexity/concavity) or smoothness (bounds on derivatives) or a combination of both. Once some form of regularization has been decided upon the co…
The issue of honesty in constructing confidence sets arises in nonparametric regression. While optimal rate in nonparametric estimation can be achieved and utilized to construct sharp confidence sets, severe degradation of confidence level often happens after estimating the degree of smoothness. Similarly, for high-dimensional regression, oracle inequalities for sparse estimators could be utilized to construct sharp confidence sets. Yet the degree of sparsity itself is unknown and needs to be estimated, causing the honesty problem. To resolve this issue, we develop a novel method to construct …
Uniform asymptotic confidence bands for a multivariate regression function in an inverse regression model with a convolution-type operator are constructed. The results are derived using strong approximation methods and a limit theorem for the supremum of a stationary Gaussian field over an increasing system of sets. As a particular application, asymptotic confidence bands for a time dependent regression function $f_t(x)$ ($x\in \mathbb {R}^d,t\in \mathbb {R}$) in a convolution-type inverse regression model are obtained. Finally, we demonstrate the practical feasibility of our proposed methods …
The problem of existence of adaptive confidence bands for an unknown density $f$ that belongs to a nested scale of Hölder classes over $\mathbb{R}$ or $[0,1]$ is considered. Whereas honest adaptive inference in this problem is impossible already for a pair of Hölder balls $Σ(r),Σ(s),r\ne s$, of fixed radius, a nonparametric distinguishability condition is introduced under which adaptive confidence bands can be shown to exist. It is further shown that this condition is necessary and sufficient for the existence of honest asymptotic confidence bands, and that it is strictly weaker than similar a…
We consider nonparametric regression in the context of functional data, that is, when a random sample of functions is observed on a fine grid. We obtain a functional asymptotic normality result allowing to build simultaneous confidence bands (SCB) for various estimation and inference tasks. Two applications to a SCB procedure for the regression function and to a goodness-of-fit test for curvilinear regression models are proposed. The first one has improved accuracy upon the other available methods while the second can detect local departures from a parametric shape, as opposed to the usual goo…
Consider a linear regression model with n-dimensional response vector, p-dimensional regression parameter beta and independent normally distributed errors. Suppose that the parameter of interest is theta = a^T beta where a is a specified vector. Define the s-dimensional parameter vector tau=C^T beta-t where C and t are specified. Also suppose that we have uncertain prior information that tau=0. Part of our evaluation of a frequentist confidence interval for theta is the ratio (expected length of this confidence interval)/(expected length of standard 1-alpha confidence interval), the scaled exp…
Weak-instrument-robust subvector inference in instrumental variables regression: A subvector Lagrange multiplier test and properties of subvector Anderson-Rubin confidence sets
本文提出了一种弱工具变量鲁棒的子向量拉格朗日乘数检验，并探讨了子向量安德森-鲁宾置信集的性质。
We focus on the construction of confidence corridors for multivariate nonparametric generalized quantile regression functions. This construction is based on asymptotic results for the maximal deviation between a suitable nonparametric estimator and the true function of interest which follow after a series of approximation steps including a Bahadur representation, a new strong approximation theorem and exponential tail inequalities for Gaussian random fields. As a byproduct we also obtain confidence corridors for the regression function in the classical mean regression. In order to deal with th…
We construct confidence sets for the regression function in nonparametric binary regression with an unknown design density. These confidence sets are adaptive in $L^2$ loss over a continuous class of Sobolev type spaces. Adaptation holds in the smoothness of the regression function, over the maximal parameter spaces where adaptation is possible, provided the design density is smooth enough. We identify two key regimes --- one where adaptation is possible, and one where some critical regions must be removed. We address related questions about goodness of fit testing and adaptive estimation of r…
New Confidence Regions for Linear Regression Parameters with Stationary-Ergodic Dependent Errors
本文提出了一种在回归变量和误差 jointly stationary and ergodic 且未指定序列依赖的情况下，利用随机平滑和辅助样本进行回归系数联合置信区域估计的方法，无需直接估计长期方差或参数依赖模型，且在实际应用中表现出良好的覆盖性能和区域体积。
Sparse maximum likelihood estimation for regression models
本文通过似然比置信区域的渐近性质，提出一种具有稀疏最大似然估计解释的一致模型选择准则，可识别所有活跃变量并优于传统信息准则。
This article presents methods for constructing an asymptotic hyperbolic band under the multiple logistic regression model when the predictor variables are restricted to a specific region $\mathscr{X}$. Scheffé's method yields unnecessarily wide, and hence conservative, bands if the predictor variables can be restricted to a certain region. Piegorsch and Casella (1988) developed a procedure to build an asymptotic confidence band for the multiple logistic regression model over particular regions. Those regions are shown to be special cases of the region $\mathscr{X}$, which was first investigate…
This article presents methods for the construction of two-sided and one-sided simultaneous hyperbolic bands for the logistic and probit regression models when the predictor variable is restricted to a given interval. The bands are constructed based on the asymptotic properties of the maximum likelihood estimators. Past articles have considered building two-sided asymptotic confidence bands for the logistic model, such as Piegorsch and Casella (1988). However, the confidence bands given by Piegorsch and Casella are conservative under a single interval restriction, and it is shown in this articl…
This paper studies higher-order inference properties of nonparametric local polynomial regression methods under random sampling. We prove Edgeworth expansions for $t$ statistics and coverage error expansions for interval estimators that (i) hold uniformly in the data generating process, (ii) allow for the uniform kernel, and (iii) cover estimation of derivatives of the regression function. The terms of the higher-order expansions, and their associated rates as a function of the sample size and bandwidth sequence, depend on the smoothness of the population regression function, the smoothness ex…
In the present paper we study the problem of existence of honest and adaptive confidence sets for matrix completion. We consider two statistical models: the trace regression model and the Bernoulli model. In the trace regression model, we show that honest confidence sets that adapt to the unknown rank of the matrix exist even when the error variance is unknown. Contrary to this, we prove that in the Bernoulli model, honest and adaptive confidence sets exist only when the error variance is known a priori. In the course of our proofs we obtain bounds for the minimax rates of certain composite hy…
For a regression problem with a binary label response, we examine the problem of constructing confidence intervals for the label probability conditional on the features. In a setting where we do not have any information about the underlying distribution, we would ideally like to provide confidence intervals that are distribution-free---that is, valid with no assumptions on the distribution of the data. Our results establish an explicit lower bound on the length of any distribution-free confidence interval, and construct a procedure that can approximately achieve this length. In particular, thi…
We consider the problem of online linear regression on individual sequences. The goal in this paper is for the forecaster to output sequential predictions which are, after $T$ time rounds, almost as good as the ones output by the best linear predictor in a given $\ell^1$-ball in $\\R^d$. We consider both the cases where the dimension~$d$ is small and large relative to the time horizon $T$. We first present regret bounds with optimal dependencies on $d$, $T$, and on the sizes $U$, $X$ and $Y$ of the $\ell^1$-ball, the input data and the observations. The minimax regret is shown to exhibit a reg…
Consider the standard linear regression model $\y = \Xmat \betastar + w$, where $\y \in \real^\numobs$ is an observation vector, $\Xmat \in \real^{\numobs \times \pdim}$ is a design matrix, $\betastar \in \real^\pdim$ is the unknown regression vector, and $w \sim \mathcal{N}(0, σ^2 I)$ is additive Gaussian noise. This paper studies the minimax rates of convergence for estimation of $\betastar$ for $\ell_\rpar$-losses and in the $\ell_2$-prediction loss, assuming that $\betastar$ belongs to an $\ell_{\qpar}$-ball $\Ballq(\myrad)$ for some $\qpar \in [0,1]$. We show that under suitable regularit…
We demonstrate and discuss nonasymptotic bounds in probability for the cost of a regression scheme with a general loss function from the perspective of the Rademacher theory, and for the optimality with respect to the average $L^{2}$-distance to the underlying conditional expectations of least squares regression outcomes from the perspective of the Vapnik-Chervonenkis theory. The results follow from an analysis involving independent but possibly nonstationary training samples and can be extended, in a manner that we explain and illustrate, to relevant cases in which the training sample exhibit…
We consider a linear regression model, with the parameter of interest a specified linear combination of the regression parameter vector. We suppose that, as a first step, a data-based model selection (e.g. by preliminary hypothesis tests or minimizing AIC) is used to select a model. It is common statistical practice to then construct a confidence interval for the parameter of interest based on the assumption that the selected model had been given to us a priori. This assumption is false and it can lead to a confidence interval with poor coverage properties. We provide an easily-computed finite…
Gaussian Process Regression and Kernel Ridge Regression are popular nonparametric regression approaches. Unfortunately, they suffer from high computational complexity rendering them inapplicable to the modern massive datasets. To that end a number of approximations have been suggested, some of them allowing for a distributed implementation. One of them is the divide and conquer approach, splitting the data into a number of partitions, obtaining the local estimates and finally averaging them. In this paper we suggest a novel computationally efficient fully data-driven algorithm, quantifying unc…
Consider a linear regression model with regression parameter beta=(beta_1,..., beta_p) and independent normal errors. Suppose the parameter of interest is theta = a^T beta, where a is specified. Define the s-dimensional parameter vector tau = C^T beta - t, where C and t are specified. Suppose that we carry out a preliminary F test of the null hypothesis H_0: tau = 0 against the alternative hypothesis H_1: tau not equal to 0. It is common statistical practice to then construct a confidence interval for theta with nominal coverage 1-alpha, using the same data, based on the assumption that the se…
We propose an optimal experimental design for a curvilinear regression model that minimizes the band-width of simultaneous confidence bands. Simultaneous confidence bands for curvilinear regression are constructed by evaluating the volume of a tube about a curve that is defined as a trajectory of a regression basis vector (Naiman, 1986). The proposed criterion is constructed based on the volume of a tube, and the corresponding optimal design that minimizes the volume of tube is referred to as the tube-volume optimal (TV-optimal) design. For Fourier and weighted polynomial regressions, the prob…
This paper develops inferential methods for a very general class of ill-posed models in econometrics encompassing the nonparametric instrumental variable regression, various functional regressions, and the density deconvolution. We focus on uniform confidence sets for the parameter of interest estimated with Tikhonov regularization, as in Darolles, Fan, Florens, and Renault (2011). Since it is impossible to have inferential methods based on the central limit theorem, we develop two alternative approaches relying on the concentration inequality and bootstrap approximations. We show that expecte…
Asymptotic equivalence of non-parametric regression with spherical regressors and Gaussian white noise
该研究证明了球形t-设计与随机均匀设计在非参数回归中渐近等价于高斯白噪声实验，同时确定了光滑性假设的尖锐性。
Let $S=\sum_{i=1}^{+\infty}λ_{i}Z_{i}$ where the $Z_{i}$'s are i.d.d. positive with $\mathbb{E}\| Z\| ^{3}<+\infty$ and $(λ_{i})_{i\in\mathbb{N}}$ a positive nonincreasing sequence such that $\sumλ_{i}<+\infty$. We study the small ball probability $\mathbb{P}(S<ε) $ when $ε\downarrow0$. We start from a result by Lifshits (1997) who computed this probability by means of the Laplace transform of $S$. We prove that $\mathbb{P}(S<\cdot) $ belongs to a class of functions introduced by de Haan, well-known in extreme value theory, the class of Gamma-varying functions, for which an exponential-integra…
Finite Sample Confidence Regions for Linear Regression Parameters Using Arbitrary Predictors
本研究提出一种利用任意预测器构建线性回归参数有限样本置信区域的方法，该方法对噪声假设少、适用函数范围广，可转化为混合整数线性规划约束实现优化与区间提取，置信区域为空的特性可用于假设检验，且已在合成数据上验证了适用性。
In the Gaussian linear regression model (with unknown mean and variance), we show that the standard confidence set for one or two regression coefficients is admissible in the sense of Joshi (1969). This solves a long-standing open problem in mathematical statistics, and this has important implications on the performance of modern inference procedures post-model-selection or post-shrinkage, particularly in situations where the number of parameters is larger than the sample size. As a technical contribution of independent interest, we introduce a new class of conjugate priors for the Gaussian lo…
Data consisting of samples of probability density functions are increasingly prevalent, necessitating the development of methodologies for their analysis that respect the inherent nonlinearities associated with densities. In many applications, density curves appear as functional response objects in a regression model with vector predictors. For such models, inference is key to understand the importance of density-predictor relationships, and the uncertainty associated with the estimated conditional mean densities, defined as conditional Fréchet means under a suitable metric. Using the Wasserst…
In this paper, we propose to construct confidence bands by bootstrapping the debiased kernel density estimator (for density estimation) and the debiased local polynomial regression estimator (for regression analysis). The idea of using a debiased estimator was recently employed by Calonico et al. (2018b) to construct a confidence interval of the density function (and regression function) at a given point by explicitly estimating stochastic variations. We extend their ideas of using the debiased estimator and further propose a bootstrap approach for constructing simultaneous confidence bands. T…
We consider the problem of constructing honest confidence intervals (CIs) for a scalar parameter of interest, such as the regression discontinuity parameter, in nonparametric regression based on kernel or local polynomial estimators. To ensure that our CIs are honest, we use critical values that take into account the possible bias of the estimator upon which the CIs are based. We show that this approach leads to CIs that are more efficient than conventional CIs that achieve coverage by undersmoothing or subtracting an estimate of the bias. We give sharp efficiency bounds of using different ker…
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。