Maryna Prus
We consider minimax-optimal designs for the prediction of individual parameters in random coefficient regression models. We focus on the minimax-criterion, which minimizes the "worst case" for the basic criterion with respect to the covariance matrix of random effects. We discuss particular models: linear and quadratic regression, in detail.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
We study minimax rates for high-dimensional linear regression with additive errors under the $\ell_p\ (1\leq p<\infty)$-losses, where the regression parameter is of weak sparsity. Our lower and upper bounds agree up to constant factors, implying that the proposed estimator is minimax optimal.
Distribution-on-distribution regression considers the problem of formulating and estimating a regression relationship where both covariate and response are probability distributions. The optimal transport distributional regression model postulates that the conditional Fréchet mean of the response distribution is linked to the covariate distribution via an optimal transport map. We establish the minimax rate of estimation of such a regression function, by deriving a lower-bound that matches the convergence rate attained by the Fréchet least squares estimator.
We consider the problem of testing the hypothesis that the parameter of linear regression model is 0 against an s-sparse alternative separated from 0 in the l2-distance. We show that, in Gaussian linear regression model with p < n, where p is the dimension of the parameter and n is the sample size, the non-asymptotic minimax rate of testing has the form sqrt((s/n) log(1 + sqrt(p)/s )). We also show that this is the minimax rate of estimation of the l2-norm of the regression parameter.
We consider the problem of estimating the predictive density of future observations from a non-parametric regression model. The density estimators are evaluated under Kullback--Leibler divergence and our focus is on establishing the exact asymptotics of minimax risk in the case of Gaussian errors. We derive the convergence rate and constant for minimax risk among Bayesian predictive densities under Gaussian priors and we show that this minimax risk is asymptotically equivalent to that among all density estimators.
Minimax $L_2$ risks for high-dimensional nonparametric regression are derived under two sparsity assumptions: (1) the true regression surface is a sparse function that depends only on $d=O(\log n)$ important predictors among a list of $p$ predictors, with $\log p=o(n)$; (2) the true regression surface depends on $O(n)$ predictors but is an additive function where each additive component is sparse but may contain two or more interacting predictors and may have a smoothness level different from other components. For either modeling assumption, a practicable extension of the widely used Bayesian …
Regularization is an essential element of virtually all kernel methods for nonparametric regression problems. A critical factor in the effectiveness of a given kernel method is the type of regularization that is employed. This article compares and contrasts members from a general class of regularization techniques, which notably includes ridge regression and principal component regression. We derive an explicit finite-sample risk bound for regularization-based estimators that simultaneously accounts for (i) the structure of the ambient function space, (ii) the regularity of the true regression…
We consider the problem of testing a particular type of composite null hypothesis under a nonparametric multivariate regression model. For a given quadratic functional $Q$, the null hypothesis states that the regression function $f$ satisfies the constraint $Q[f]=0$, while the alternative corresponds to the functions for which $Q[f]$ is bounded away from zero. On the one hand, we provide minimax rates of testing and the exact separation constants, along with a sharp-optimal testing procedure, for diagonal and nonnegative quadratic functionals. We consider smoothness classes of ellipsoidal form…
Confidence sets play a fundamental role in statistical inference. In this paper, we consider confidence intervals for high dimensional linear regression with random design. We first establish the convergence rates of the minimax expected length for confidence intervals in the oracle setting where the sparsity parameter is given. The focus is then on the problem of adaptation to sparsity for the construction of confidence intervals. Ideally, an adaptive confidence interval should have its length automatically adjusted to the sparsity of the unknown regression vector, while maintaining a prespec…
A Minimax Theory of Nonparametric Regression Under Covariate Shift
本文提出了一种非参数回归在协变量偏移下的极小极大理论，引入转移函数并展示了其域性质对收敛速率的影响，覆盖了协变量无界支持的情况。
Minimax Optimal rates of convergence in the shuffled regression, unlinked regression, and deconvolution under vanishing noise
本文研究消失噪声下的单调函数估计问题，推导了混洗回归、无链接回归的极小极大风险阶数关系及反卷积误差界，且未对链接函数加光滑性假设，填补了相关领域空白。
The question of fast convergence in the classical problem of high dimensional linear regression has been extensively studied. Arguably, one of the fastest procedures in practice is Iterative Hard Thresholding (IHT). Still, IHT relies strongly on the knowledge of the true sparsity parameter $s$. In this paper, we present a novel fast procedure for estimation in the high dimensional linear regression. Taking advantage of the interplay between estimation, support recovery and optimization we achieve both optimal statistical accuracy and fast convergence. The main advantage of our procedure is tha…
Minimax Optimal Rates for Regression on Manifolds and Distributions
针对协变量与响应均位于低维流形的分布回归问题，本文推导 minimax 下界，并提出结合对抗学习和同步最小二乘的混合估计量以获得匹配上界。
We consider high-dimensional multivariate linear regression models, where the joint distribution of covariates and response variables is a multivariate normal distribution with a bandable covariance matrix. The main goal of this paper is to estimate the regression coefficient matrix, which is a function of the bandable covariance matrix. Although the tapering estimator of covariance has the minimax optimal convergence rate for the class of bandable covariances, we show that it has a sub-optimal convergence rate for the regression coefficient; that is, a minimax estimator for the class of banda…
Minimax rates of convergence for nonparametric regression under adversarial attacks
本文研究非参数回归在输入对抗攻击下的鲁棒性极限，推导对抗上确界范数的极小极大收敛速率，并证明由标准极小极大最优估计量构造的对抗式插入方法可达最优速率。
Demographic Parity Constrained Minimax Optimal Regression under Linear Model
该研究在线性模型框架下探究人口统计均等约束回归的极小极大最优误差，提出涵盖更广泛歧视偏差来源的模型，推导得出误差阶为$Θ(\ rac{dM}{n})$，并验证误差随模型偏差增大而上升的规律。
The theoretical advances on the properties of scoring rules over the past decades have broadened the use of scoring rules in probabilistic forecasting. In meteorological forecasting, statistical postprocessing techniques are essential to improve the forecasts made by deterministic physical models. Numerous state-of-the-art statistical postprocessing techniques are based on distributional regression evaluated with the Continuous Ranked Probability Score (CRPS). However, theoretical properties of such evaluation with the CRPS have solely considered the unconditional framework (i.e. without covar…
We present estimators for a well studied statistical estimation problem: the estimation for the linear regression model with soft sparsity constraints ($\ell_q$ constraint with $0<q\leq1$) in the high-dimensional setting. We first present a family of estimators, called the projected nearest neighbor estimator and show, by using results from Convex Geometry, that such estimator is within a logarithmic factor of the optimal for any design matrix. Then by utilizing a semi-definite programming relaxation technique developed in [SIAM J. Comput. 36 (2007) 1764-1776], we obtain an approximation algor…
Adaptive Kernel Ridge Regression with Linear Structure: Sharp Oracle Inequalities and Minimax Optimality
本文提出一种改进的岭回归方法，通过引入显式线性成分提升模型性能，理论证明其在一般核函数下达到最小最大最优预测风险。
Adaptive and non-adaptive minimax rates for weighted Laplacian-eigenmap based nonparametric regression
本文针对基于加权拉普拉斯特征映射的非参数回归，在索伯列夫空间设定下建立了自适应与非自适应极小极大收敛速率，将已有结果推广到包括非归一化和随机游走拉普拉斯在内的广泛加权拉普拉斯矩阵。
An Efficient Minimax Optimal Estimator For Multivariate Convex Regression
本文提出了一种高效且最小最大最优的多变量凸回归估计器，适用于高维数据和多面体支持条件，解决了传统方法的最小二乘估计器次优问题。
Minimax rates of convergence for nonparametric location-scale models
本文在Hellinger可微性等温和条件下，证明非参数位置-尺度模型中回归函数估计的极小极大收敛速率由函数类的度量熵决定，并通过多种误差分布实例及低阶交互模型和多指标模型加以验证。
Minimax Theory of Likelihood-Based Deep Learning for Speckle Regression
研究乘性散斑噪声和加性高斯噪声模型下平滑非参数回归函数的极小极大估计，用基于似然的深度神经网络估计器，建立估计误差上界与极小极大下界，证明估计难度不变，数值实验验证方法有效性。
Minimax Linear Regression under the Quantile Risk
本文研究分位数风险下的线性回归 minimax 估计，给出高斯可实现情形的精确风险与 OLS 最优性，并证明 min-max regression 变体在更广分布类上近似 minimax。
k Nearest Neighbor (kNN) method is a simple and popular statistical method for classification and regression. For both classification and regression problems, existing works have shown that, if the distribution of the feature vector has bounded support and the probability density function is bounded away from zero in its support, the convergence rate of the standard kNN method, in which k is the same for all test samples, is minimax optimal. On the contrary, if the distribution has unbounded support, we show that there is a gap between the convergence rate achieved by the standard kNN method a…
We study minimax lower bounds for function estimation problems on large graph when the target function is smoothly varying over the graph. We derive minimax rates in the context of regression and classification problems on graphs that satisfy an asymptotic shape assumption and with a smoothness condition on the target function, both formulated in terms of the graph Laplacian.
In this paper we study the statistical properties of Principal Components Regression with Laplacian Eigenmaps (PCR-LE), a method for nonparametric regression based on Laplacian Eigenmaps (LE). PCR-LE works by projecting a vector of observed responses ${\bf Y} = (Y_1,\ldots,Y_n)$ onto a subspace spanned by certain eigenvectors of a neighborhood graph Laplacian. We show that PCR-LE achieves minimax rates of convergence for random design regression over Sobolev spaces. Under sufficient smoothness conditions on the design density $p$, PCR-LE achieves the optimal rates for both estimation (where th…
The Cost of Discretization in Functional Linear Regression: Minimax Rates and Adaptation
研究标量对函数线性回归在有限噪声点评估下的极小极大估计与预测风险，推导两种采样方案下的速率，构建自适应估计器，揭示不同设计下的相变及丰富相图，数值模拟和实际数据示例验证结果。
This paper considers point and interval estimation of the $\ell_q$ loss of an estimator in high-dimensional linear regression with random design. We establish the minimax rate for estimating the $\ell_{q}$ loss and the minimax expected length of confidence intervals for the $\ell_{q}$ loss of rate-optimal estimators of the regression vector, including commonly used estimators such as Lasso, scaled Lasso, square-root Lasso and Dantzig Selector. Adaptivity of the confidence intervals for the $\ell_{q}$ loss is also studied. Both the setting of known identity design covariance matrix and known no…
In this paper we study the statistical properties of Laplacian smoothing, a graph-based approach to nonparametric regression. Under standard regularity conditions, we establish upper bounds on the error of the Laplacian smoothing estimator $\widehat{f}$, and a goodness-of-fit test also based on $\widehat{f}$. These upper bounds match the minimax optimal estimation and testing rates of convergence over the first-order Sobolev class $H^1(\mathcal{X})$, for $\mathcal{X}\subseteq \mathbb{R}^d$ and $1 \leq d < 4$; in the estimation problem, for $d = 4$, they are optimal modulo a $\log n$ factor. Ad…
Quantile regression is the task of estimating a specified percentile response, such as the median, from a collection of known covariates. We study quantile regression with rectified linear unit (ReLU) neural networks as the chosen model class. We derive an upper bound on the expected mean squared error of a ReLU network used to estimate any quantile conditional on a set of covariates. This upper bound only depends on the best possible approximation error, the number of layers in the network, and the number of nodes per layer. We further show upper bounds that are tight for two large classes of…
A minimax optimal approach to high-dimensional double sparse linear regression
针对高维双稀疏线性回归问题，提出动态更新阈值的IHT-style迭代硬阈值流程，证明其极小极大最优性，设计自适应方案应对未知稀疏度与噪声，实现更优估计与支撑集恢复效果。
In this contribution we introduce weakly locally stationary time series through the local approximation of the non-stationary covariance structure by a stationary one. This allows us to define autoregression coefficients in a non-stationary context, which, in the particular case of a locally stationary Time Varying Autoregressive (TVAR) process, coincide with the generating coefficients. We provide and study an estimator of the time varying autoregression coefficients in a general setting. The proposed estimator of these coefficients enjoys an optimal minimax convergence rate under limited smo…
Minimax Analysis for Inverse Risk in Nonparametric Planer Invertible Regression
本文研究二维平面上可逆双Lipschitz函数逆估计的最小极大风险，提出水平集表示法并构造渐近可逆估计器，证明其速率与非可逆情形一致，表明可逆约束不增加估计复杂度。
Minimax Adaptive Online Nonparametric Regression over Besov Spaces
本文针对Besov空间中的在线对抗回归问题，提出无需预知光滑性参数的小波自适应算法，并进一步设计局部自适应扩展以动态追踪空间非均匀光滑性，从而在异质环境中获得超越全局方法的更精细极小极大遗憾界。
Resolvable Fourier Degree and Minimax Estimation on theBoolean Hypercube
本文研究布尔超立方体上傅里叶-索博列夫预算下的高维非参数回归，提出可解傅里叶度刻画极小极大风险，并扩展至单调回归及未知支撑情形。
Efficient and Minimax Optimal In-context Nonparametric Regression with Transformers
本文研究了基于上下文学习的非参数回归，针对α-Holder光滑回归函数，证明了使用预训练的变换器可以达到最优收敛率，且参数和预训练序列数量显著少于现有文献。
Minimaxity and Efficiency in Exponential Family Regression: From Star-Shaped to Convex Constraints
本文针对星形约束下非参数指数族回归，确立了其极小极大估计率，提出多项式时间高效算法并推导了具体例子的极小极大最优率。
Consider the heteroscedastic nonparametric regression model with random design \begin{align*} Y_i = f(X_i) + V^{1/2}(X_i)\varepsilon_i, \quad i=1,2,\ldots,n, \end{align*} with $f(\cdot)$ and $V(\cdot)$ $α$- and $β$-Hölder smooth, respectively. We show that the minimax rate of estimating $V(\cdot)$ under both local and global squared risks is of the order \begin{align*} n^{-\frac{8αβ}{4αβ+ 2α+ β}} \vee n^{-\frac{2β}{2β+1}}, \end{align*} where $a\vee b := \max\{a,b\}$ for any two real numbers $a,b$. This result extends the fixed design rate $n^{-4α} \vee n^{-2β/(2β+1)}$ derived in Wang et al. [2…
In this article we study the estimation of the location of jump points in the first derivative (referred to as kinks) of a regression function μin two random design models with different long-range dependent (LRD) structures. The method is based on the zero-crossing technique and makes use of high-order kernels. The rate of convergence of the estimator is contingent on the level of dependence and the smoothness of the regression function μ. In one of the models, the convergence rate is the same as the minimax rate for kink estimation in the fixed design scenario with i.i.d. errors which sugges…
In this paper we study minimax and adaptation rates in general isotonic regression. For uniform deterministic and random designs in $[0,1]^d$ with $d\ge 2$ and $N(0,1)$ noise, the minimax rate for the $\ell_2$ risk is known to be bounded from below by $n^{-1/d}$ when the unknown mean function $f$ is nondecreasing and its range is bounded by a constant, while the least squares estimator (LSE) is known to nearly achieve the minimax rate up to a factor $(\log n)^γ$ where $n$ is sample size, $γ= 4$ in the lattice design and $γ= \max\{9/2, (d^2+d+1)/2 \}$ in the random design. Moreover, the LSE is …
Characterizing the minimax rate of nonparametric regression under bounded star-shaped constraints
本文在星形函数类上刻画非参数回归的极小极大速率，提出同时推广 sup-范数有界与 L-亚高斯条件的假设，并给出首个在凸约束下对真实点自适应的估计量。
Minimax optimal transfer learning for high-dimensional additive regression
本文在迁移学习框架下研究高维可加回归，提出基于平滑后拟合估计器的目标估计程序和两阶段估计方法，在次Weibull噪声下建立误差界，并在分布足够接近时实现极小化最优速率。
We establish optimal convergence rates for a decomposition-based scalable approach to kernel ridge regression. The method is simple to describe: it randomly partitions a dataset of size N into m subsets of equal size, computes an independent kernel ridge regression estimator for each subset, then averages the local solutions into a global predictor. This partitioning leads to a substantial reduction in computation time versus the standard approach of performing kernel ridge regression on all N samples. Our two main theorems establish that despite the computational speed-up, statistical optimal…
Regression problems are traditionally analyzed via univariate characteristics like the regression function, scale function and marginal density of regression errors. These characteristics are useful and informative whenever the association between the predictor and the response is relatively simple. More detailed information about the association can be provided by the conditional density of the response given the predictor. For the first time in the literature, this article develops the theory of minimax estimation of the conditional density for regression settings with fixed and random desig…
Adversarial learning for nonparametric regression: Minimax rate and adaptive estimation
本文在非参数回归框架下研究未来X攻击的对抗学习统计最优性，建立了对抗L_q风险下的极小极大收敛速率，提出达到该速率的分段局部多项式估计量，并构造了在广泛非参数与对抗类上近乎最优的数据驱动自适应估计量。
Conformalized Quantile Regression and Minimax Limits of Fixed-Score Calibration under Known Covariate Shift
本文为分裂保形分位数回归建立非渐近$L^p$误差界，并在已知协变量偏移下推导匹配的极小极大上下界，适用于稀疏ReLU网络。\n
On the Optimality of Functional Sliced Inverse Regression
本文证明函数型切片逆回归在估计函数型充分降维中心空间时达极小极大最优速率，通过集中不等式、截断方案及模拟验证。
Consider the standard linear regression model $\y = \Xmat \betastar + w$, where $\y \in \real^\numobs$ is an observation vector, $\Xmat \in \real^{\numobs \times \pdim}$ is a design matrix, $\betastar \in \real^\pdim$ is the unknown regression vector, and $w \sim \mathcal{N}(0, σ^2 I)$ is additive Gaussian noise. This paper studies the minimax rates of convergence for estimation of $\betastar$ for $\ell_\rpar$-losses and in the $\ell_2$-prediction loss, assuming that $\betastar$ belongs to an $\ell_{\qpar}$-ball $\Ballq(\myrad)$ for some $\qpar \in [0,1]$. We show that under suitable regularit…
This paper concerns the estimation of the regression function at a given point in nonparametric heteroscedastic models with Gaussian noise or with noise having unknown distribution. In the two cases an asymptotically efficient kernel estimator is constructed for the minimax absolute error risk.
This work deals with the ill-posed inverse problem of reconstructing a function $f$ given implicitly as the solution of $g = Af$, where $A$ is a compact linear operator with unknown singular values and known eigenfunctions. We observe the function $g$ and the singular values of the operator subject to Gaussian white noise with respective noise levels $\varepsilon$ and $σ$. We develop a minimax theory in terms of both noise levels and propose an orthogonal series estimator attaining the minimax rates. This estimator requires the optimal choice of a dimension parameter depending on certain chara…
The paper considers model selection in regression under the additional structural constraints on admissible models where the number of potential predictors might be even larger than the available sample size. We develop a Bayesian formalism as a natural tool for generating a wide class of model selection criteria based on penalized least squares estimation with various complexity penalties associated with a prior on a model size. The resulting criteria are adaptive to structural constraints. We establish the upper bound for the quadratic risk of the resulting MAP estimator and the correspondin…
Minimax-optimal and Locally-adaptive Online Nonparametric Regression
本文针对对抗性在线非参数回归提出一种无参数算法，通过修剪链式树动态适应局部 Hölder 平滑性，首次实现计算高效且局部自适应的极小极大最优速率。
Under a standard assumption in complexity theory (NP not in P/poly), we demonstrate a gap between the minimax prediction risk for sparse linear regression that can be achieved by polynomial-time algorithms, and that achieved by optimal algorithms. In particular, when the design matrix is ill-conditioned, the minimax prediction loss achievable by polynomial-time algorithms can be substantially greater than that of an optimal algorithm. This result is the first known gap between polynomial and optimal algorithms for sparse linear regression, and does not depend on conjectures in average-case com…
Transfer Learning for Nonparametric Regression: Non-asymptotic Minimax Analysis and Adaptive Procedure
本文研究非参数回归迁移学习的非渐近极小极大风险，提出置信度阈值估计量与数据驱动自适应算法，揭示自动平滑和超加速现象，并通过模拟与真实案例验证其性能。
Minimax Semiparametric Learning With Approximate Sparsity
该研究针对高维下精确稀疏假设适用范围有限的问题，形式化近似稀疏性概念，推导其下回归斜率与平均导数的极小极大速率，提出达最优速率的估计器，还发现有序与无序非参数回归的最优性边界存在差异。
The problem of estimating the baseline signal from multisample noisy curves is investigated. We consider the functional mixed effects model, and we suppose that the functional fixed effect belongs to the Besov class. This framework allows us to model curves that can exhibit strong irregularities, such as peaks or jumps for instance. The lower bound for the $L_2$ minimax risk is provided, as well as the upper bound of the minimax rate, that is derived by constructing a wavelet estimator for the functional fixed effect. Our work constitutes the first theoretical functional results in multisample…
This paper characterizes the minimax linear estimator of the value of an unknown function at a boundary point of its domain in a Gaussian white noise model under the restriction that the first-order derivative of the unknown function is Lipschitz continuous (the second-order Hölder class). The result is then applied to construct the minimax optimal estimator for the regression discontinuity design model, where the parameter of interest involves function values at boundary points.
We investigate if kernel regularization methods can achieve minimax convergence rates over a source condition regularity assumption for the target function. These questions have been considered in past literature, but only under specific assumptions about the decay, typically polynomial, of the spectrum of the the kernel mapping covariance operator. In the perspective of distribution-free results, we investigate this issue under much weaker assumption on the eigenvalue decay, allowing for more complex behavior that can reflect different structure of the data at different scales.
In this work we study the semi-supervised framework of confidence set classification with controlled expected size in minimax settings. We obtain semi-supervised minimax rates of convergence under the margin assumption and a H{ö}lder condition on the regression function. Besides, we show that if no further assumptions are made, there is no supervised method that outperforms the semi-supervised estimator proposed in this work. We establish that the best achievable rate for any supervised method is n^{--1/2} , even if the margin assumption is extremely favorable. On the contrary, semi-supervised…
Phase Transition in Nonparametric Minimax Rates for Covariate Shifts on Approximate Manifolds
研究结构化协变量偏移下的非参数回归，提出局部多项式回归估计器与自适应流程，在目标分布近似流形的假设下达到最优极小化率，并揭示了由距离与维度等因素控制的相变现象。
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。