A. B. Németh, S. Z. Németh
The note describes the cones in the Euclidean space admitting isotonic metric projection with respect to the coordinate-wise ordering. As a consequence it is showed that the metric projection onto the regression cone (the cone defined by the general isotonic regression problem) admits a projection which is isotonic with respect to the coordinate-wise ordering.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
We consider the problem of isotonic regression, where the underlying signal $x$ is assumed to satisfy a monotonicity constraint, that is, $x$ lies in the cone $\{ x\in\mathbb{R}^n : x_1 \leq \dots \leq x_n\}$. We study the isotonic projection operator (projection to this cone), and find a necessary and sufficient condition characterizing all norms with respect to which this projection is contractive. This enables a simple and non-asymptotic analysis of the convergence properties of isotonic regression, yielding uniform confidence bands that adapt to the local Lipschitz properties of the signal…
We study the isotonic regression estimator over a general countable pre-ordered set. We obtain the limiting distribution of the estimator and study its properties. It is proved that, under some general assumptions, the limiting distribution of the isotonized estimator is given by the concatenation of the separate isotonic regressions of the certain subvectors of an unrestrecred estimator's asymptotic distribution. Also, we show that the isotonization preserves the rate of convergence of the underlying estimator. We apply these results to the problems of estimation of a bimonotone regression fu…
We study the bias of the isotonic regression estimator. While there is extensive work characterizing the mean squared error of the isotonic regression estimator, relatively little is known about the bias. In this paper, we provide a sharp characterization, proving that the bias scales as $O(n^{-β/3})$ up to log factors, where $1 \leq β\leq 2$ is the exponent corresponding to H{ö}lder smoothness of the underlying mean. Importantly, this result only requires a strictly monotone mean and that the noise distribution has subexponential tails, without relying on symmetric noise or other restrictive …
We study the non-parametric isotonic regression problem for bivariate elicitable functionals that are given as an elicitable univariate functional and its Bayes risk. Prominent examples for functionals of this type are (mean, variance) and (Value-at-Risk, Expected Shortfall), where the latter pair consists of important risk measures in finance. We present our results for totally ordered covariates but extenstions to partial orders are given in the appendix.
This article introduces a new nonparametric method for estimating a univariate regression function of bounded variation. The method exploits the Jordan decomposition which states that a function of bounded variation can be decomposed as the sum of a non-decreasing function and a non-increasing function. This suggests combining the backfitting algorithm for estimating additive functions with isotonic regression for estimating monotone functions. The resulting iterative algorithm is called Iterative Isotonic Regression (I.I.R.). The main technical result in this paper is the consistency of the p…
We consider the problem of estimating an unknown coordinate-wise monotone function given noisy measurements, known as the isotonic regression problem. Often, only a small subset of the features affects the output. This motivates the sparse isotonic regression setting, which we consider here. We provide an upper bound on the expected VC entropy of the space of sparse coordinate-wise monotone functions, and identify the regime of statistical consistency of our estimator. We also propose a linear program to recover the active coordinates, and provide theoretical recovery guarantees. We close with…
In this paper we propose a family of robust estimates for isotonic regression: isotonic M-estimators. We show that their asymptotic distribution is, up to an scalar factor, the same as that of Brunk's classical isotonic estimator. We also derive the influence function and the breakdown point of these estimates. Finally we perform a Monte Carlo study that shows that the proposed family includes estimators that are simultaneously highly efficient under gaussian errors and highly robust when the error distribution has heavy tails.
It is well known that the isotonic least squares estimator is characterized as the derivative of the greatest convex minorant of a random walk. Provided the walk has exchangeable increments, we prove that the slopes of the greatest convex minorant are distributed as order statistics of the running averages. This result implies an exact non-asymptotic formula for the squared error risk of least squares in isotonic regression when the true sequence is constant that holds for every exchangeable error distribution.
We propose a strategy for computing the isotonic least-squares estimate of a monotone function in a general regression setting where the data are distributed across different servers and the observations across servers, though independent, can come from heterogeneous sub-populations, thereby violating the identically distributed assumption. Our strategy fixes the super-efficiency phenomenon observed in prior work on distributed computing in the isotonic regression framework, where averaging several isotonic estimates (each computed at a local server) on a central server produces super-efficien…
In general, the solution to a regression problem is the minimizer of a given loss criterion, and depends on the specified loss function. The nonparametric isotonic regression problem is special, in that optimal solutions can be found by solely specifying a functional. These solutions will then be minimizers under all loss functions simultaneously as long as the loss functions have the requested functional as the Bayes act. For the functional, the only requirement is that it can be defined via an identification function, with examples including the expectation, quantile, and expectile functiona…
Given a directed acyclic graph $G,$ and a set of values $y$ on the vertices, the Isotonic Regression of $y$ is a vector $x$ that respects the partial order described by $G,$ and minimizes $||x-y||,$ for a specified norm. This paper gives improved algorithms for computing the Isotonic Regression for all weighted $\ell_{p}$-norms with rigorous performance guarantees. Our algorithms are quite practical, and their variants can be implemented to run fast in practice.
We introduce a new test statistic for testing the null hypothesis that the sampling distribution has an increasing hazard rate on a specified interval [0,a]. It is based on a comparison of the empirical distribution function with an isotonic estimate, using the restriction that the hazard is increasing, and measures the excursions of the empirical distribution above the isotonic estimate, due to local non-monotonicity. It is proved in the companion paper Groeneboom and Jongbloed (2011a) that the test statistic is asymptotically normal if the hazard is strictly increasing on the interval [0,a] …
This paper studies the estimation and inference for the isotonic regression at the boundary point, an object that is particularly interesting and required in the analysis of monotone regression discontinuity designs. We show that the isotonic regression is inconsistent in this setting and derive the asymptotic distributions of boundary corrected estimators. Interestingly, the boundary corrected estimators can be bootstrapped without subsampling or additional nonparametric smoothing which is not the case for the interior point. The Monte Carlo experiments indicate that shape restrictions can im…
We consider the problem of estimating an unknown $θ\in {\mathbb{R}}^n$ from noisy observations under the constraint that $θ$ belongs to certain convex polyhedral cones in ${\mathbb{R}}^n$. Under this setting, we prove bounds for the risk of the least squares estimator (LSE). The obtained risk bound behaves differently depending on the true sequence $θ$ which highlights the adaptive behavior of $θ$. As special cases of our general result, we derive risk bounds for the LSE in univariate isotonic and convex regression. We study the risk bound in isotonic regression in greater detail: we show that…
Isotonic distributional regression (IDR) is a powerful nonparametric technique for the estimation of conditional distributions under order restrictions. In a nutshell, IDR learns conditional distributions that are calibrated, and simultaneously optimal relative to comprehensive classes of relevant loss functions, subject to isotonicity constraints in terms of a partial order on the covariate space. Nonparametric isotonic quantile regression and nonparametric isotonic binary regression emerge as special cases. For prediction, we propose an interpolation method that generalizes extant specificat…
We consider a general monotone regression estimation where we allow for independent and dependent regressors. We propose a modification of the classical isotonic least squares estimator and establish its rate of convergence for the integrated $L_1$-loss function. The methodology captures the shape of the data without assuming additivity or a parametric form for the regression function. Furthermore, the degree of smoothing is chosen automatically and no auxiliary tuning is required for the theoretical analysis. Some simulations and two real data illustrations complement the study of the propose…
In the context of estimating stochastically ordered distribution functions, the pool-adjacent-violators algorithm (PAVA) can be modified such that the computation times are reduced substantially. This is achieved by studying the dependence of antitonic weighted least squares fits on the response vector to be approximated.
Classical univariate isotonic regression involves nonparametric estimation under a monotonicity constraint of the true signal. We consider a variation of this generating process, which we term adversarial sign-corrupted isotonic (\texttt{ASCI}) regression. Under this \texttt{ASCI} setting, the adversary has full access to the true isotonic responses, and is free to sign-corrupt them. Estimating the true monotonic signal given these sign-corrupted responses is a highly challenging task. Notably, the sign-corruptions are designed to violate monotonicity, and possibly induce heavy dependence betw…
We consider component-wise estimation of order restricted location/scale parameters $θ_1$ and $θ_2$ ($θ_1\leq θ_2$) of a general bivariate distribution under the squared error loss function. To find improvements over the best location/scale equivariant estimators (BLEE/BSEE) of $θ_1$ and $θ_2$, we study isotonic regression of suitable location/scale equivariant estimators (LEE/SEE) of $θ_1$ and $θ_2$ with general weights. Let $\mathcal{D}_{1,ν}$ and $\mathcal{D}_{2,β}$ denote suitable classes of isotonic regression estimators of $θ_1$ and $θ_2$, respectively. Under the squared error loss funct…
We study the problem of estimating piecewise monotone vectors. This problem can be seen as a generalization of the isotonic regression that allows a small number of order-violating changepoints. We focus mainly on the performance of the nearly-isotonic regression proposed by Tibshirani et al. (2011). We derive risk bounds for the nearly-isotonic regression estimators that are adaptive to piecewise monotone signals. The estimator achieves a near minimax convergence rate over certain classes of piecewise monotone signals under a weak assumption. Furthermore, we present an algorithm that can be a…
The problem of simultaneous estimation of location/scale parameters $θ_1$ and $θ_2$ of a general bivariate location/scale model, when the ordering between the parameters is known apriori (say, $θ_1\leq θ_2$), has been considered. We consider isotonic regression estimators based on the best location/scale equivariant estimators (BLEEs/BSEEs) of $θ_1$ and $θ_2$ with general weight functions. Let $\mathcal{D}$ denote the corresponding class of isotonic regression estimators of $(θ_1,θ_2)$. Under the sum of the weighted squared error loss function, we characterize admissible estimators within the …
A Chernoff-type distribution is a nonnormal distribution defined by the slope at zero of the greatest convex minorant of a two-sided Brownian motion with a polynomial drift. While a Chernoff-type distribution is known to appear as the distributional limit in many non-regular statistical estimation problems, the accuracy of Chernoff-type approximations has remained largely unknown. In the present paper, we tackle this problem and derive Berry-Esseen bounds for Chernoff-type limit distributions in the canonical non-regular statistical estimation problem of isotonic (or monotone) regression. The …
Isotonic regression is a standard problem in shape-constrained estimation where the goal is to estimate an unknown nondecreasing regression function $f$ from independent pairs $(x_i, y_i)$ where $\mathbb{E}[y_i]=f(x_i), i=1, \ldots n$. While this problem is well understood both statistically and computationally, much less is known about its uncoupled counterpart where one is given only the unordered sets $\{x_1, \ldots, x_n\}$ and $\{y_1, \ldots, y_n\}$. In this work, we leverage tools from optimal transport theory to derive minimax rates under weak moments conditions on $y_i$ and to give an e…
In this paper, we prove maximal inequalities and study the functional central limit theorem for the partial sums of linear processes generated by dependent innovations. Due to the general weights, these processes can exhibit long-range dependence and the limiting distribution is a fractional Brownian motion. The proofs are based on new approximations by a linear process with martingale difference innovations. The results are then applied to study an estimator of the isotonic regression when the error process is a (possibly long-range dependent) time series.
Insurance pricing systems should fulfill the auto-calibration property to ensure that there is no systematic cross-financing between different price cohorts. Often, regression models are not auto-calibrated. We propose to apply isotonic recalibration to a given regression model to ensure auto-calibration. Our main result proves that under a low signal-to-noise ratio, this isotonic recalibration step leads to explainable pricing systems because the resulting isotonically recalibrated regression functions have a low complexity.
Isotonic subgroup selection
针对识别回归函数超阈值子集的子群选择问题，提出基于鞅检验与多重检验的多变量保序回归方法，满足非渐近一致第一类错误率保证且功效达极小极大最优速率，并在R包ISS中实现。
Deconvolution is a statistical inverse problem to estimate the distribution of a random variable based on its noisy observations. Despite the extensive studies on the topic, deconvolution with unknown noise distribution remains as a notoriously hard problem. We propose a matrix-based viewpoint for collective deconvolution that subsumes the setup with repeated measurements as a special case. As the main result, we describe a simple algorithm that partially utilizes matrix structure to solve deconvolution problem and provide non-asymptotic error analysis for the algorithm. We show that the propo…
Best $L_p$ Isotonic Regressions, $p \in \{0, 1, \infty\}$
本文研究 $p \in \{0,1,\infty\}$ 时最佳 $L_p$ 保序回归的唯一性与等价性，比较严格回归和 lex 回归，并通过扩展到 $p<0$ 证明 $p \nearrow 0$ 与 lex 回归相同，同时给出计算算法。
From Isotonic to Lipschitz Regression: A New Interpolative Perspective on Shape-restricted Estimation
本文提出了一种新的插值视角，将非参数光滑性和形状限制估计连接起来，通过将利普希茨函数分解为单调函数和线性函数，提出了一种新的估计方法，具有良好的方法论、理论和计算性质，并在异方差和厚尾误差下提供了收敛保证。
We study the least squares regression function estimator over the class of real-valued functions on $[0,1]^d$ that are increasing in each coordinate. For uniformly bounded signals and with a fixed, cubic lattice design, we establish that the estimator achieves the minimax rate of order $n^{-\min\{2/(d+2),1/d\}}$ in the empirical $L_2$ loss, up to poly-logarithmic factors. Further, we prove a sharp oracle inequality, which reveals in particular that when the true regression function is piecewise constant on $k$ hyperrectangles, the least squares estimator enjoys a faster, adaptive rate of conve…
Generalized Asymptotic Limit Theory and Inference for Isotonic Regression
研究保序回归问题，引入真实函数局部行为的一般条件，揭示更丰富渐近分布族。因估计干扰参数，经典框架推断有挑战。利用新极限分布对称性，使HulC程序产生有效置信区间，框架让推断在特定函数类上一致有效。
In this paper, we consider the nonparametric regression problem with multivariate predictors. We provide a characterization of the degrees of freedom and divergence for estimators of the unknown regression function, which are obtained as outputs of linearly constrained quadratic optimization procedures, namely, minimizers of the least squares criterion with linear constraints and/or quadratic penalties. As special cases of our results, we derive explicit expressions for the degrees of freedom in many nonparametric regression problems, e.g., bounded isotonic regression, multivariate (penalized)…
We consider the nonparametric multivariate isotonic regression problem, where the regression function is assumed to be nondecreasing with respect to each predictor. Our goal is to construct a Bayesian credible interval for the function value at a given interior point with assured limiting frequentist coverage. We put a prior on unrestricted step-functions, but make inference using the induced posterior measure by an "immersion map" from the space of unrestricted functions to that of multivariate monotone functions. This allows maintaining the natural conjugacy for posterior sampling. A natural…
We study limit distributions for the tuning-free max-min block estimator originally proposed in [FLN17] in the problem of multiple isotonic regression, under both fixed lattice design and random design settings. We show that, if the regression function $f_0$ admits vanishing derivatives up to order $α_k$ along the $k$-th dimension ($k=1,\ldots,d$) at a fixed point $x_0 \in (0,1)^d$, and the errors have variance $σ^2$, then the max-min block estimator $\hat{f}_n$ satisfies \begin{align*} (n_\ast/σ^2)^{\frac{1}{2+\sum_{k \in \mathcal{D}_\ast} α_k^{-1}}}\big(\hat{f}_n(x_0)-f_0(x_0)\big)\rightsqui…
Survival Isotonic Distributional Regression
本研究提出S-IDR，将IDR扩展至右删失场景，无调参且适配多类型协变量，在基准套件及肝移植MELD评分验证案例中表现良好，配套多语言包可用。
In this paper we study minimax and adaptation rates in general isotonic regression. For uniform deterministic and random designs in $[0,1]^d$ with $d\ge 2$ and $N(0,1)$ noise, the minimax rate for the $\ell_2$ risk is known to be bounded from below by $n^{-1/d}$ when the unknown mean function $f$ is nondecreasing and its range is bounded by a constant, while the least squares estimator (LSE) is known to nearly achieve the minimax rate up to a factor $(\log n)^γ$ where $n$ is sample size, $γ= 4$ in the lattice design and $γ= \max\{9/2, (d^2+d+1)/2 \}$ in the random design. Moreover, the LSE is …
We consider the problem of constructing pointwise confidence intervals in the multiple isotonic regression model. Recently, [HZ19] obtained a pointwise limit distribution theory for the so-called block max-min and min-max estimators [FLN17] in this model, but inference remains a difficult problem due to the nuisance parameter in the limit distribution that involves multiple unknown partial derivatives of the true regression function. In this paper, we show that this difficult nuisance parameter can be effectively eliminated by taking advantage of information beyond point estimates in the block…
We consider the nonparametric regression problem with multiple predictors and an additive error, where the regression function is assumed to be coordinatewise nondecreasing. We propose a Bayesian approach to make an inference on the multivariate monotone regression function, obtain the posterior contraction rate, and construct a universally consistent Bayesian testing procedure for multivariate monotonicity. To facilitate posterior analysis, we set aside the shape restrictions temporarily, and endow a prior on blockwise constant regression functions with heights independently normally distribu…
We study the statistical properties of the least squares estimator in unimodal sequence estimation. Although closely related to isotonic regression, unimodal regression has not been as extensively studied. We show that the unimodal least squares estimator is adaptive in the sense that the risk scales as a function of the number of values in the true underlying sequence. Such adaptivity properties have been shown for isotonic regression by Chatterjee et al(2015) and Bellec(2015). A technical complication in unimodal regression is the non-convexity of the underlying parameter space. We develop a…
We consider the problem of nonparametric regression when the covariate is $d$-dimensional, where $d \geq 1$. In this paper we introduce and study two nonparametric least squares estimators (LSEs) in this setting---the entirely monotonic LSE and the constrained Hardy-Krause variation LSE. We show that these two LSEs are natural generalizations of univariate isotonic regression and univariate total variation denoising, respectively, to multiple dimensions. We discuss the characterization and computation of these two LSEs obtained from $n$ data points. We provide a detailed study of their risk pr…
Motivated by models for multiway comparison data, we consider the problem of estimating a coordinate-wise isotonic function on the domain $[0, 1]^d$ from noisy observations collected on a uniform lattice, but where the design points have been permuted along each dimension. While the univariate and bivariate versions of this problem have received significant attention, our focus is on the multivariate case $d \geq 3$. We study both the minimax risk of estimation (in empirical $L_2$ loss) and the fundamental limits of adaptation (quantified by the adaptivity index) to a family of piecewise const…
We consider the problem of nonparametric regression under shape constraints. The main examples include isotonic regression (with respect to any partial order), unimodal/convex regression, additive shape-restricted regression, and constrained single index model. We review some of the theoretical properties of the least squares estimator (LSE) in these problems, emphasizing on the adaptive nature of the LSE. In particular, we study the behavior of the risk of the LSE, and its pointwise limiting distribution theory, with special emphasis to isotonic regression. We survey various methods for const…
In many problems, a sensible estimator of a possibly multivariate monotone function may itself fail to be monotone. We study the correction of such an estimator obtained via projection onto the space of functions monotone over a finite grid in the domain. We demonstrate that this corrected estimator has no worse supremal estimation error than the initial estimator, and that analogously corrected confidence bands contain the true function whenever the initial bands do, at no loss to average or maximal band width. Additionally, we demonstrate that the corrected estimator is uniformly asymptotica…
We propose two estimators of a monotone spectral density, that are based on the periodogram. These are the isotonic regression of the periodogram and the isotonic regression of the log-periodogram. We derive pointwise limit distribution results for the proposed estimators for short memory linear processes and long memory Gaussian processes and also that the estimators are rate optimal.
Limit distributions for the greatest convex minorant and its derivative are considered for a general class of stochastic processes including partial sum processes and empirical processes, for independent, weakly dependent and long range dependent data. The results are applied to isotonic regression, isotonic regression after kernel smoothing, estimation of convex regression functions, and estimation of monotone and convex density functions. Various pointwise limit distributions are obtained, and the rate of convergence depends on the self similarity properties and on the rate of convergence of…
We study the estimation of flows on trees, a structured generalization of isotonic regression. A tree flow is defined recursively as a positive flow value into a node that is partitioned into an outgoing flow to the children nodes, with some amount of the flow possibly leaking outside. We study the behavior of the least squares estimator for flows, and the associated minimax lower bounds. We characterize the risk of the least squares estimator in two regimes. In the first regime the diameter of the tree grows at most logarithmically with the number of nodes. In the second regime, the tree cont…
Shape restricted regressions, including isotonic regression and concave regression as special cases, are studied using priors on Bernstein polynomials and Markov chain Monte Carlo methods. These priors have large supports, select only smooth functions, can easily incorporate geometric information into the prior, and can be generated without computational difficulty. Algorithms generating priors and posteriors are proposed, and simulation studies are conducted to illustrate the performance of this approach. Comparisons with the density-regression method of Dette et al. (2006) are included.
Fitting linear regression models can be computationally very expensive in large-scale data analysis tasks if the sample size and the number of variables are very large. Random projections are extensively used as a dimension reduction tool in machine learning and statistics. We discuss the applications of random projections in linear regression problems, developed to decrease computational costs, and give an overview of the theoretical guarantees of the generalization error. It can be shown that the combination of random projections with least squares regression leads to similar recovery as rid…
In this paper we consider the stacking of isotonic regression and the method of rearrangement with the empirical estimator to estimate a discrete distribution with an infinite support. The estimators are proved to be strongly consistent with $\sqrt{n}$-rate of convergence. We obtain the asymptotic distributions of the estimators and construct the asymptotically correct conservative global confidence bands. We show that stacked Grenander estimator outperforms the stacked rearrangement estimator. The new estimators behave well even for small sized data sets and provide a trade-off between goodne…
On a Projection Estimator of the Regression Function Derivative
本文研究单变量回归模型中回归函数导数的估计，提出两种估计构造方式，推导不同假设下的风险界与最优收敛速率，并给出模型选择程序及对应风险界。
Principal Components Regression (PCR) is a traditional tool for dimension reduction in linear regression that has been both criticized and defended. One concern about PCR is that obtaining the leading principal components tends to be computationally demanding for large data sets. While random projections do not possess the optimality properties of the leading principal subspace, they are computationally appealing and hence have become increasingly popular in recent years. In this paper, we present an analysis showing that for random projections satisfying a Johnson-Lindenstrauss embedding prop…
Depth notions in regression have been systematically proposed and examined in Zuo (2018). One of the prominent advantages of notion of depth is that it can be directly utilized to introduce median-type deepest estimating functionals (or estimators in empirical distribution case) for location or regression parameters in a multi-dimensional setting. Regression depth shares the advantage. Depth induced deepest estimating functionals are expected to inherit desirable and inherent robustness properties ( e.g. bounded maximum bias and influence function and high breakdown point) as their univariate …
This paper deals with sparse feature selection and grouping for classification and regression. The classification or regression problems under consideration consists in minimizing a convex empirical risk function subject to an $\ell^1$ constraint, a pairwise $\ell^\infty$ constraint, or a pairwise $\ell^1$ constraint. Existing work, such as the Lasso formulation, has focused mainly on Lagrangian penalty approximations, which often require ad hoc or computationally expensive procedures to determine the penalization parameter. We depart from this approach and address the constrained problem dire…
Valid Bayesian Inference based on Variance Weighted Projection for High-Dimensional Logistic Regression with Binary Covariates
针对高维基线协变量下二元结果相关分类处理效应的推断挑战，提出基于方差加权投影的正交得分及新型贝叶斯框架，提供一致收敛结果并通过模拟与真实数据分析验证方法有效性。
The Minimax Rate of Denoising Flows on Trees
本文完整解决了有根树上从高斯噪声恢复流的去噪问题，确定了最小最大风险为 $\min\{V^2H_T,\sigma^2k_0\}$，并提出了达到该速率的确定性指数加权聚合估计器，同时揭示了最小二乘法的次优性。
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。