Zuo (2019) (Z19) addressed the computation of the projection regression depth (PRD) and its induced median (the maximum depth estimator). Z19 achieved the exact computation of PRD via a modified version of regular univariate sample median, which resulted in the loss of invariance of PRD and the equivariance of depth induced median. This article achieves the exact computation without scarifying the invariance of PRD and the equivariance of the regression median. Z19 also addressed the approximate computation of PRD induced median, the naive algorithm in Z19 is very slow. This article modifies the approximation in Z19 and adopts Rcpp package and consequently obtains a much (could be $100$ times) faster algorithm with an even better level of accuracy meanwhile. Furthermore, as the third major contribution, this article introduces three new depth induced estimators which can run $300$ times faster than that of Z19 meanwhile maintaining the same (or a bit better) level of accuracy. Real as well as simulated data examples are presented to illustrate the difference between the algorithms of Z19 and the ones proposed in this article. Findings support the statements above and manifest the major contributions of the article.
The median absolute deviation is a widely used robust measure of statistical dispersion. Using a scale constant, we can use it as an asymptotically consistent estimator for the standard deviation under normality. For finite samples, the scale constant should be corrected in order to obtain an unbiased estimator. The bias-correction factor depends on the sample size and the median estimator. When we use the traditional sample median, the factor values are well known, but this approach does not provide optimal statistical efficiency. In this paper, we present the bias-correction factors for the …
In a linear instrumental variables (IV) setting for estimating the causal effects of multiple confounded exposure/treatment variables on an outcome, we investigate the adaptive Lasso method for selecting valid instrumental variables from a set of available instruments that may contain invalid ones. An instrument is invalid if it fails the exclusion conditions and enters the model as an explanatory variable. We extend the results as developed in Windmeijer et al. (2019) for the single exposure model to the multiple exposures case. In particular we propose a median-of-medians estimator and show …
This paper presents a novel estimation approach for cumulative link models, based on median bias reduction as developed in Kenne Pagui et al. (2017). The median bias reduced estimator is obtained as solution of an estimating equation based on an adjustment of the score. It allows to obtain higher-order median centering of maximum likelihood estimates without requiring their finiteness. Moreover, the estimator is equivariant under componentwise monotone reparameterizations and the method is effective in preventing boundary estimates. We evaluate the properties of the median bias reduced estimat…
This article suggests an efficient class of estimators of population median of the study variable using an auxiliary variable. Asymptotic expressions of bias and mean square error of the proposed class of estimators have been obtained. Asymptotic optimum estimator has been investigated along with its approximate mean square error. We have shown that proposed class of estimator is more efficient than estimator considered by Srivastava (1967), Gross (1980), Kuk and Mak (1989) Singh et al. (2003b), Al and Chingi (2009) and Singh and Solanki (2013). In addition theoretical findings are supported b…
The sample $d_θ$-median is a robust estimator of the central tendency or location of an interval-valued random variable. While the interval-valued sample mean can be highly influenced by outliers, this spatial-type interval-valued median remains much more reliable. In this paper, we show that under general conditions the sample $d_θ$-median is a strongly consistent estimator of the $d_θ$-median of an interval-valued random variable.
We introduce a new notion of regularity of an estimator called median regularity. We prove that uniformly valid (honest) inference for a functional is possible if and only if there exists a median regular estimator of that functional. To our knowledge, such a notion of regularity that is necessary for uniformly valid inference is unavailable in the literature.
Shanghai Jiao Tong University（上海交通大学）；New York University（纽约大学）
This paper develops an efficient distributed inference algorithm, which is robust against a moderate fraction of Byzantine nodes, namely arbitrary and possibly adversarial machines in a distributed learning system. In robust statistics, the median-of-means (MOM) has been a popular approach to hedge against Byzantine failures due to its ease of implementation and computational efficiency. However, the MOM estimator has the shortcoming in terms of statistical efficiency. The first main contribution of the paper is to propose a variance reduced median-of-means (VRMOM) estimator, which improves th…
In numerous regular statistical models, median bias reduction (Kenne Pagui et al., 2017) has proven to be a noteworthy improvement over maximum likelihood, alternative to mean bias reduction. The estimator is obtained as solution to a modified score equation ensuring smaller asymptotic median bias than the maximum likelihood estimator. This paper provides a simplified algebraic form of the adjustment term for general regular models. With the new formula, the estimation procedure benefits from a considerable computational gain by avoiding multiple summations and thus allows an efficient impleme…
In this paper, a nonparametric estimator is proposed for estimating the L1-median for multivariate conditional distribution when the covariates take values in an infinite dimensional space. The multivariate case is more appropriate to predict the components of a vector of random variables simultaneously rather than predicting each of them separately. While estimating the conditional L1-median function using the well-known Nadarya-Waston estimator, we establish the strong consistency of this estimator as well as the asymptotic normality. We also present some simulations and provide how to built…
Hurst exponent is an important feature summarizing the noisy high-frequency data when the inherent scaling pattern cannot be described by standard statistical models. In this paper, we study the robust estimation of Hurst exponent based on non-decimated wavelet transforms (NDWT). The robustness is achieved by applying a general trimean estimator on non-decimated wavelet coefficients of the transformed data. The general trimean estimator is derived as a weighted average of the distribution's median and quantiles, combining the median's emphasis on central values with the quantiles' attention to…
Mean profiles are widely used as indicators of the electricity consumption habits of customers. Currently, in Électricité De France (EDF), class load profiles are estimated using point-wise mean function. Unfortunately, it is well known that the mean is highly sensitive to the presence of outliers, such as one or more consumers with unusually high-levels of consumption. In this paper, we propose an alternative to the mean profile: the $L_1$-median profile which is more robust. When dealing with large datasets of functional data (load curves for example), survey sampling approaches are useful f…
The goal of nonparametric regression is to recover an underlying regression function from noisy observations, under the assumption that the regression function belongs to a pre-specified infinite dimensional function space. In the online setting, when the observations come in a stream, it is generally computationally infeasible to refit the whole model repeatedly. There are as of yet no methods that are both computationally efficient and statistically rate-optimal. In this paper, we propose an estimator for online nonparametric regression. Notably, our estimator is an empirical risk minimizer …
We consider the problem of estimating the regression function in functional linear regression models by proposing a new type of projection estimators which combine dimension reduction and thresholding. The introduction of a threshold rule allows to get consistency under broad assumptions as well as minimax rates of convergence under additional regularity hypotheses. We also consider the particular case of Sobolev spaces generated by the trigonometric basis which permits to get easily mean squared error of prediction as well as estimators of the derivatives of the regression function. We prove …
Shape-constrained inference has wide applicability in bioassay, medicine, economics, risk assessment, and many other fields. Although there has been a large amount of work on monotone-constrained univariate curve estimation, multivariate shape-constrained problems are much more challenging, and fewer advances have been made in this direction. With a focus on monotone regression with multiple predictors, this current work proposes a projection approach to estimate a multiple monotone regression function. An initial unconstrained estimator -- such as a local polynomial estimator or spline estima…
High-frequency measurements and images acquired from various sources in the real world often possess a degree of self-similarity and inherent regular scaling. When data look like a noise, the scaling exponent may be the only informative feature that summarizes such data. Methods for the assessment of self-similarity by estimating Hurst exponent often involve analysis of rate of decay in a spectrum defined in various multiresolution domains. When this spectrum is calculated using discrete non-decimated wavelet transforms, due to increased autocorrelation in wavelet coefficients, the estimators …
Convex Clustering Redefined: Robust Learning with the Median of Means Estimator
Indian Statistical Institute, Kolkata（印度统计研究院，科希马）；Department of Statistics and Data Science, University of Texas at Austin（德克萨斯大学奥斯汀分校统计与数据科学系）；Electronics and Communication Sciences, Indian Statistical Institute, Kolkata（印度统计研究院，科希马电子与通信科学）；Department of Statistics, University of Oxford（牛津大学统计系）；Department of Statistics, University of Michigan（密歇根大学统计系）
We propose a robust and scalable procedure for general optimization and inference problems on manifolds leveraging the classical idea of `median-of-means' estimation. This is motivated by ubiquitous examples and applications in modern data science in which a statistical learning problem can be cast as an optimization problem over manifolds. Being able to incorporate the underlying geometry for inference while addressing the need for robustness and scalability presents great challenges. We address these challenges by first proving a key lemma that characterizes some crucial properties of geomet…
For $J \ge 2$ independent groups, the paper deals with testing the global hypothesis that all $J$ groups have a common population median or identical quantiles, with an emphasis on the quartiles. Classic rank-based methods are sometimes suggested for comparing medians, but it is well known that under general conditions they do not adequately address this goal. Extant methods based on the usual sample median are unsatisfactory when there are tied values except for the special case $J=2$. A variation of the percentile bootstrap used in conjunction with the Harrell--Davis quantile estimator perfo…
Wigner Research Centre for Physics（维格纳物理研究中心）；Semmelweis University（塞梅尔维斯大学）；National Institute of Clinical Neurosciences（国家临床神经科学研究所）；Péter Pázmány Catholic University（帕兹马尼·彼得天主教大学）；Budapest University of Technology and Economics（布达佩斯技术与经济大学）；University of Pannonia（潘诺尼亚大学）
Data dimensionality informs us about data complexity and sets limit on the structure of successful signal processing pipelines. In this work we revisit and improve the manifold-adaptive Farahmand-Szepesvári-Audibert (FSA) dimension estimator, making it one of the best nearest neighbor-based dimension estimators available. We compute the probability density function of local FSA estimates, if the local manifold density is uniform. Based on the probability density function, we propose to use the median of local estimates as a basic global measure of intrinsic dimensionality, and we demonstrate t…
The era of big data is coming, and evidence-based medicine is attracting increasing attention to improve decision making in medical practice via integrating evidence from well designed and conducted clinical research. Meta-analysis is a statistical technique widely used in evidence-based medicine for analytically combining the findings from independent clinical trials to provide an overall estimation of a treatment effectiveness. The sample mean and standard deviation are two commonly used statistics in meta-analysis but some trials use the median, the minimum and maximum values, or sometimes …
Shanghai Jiao Tong University（上海交通大学）；Iowa State University（爱荷华州立大学）
Decentralized sparsity learning has attracted a significant amount of attention recently due to its rapidly growing applications. To obtain the robust and sparse estimators, a natural idea is to adopt the non-smooth median loss combined with a $\ell_1$ sparsity regularizer. However, most of the existing methods suffer from slow convergence performance caused by the {\em double} non-smooth objective. To accelerate the computation, in this paper, we proposed a decentralized surrogate median regression (deSMR) method for efficiently solving the decentralized sparsity learning problem. We show tha…
In this paper we study nonparametric estimators of copulas and copula densities. We first focus our study on a density copula estimator based on a polynomial orthogonal projection of the joint density. A new copula estimator is then deduced. Its asymptotic properties are studied: we provide a large functional class for which this construction is optimal in the minimax and maxiset sense and we propose a method selection for the smoothing parameter. An intensive simulation study shows the very good performance of both copulas and copula densities estimators which we compare to a large panel of c…
Current estimation methods for the probability of causation (PC) make strong parametric assumptions or are inefficient. We derive a nonparametric influence-function-based estimator for a projection of PC, which allows for simple interpretation and valid inference by making weak structural assumptions. We apply our estimator to real data from an experiment in Kenya, which found, by estimating the average treatment effect, that protecting water springs reduces childhood disease. However, before scaling up this intervention, it is important to determine whether it was the exposure, and not someth…
The method of 1-bit ("sign-sign") random projections has been a popular tool for efficient search and machine learning on large datasets. Given two $D$-dim data vectors $u$, $v\in\mathbb{R}^D$, one can generate $x = \sum_{i=1}^D u_i r_i$, and $y = \sum_{i=1}^D v_i r_i$, where $r_i\sim N(0,1)$ iid. The "collision probability" is ${Pr}\left(sgn(x)=sgn(y)\right) = 1-\frac{\cos^{-1}ρ}π$, where $ρ= ρ(u,v)$ is the cosine similarity. We develop "sign-full" random projections by estimating $ρ$ from (e.g.,) the expectation $E(sgn(x)y)=\sqrt{\frac{2}π} ρ$, which can be further substantially improved by …
With continuous outcomes, the average causal effect is typically defined using a contrast of expected potential outcomes. However, in the presence of skewed outcome data, the expectation may no longer be meaningful. In practice the typical approach is to either "ignore or transform" - ignore the skewness altogether or transform the outcome to obtain a more symmetric distribution, although neither approach is entirely satisfactory. Alternatively the causal effect can be redefined as a contrast of median potential outcomes, yet discussion of confounding-adjustment methods to estimate this parame…
In this article, we study large-dimensional matrix factor models and estimate the factor loading matrices and factor score matrix by minimizing square loss function. Interestingly, the resultant estimators coincide with the Projected Estimators (PE) in Yu et al.(2022), which was proposed from the perspective of simultaneous reduction of the dimensionality and the magnitudes of the idiosyncratic error matrix. In other word, we provide a least-square interpretation of the PE for matrix factor model, which parallels to the least-square interpretation of the PCA for the vector factor model. We der…
We consider the non-parametric regression problem under Huber's $ε$-contamination model, in which an $ε$ fraction of observations are subject to arbitrary adversarial noise. We first show that a simple local binning median step can effectively remove the adversary noise and this median estimator is minimax optimal up to absolute constants over the Hölder function class with smoothness parameters smaller than or equal to 1. Furthermore, when the underlying function has higher smoothness, we show that using local binning median as pre-preprocessing step to remove the adversarial noise, then we c…
For massive data sets, efficient computation commonly relies on distributed algorithms that store and process subsets of the data on different machines, minimizing communication costs. Our focus is on regression and classification problems involving many features. A variety of distributed algorithms have been proposed in this context, but challenges arise in defining an algorithm with low communication, theoretical guarantees and excellent practical performance in general settings. We propose a MEdian Selection Subset AGgregation Estimator (message) algorithm, which attempts to solve these pro…
Random-effects models are frequently used to synthesise information from different studies in meta-analysis. While likelihood-based inference is attractive both in terms of limiting properties and of implementation, its application in random-effects meta-analysis may result in misleading conclusions, especially when the number of studies is small to moderate. The current paper shows how methodology that reduces the asymptotic bias of the maximum likelihood estimator of the variance component can also substantially improve inference about the mean effect size. The results are derived for the mo…
Centrality descriptors are widely used to rank nodes according to specific concept(s) of importance. Despite the large number of centrality measures available nowadays, it is still poorly understood how to identify the node which can be considered as the `centre' of a complex network. In fact, this problem corresponds to finding the median of a complex network. The median is a non-parametric and robust estimator of the location parameter of a probability distribution. In this work, we present the most natural generalisation of the concept of median to the realm of complex networks, discussing …
We study the use of very sparse random projections for compressed sensing (sparse signal recovery) when the signal entries can be either positive or negative. In our setting, the entries of a Gaussian design matrix are randomly sparsified so that only a very small fraction of the entries are nonzero. Our proposed decoding algorithm is simple and efficient in that the major cost is one linear scan of the coordinates. We have developed two estimators: (i) the {\em tie estimator}, and (ii) the {\em absolute minimum estimator}. Using only the tie estimator, we are able to recover a $K$-sparse sign…
CLT and Edgeworth Expansion for m-out-of-n Bootstrap Estimators of The Studentized Median
Department of Industrial Engineering and Management Science Northwestern University（工业工程与管理科学系西北大学）；Department of Computer Science Northwestern University（计算机科学系西北大学）
A Plasmode Benchmark Can Refute but Cannot Certify: Projection and Regularity in Causal-Inference Simulation
School of Population and Public Health, University of British Columbia（不列颠哥伦比亚大学公共卫生与人口健康学院）；Centre for Advancing Health Outcomes, St. Paul’s Hospital（圣保罗医院健康成果促进中心）
We develop uniformly valid confidence regions for regression coefficients in a high-dimensional sparse median regression model with homoscedastic errors. Our methods are based on a moment equation that is immunized against non-regular estimation of the nuisance part of the median regression function by using Neyman's orthogonalization. We establish that the resulting instrumental median regression estimator of a target regression coefficient is asymptotically normally distributed uniformly with respect to the underlying sparse model and is semi-parametrically efficient. We also generalize our …
Network tomography has been regarded as one of the most promising methodologies for performance evaluation and diagnosis of the massive and decentralized Internet. This paper proposes a new estimation approach for solving a class of inverse problems in network tomography, based on marginal distributions of a sequence of one-dimensional linear projections of the observed data. We give a general identifiability result for the proposed method and study the design issue of these one dimensional projections in terms of statistical efficiency. We show that for a simple Gaussian tomography model, the…
