This paper introduces a new class of robust estimates for ARMA models. They are M-estimates, but the residuals are computed so the effect of one outlier is limited to the period where it occurs. These estimates are closely related to those based on a robust filter, but they have two important advantages: they are consistent and the asymptotic theory is tractable. We perform a Monte Carlo where we show that these estimates compare favorably with respect to standard M-estimates and to estimates based on a diagnostic procedure.
In this paper we present a robust estimator for the parameters of a continuous-time ARMA(p,q) (CARMA(p,q)) process sampled equidistantly which is not necessarily Gaussian. Therefore, an indirect estimation procedure is used. It is an indirect estimation because we first estimate the parameters of the auxiliary AR(r) representation ($r\geq 2p-1$) of the sampled CARMA process using a generalized M- (GM-)estimator. Since the map which maps the parameters of the auxiliary AR(r) representation to the parameters of the CARMA process is not given explicitly, a separate simulation part is necessary wh…
We introduce the ARMA (autoregressive-moving-average) point process, which is a Hawkes process driven by a Neyman-Scott process with Poisson immigration. It contains both the Hawkes and Neyman-Scott process as special cases and naturally combines self-exciting and shot-noise cluster mechanisms, useful in a variety of applications. The name ARMA is used because the ARMA point process is an appropriate analogue of the ARMA time series model for integer-valued series. As such, the ARMA point process framework accommodates a flexible family of models sharing methodological and mathematical similar…
In this paper we derive the asymptotic properties of the least squares estimator (LSE) of autoregressive moving-average (ARMA) models with regime changes under the assumption that the errors are uncorrelated but not necessarily independent. Relaxing the independence assumption considerably extends the range of application of the class of ARMA models with regime changes. Conditions are given for the consistency and asymptotic normality of the LSE. A particular attention is given to the estimation of the asymptotic covariance matrix, which may be very different from that obtained in the standard…
In this paper the stochastic complexity criterion is applied to estimation of the order in AR and ARMA models. The power of the criterion for short strings is illustrated by simulations. It requires an integral of the square root of Fisher information, which is done by Monte Carlo technique. The stochastic complexity, which is the negative logarithm of the Normalized Maximum Likelihood universal density function, is given. Also, exact asymptotic formulas for the Fisher information matrix are derived.
The problem of estimating ARMA models is computationally interesting due to the nonconcavity of the log-likelihood function. Recent results were based on the convex minimization. Joint model selection using penalization by a convex norm, e.g. the nuclear norm of a certain matrix related to the state space formulation was extensively studied from a computational viewpoint. The goal of the present short note is to present a theoretical study of a nuclear norm penalization based variant of the method of \cite{Bauer:Automatica05,Bauer:EconTh05} under the assumption of a Gaussian noise process.
A new likelihood based AR approximation is given for ARMA models. The usual algorithms for the computation of the likelihood of an ARMA model require $O(n)$ flops per function evaluation. Using our new approximation, an algorithm is developed which requires only $O(1)$ flops in repeated likelihood evaluations. In most cases, the new algorithm gives results identical to or very close to the exact maximum likelihood estimate (MLE). This algorithm is easily implemented in high level Quantitative Programming Environments (QPEs) such as {\it Mathematica\/}, MatLab and R. In order to obtain reasonab…
A transformation relation between multivariate ARMA and CARMA processes is derived through a discretization procedure. This gives a direct relationship between the discrete time and continuous time analogues, serving as the basis for an estimation method for multivariate CARMA models. We will see that the autoregressive coefficients, making up the deterministic part of a multivariate CARMA model, are entirely given by the transformation relation. An Euler discretization convergence rate of jump diffusions is found for the case of small jumps of infinite variation. This substantiates applying t…
Stationary processes have been extensively studied in the literature. Their applications include modeling and forecasting numerous real life phenomena such as natural disasters, sales and market movements. When stationary processes are considered, modeling is traditionally based on fitting an autoregressive moving average (ARMA) process. However, we challenge this conventional approach. Instead of fitting an ARMA model, we apply an AR(1) characterization in modeling any strictly stationary processes. Moreover, we derive consistent and asymptotically normal estimators of the corresponding model…
We consider the nonparametric robust estimation problem for regression models in continuous time with semi-Markov noises. An adaptive model selection procedure is proposed. Under general moment conditions on the noise distribution a sharp non-asymptotic oracle inequality for the robust risks is obtained and the robust efficiency is shown. It turns out that for semi-Markov models the robust minimax convergence rate may be faster or slower than the classical one.
In this article we consider the nonparametric robust estimation problem for regression models in continuous time with semi-Markov noises observed in discrete time moments. An adaptive model selection procedure is proposed. A sharp non-asymptotic oracle inequality for the robust risks is obtained. We obtain sufficient conditions on the frequency observations under which the robust efficiency is shown. It turns out that for the semi-Markov models the robust minimax convergence rate may be faster or slower than the classical one.
Based on suitable left-truncated or censored data, two flexible classes of $M$-estimations of Weibull tail coefficient are proposed with two additional parameters bounding the impact of extreme contamination. Asymptotic normality with $\sqrt {n}$-rate of convergence is obtained. Its robustness is discussed via its asymptotic relative efficiency and influence function. It is further demonstrated by a small scale of simulations and an empirical study on CRIX.
The FOU(p) processes can be considered as an alternative to ARMA (or ARFIMA) processes to model time series. Also, there is no substantial loss when we model a time series using FOU(p) processes with the same lambda, than using differents values of lambda. In this work we propose a new method to estimate the unique value of lambda in a FOU(p) process. Under certain conditions, we will prove consistency and asymptotic normality. We will show that this new method is more easy and fast to compute. By simulations, we show that the new procedure work well and is more efficient than the general meth…
In this paper, we propose a novel variable selection approach in the framework of sparse high-dimensional GLARMA models. It consists in combining the estimation of the autoregressive moving average (ARMA) coefficients of these models with regularized methods designed for Generalized Linear Models (GLM). The properties of our approach are investigated both from a theoretical and a numerical point of view. More precisely, we establish in a specific case the consistency of the ARMA part coefficient estimators. We explain how to implement our approach and we show that it is very attractive since i…
Covariance matrix estimation is one of the most important problems in statistics. To accommodate the complexity of modern datasets, it is desired to have estimation procedures that not only can incorporate the structural assumptions of covariance matrices, but are also robust to outliers from arbitrary sources. In this paper, we define a new concept called matrix depth and then propose a robust covariance matrix estimator by maximizing the empirical depth function. The proposed estimator is shown to achieve minimax optimal rate under Huber's $ε$-contamination model for estimating covariance/sc…
University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；University of California, San Diego（加利福尼亚大学圣迭戈分校）
We study the problem of outlier robust high-dimensional mean estimation under a finite covariance assumption, and more broadly under finite low-degree moment assumptions. We consider a standard stability condition from the recent robust statistics literature and prove that, except with exponentially small failure probability, there exists a large fraction of the inliers satisfying this condition. As a corollary, it follows that a number of recently developed algorithms for robust mean estimation, including iterative filtering and non-convex gradient descent, give optimal error estimators with …
University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；University of Cambridge（剑桥大学）
Robust estimation is an important problem in statistics which aims at providing a reasonable estimator when the data-generating distribution lies within an appropriately defined ball around an uncontaminated distribution. Although minimax rates of estimation have been established in recent years, many existing robust estimators with provably optimal convergence rates are also computationally intractable. In this paper, we study several estimation problems under a Wasserstein contamination model and present computationally tractable estimators motivated by generative adversarial networks (GANs)…
University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Duke University（杜克大学）；University of Southern California（南加州大学）
We study the problem of high-dimensional robust mean estimation in the presence of a constant fraction of adversarial outliers. A recent line of work has provided sophisticated polynomial-time algorithms for this problem with dimension-independent error guarantees for a range of natural distribution families. In this work, we show that a natural non-convex formulation of the problem can be solved directly by gradient descent. Our approach leverages a novel structural lemma, roughly showing that any approximate stationary point of our non-convex objective gives a near-optimal solution to the un…
robustloggamma is an R package for robust estimation and inference in the generalized loggamma model. We briefly introduce the model, the estimation procedures and the computational algorithms. Then, we illustrate the use of the package with the help of a real data set.
We provide a new computationally-efficient class of estimators for risk minimization. We show that these estimators are robust for general statistical models: in the classical Huber epsilon-contamination model and in heavy-tailed settings. Our workhorse is a novel robust variant of gradient descent, and we provide conditions under which our gradient descent variant provides accurate estimators in a general convex risk minimization problem. We provide specific consequences of our theory for linear regression, logistic regression and for estimation of the canonical parameters in an exponential f…
We study the problem of robust mean estimation and introduce a novel Hamming distance-based measure of distribution shift for coordinate-level corruptions. We show that this measure yields adversary models that capture more realistic corruptions than those used in prior works, and present an information-theoretic analysis of robust mean estimation in these settings. We show that for structured distributions, methods that leverage the structure yield information theoretically more accurate mean estimation. We also focus on practical algorithms for robust mean estimation and study when data clea…
Robust estimation and variable selection procedure are developed for the extended t-process regression model with functional data. Statistical properties such as consistency of estimators and predictions are obtained. Numerical studies show that the proposed method performs well.
In this paper we present robust estimators for one-shot device test data under lognormal lifetimes. Based on these estimators, confidence intervals and Wald-type tests are also developed. Their robustness feature is illustrated through a simulation study and two numerical examples.
We describe the shrinking neighborhood approach of Robust Statistics, which applies to general smoothly parametrized models, especially, exponential families. Equal generality is achieved by object oriented implementation of the optimally robust estimators. We evaluate the estimates on real datasets from literature by means of our R packages ROptEst and RobLox.
Estimation of graph parameters based on a collection of graphs is essential for a wide range of graph inference tasks. In practice, weighted graphs are generally observed with edge contamination. We consider a weighted latent position graph model contaminated via an edge weight gross error model and propose an estimation methodology based on robust Lq estimation followed by low-rank adjacency spectral decomposition. We demonstrate that, under appropriate conditions, our estimator both maintains Lq robustness and wins the bias-variance tradeoff by exploiting low-rank graph structure. We illustr…
The estimation of signal dimension under heavy-tailed latent factor models is studied. As a primary contribution, robust extensions of an earlier estimator based on Gaussian Stein's unbiased risk estimation are proposed. These novel extensions are based on the framework of elliptical distributions and robust scatter matrices. Extensive simulation studies are conducted in order to compare the novel methods with several well-known competitors in both estimation accuracy and computational speed. The novel methods are applied to a financial asset return data set.
We construct efficient robust truncated sequential estimators for the pointwise estimation problem in nonparametric autoregression models with smooth coefficients. For Gaussian models we propose an adaptive procedure based on the constructed sequential estimators. The minimax nonadaptive and adaptive convergence rates are established. It turns out that in this case these rates are the same as for regression models.
Bai (2010) and Bai et al. (2012) proposed robust mixture regression method based on the M regression estimation. However, the M-estimators are robust against the outliers in response variables, but they are not robust against the outliers in explanatory variables (leverage points). In this paper, we propose a robust mixture regression procedure to handle the outliers and the leverage points, simultaneously. Our proposed mixture regression method is based on the GM regression estimation. We give an Expectation Maximization (EM) type algorithm to compute estimates for the parameters of interest.…
We consider the problem of estimating the mean of a random vector based on i.i.d. observations and adversarial contamination. We introduce a multivariate extension of the trimmed-mean estimator and show its optimal performance under minimal conditions.
This work is concerned with the estimation of the intensity parameter of a stationary determinantal point process. We consider the standard estimator, corresponding to the number of observed points per unit volume and a recently introduced median-based estimator more robust to outliers. The consistency and asymptotic normality of estimators are obtained under mild assumptions on the determinantal point process. We illustrate the efficiency of the procedures in a simulation study.
In robust statistics, the breakdown point of an estimator is the percentage of outliers with which an estimator still generates reliable estimation. The upper bound of breakdown point is 50%, which means it is not possible to generate reliable estimation with more than half outliers. In this paper, it is shown that for majority of experiences, when the outliers exceed 50%, but if they are distributed randomly enough, it is still possible to generate a reliable estimation from minority good observations. The phenomenal of that the breakdown point is larger than 50% is named as super robustness.…
This article studies the weak convergence and associated Central Limit Theorem for blurring and nonblurring processes. Then, they are applied to the estimation of location parameter. Simulation studies show that the location estimation based on the convergence point of blurring process is more robust and often more efficient than that of nonblurring process.
In this paper, we introduce a robust nonparametric density estimator combining the popular Kernel Density Estimation method and the Median-of-Means principle (MoM-KDE). This estimator is shown to achieve robustness to any kind of anomalous data, even in the case of adversarial contamination. In particular, while previous works only prove consistency results under known contamination model, this work provides finite-sample high-probability error-bounds without a priori knowledge on the outliers. Finally, when compared with other robust kernel estimators, we show that MoM-KDE achieves competitiv…
We propose a simple robust hypothesis test that has the same sample complexity as that of the optimal Neyman-Pearson test up to constants, but robust to distribution perturbations under Hellinger distance. We discuss the applicability of such a robust test for estimating distributions in Hellinger distance. We empirically demonstrate the power of the test on canonical distributions.
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
We study the robust mean estimation problem in high dimensions, where $α<0.5$ fraction of the data points can be arbitrarily corrupted. Motivated by compressive sensing, we formulate the robust mean estimation problem as the minimization of the $\ell_0$-`norm' of the outlier indicator vector, under second moment constraints on the inlier data points. We prove that the global minimum of this objective is order optimal for the robust mean estimation problem, and we propose a general framework for minimizing the objective. We further leverage the $\ell_1$ and $\ell_p$ $(0<p<1)$, minimization tech…
