This paper proposes a closed-form optimal estimator based on the theory of estimating functions for a class of linear ARCH models. The estimating function (EF) estimator has the advantage over the widely used maximum likelihood (ML) and quasi-maximum likelihood (QML) estimators that (i) it can be easily implemented, (ii) it does not depend on a distributional assumption for the innovation, and (iii) it does not require the use of any numerical optimization procedures or the choice of initial values of the conditional variance equation. In the case of normality, the asymptotic distribution of the ML and QML estimators naturally turn out to be identical and, hence, coincides with ours. Moreover, a robustness property of the EF estimator is derived by means of influence function. Simulation results show that the efficiency benefits of our estimator relative to the ML and QML estimators are substantial for some ARCH innovation distributions.
We define the minimum covariance determinant functionals for multivariate location and scatter through trimming functions and establish their existence at any multivariate distribution. We provide a precise characterization including a separating ellipsoid property and prove that the functionals are continuous. Moreover, we establish asymptotic normality for both the location and covariance estimator and derive the influence function. These results are obtained in a very general multivariate setting.
Semiparametric Efficient Empirical Higher Order Influence Function Estimators
Institute of Natural Sciences, MOE-LSC, School of Mathematical Sciences, CMA-Shanghai and SJTU-Yale Joint Center for Biostatistics and Data Science, Shanghai Jiao Tong University（上海交通大学自然学院、教育部数学科学中心、CMA-上海和上海交通大学-耶鲁生物统计与数据科学联合中心）；Department of Biostatistics, Harvard University（哈佛大学生物统计系）；Department of Economics, Massachusetts Institute of Technology（麻省理工学院经济学系）；Department of Epidemiology and Biostatistics, Harvard University（哈佛大学流行病学与生物统计系）
General depth weighted scatter estimators are introduced and investigated. For general depth functions, we find out that these affine equivariant scatter estimators are Fisher consistent and unbiased for a wide range of multivariate distributions, and show that the sample scatter estimators are strong and \sqrtn-consistent and asymptotically normal, and the influence functions of the estimators exist and are bounded in general. We then concentrate on a specific case of the general depth weighted scatter estimators, the projection depth weighted scatter estimators, which include as a special ca…
We investigate the time-varying ARCH (tvARCH) process. It is shown that it can be used to describe the slow decay of the sample autocorrelations of the squared returns often observed in financial time series, which warrants the further study of parameter estimation methods for the model. Since the parameters are changing over time, a successful estimator needs to perform well for small samples. We propose a kernel normalized-least-squares (kernel-NLS) estimator which has a closed form, and thus outperforms the previously proposed kernel quasi-maximum likelihood (kernel-QML) estimator for small…
The class of dual $ϕ$-divergence estimators (introduced in Broniatowski and Keziou (2009) is explored with respect to robustness through the influence function approach. For scale and location models, this class is investigated in terms of robustness and asymptotic relative efficiency. Some hypothesis tests based on dual divergence criterions are proposed and their robustness properties are studied. The empirical performances of these estimators and tests are illustrated by Monte Carlo simulation for both noncontaminated and contaminated data.
Many useful parameters depend on nonparametric first steps. Examples include games, dynamic discrete choice, average exact consumer surplus, and treatment effects. Often estimators of these parameters are asymptotically equivalent to a sample average of an object referred to as the influence function. The influence function is useful in local policy analysis, in evaluating local sensitivity of estimators, constructing debiased machine learning estimators, in efficiency comparisons, and in formulating primitive regularity conditions for asymptotic normality, We show that the influence function …
In this paper we propose a recursive online algorithm for estimating the parameters of a time-varying ARCH process. The estimation is done by updating the estimator at time point $t-1$ with observations about the time point $t$ to yield an estimator of the parameter at time point $t$. The sampling properties of this estimator are studied in a non-stationary context -- in particular, asymptotic normality and an expression for the bias due to non-stationarity are established. By running two recursive online algorithms in parallel with different step sizes and taking a linear combination of the e…
We prove the consistency and asymptotic normality of the Laplacian Quasi-Maximum Likelihood Estimator (QMLE) for a general class of causal time series including ARMA, AR($\infty$), GARCH, ARCH($\infty$), ARMA-GARCH, APARCH, ARMA-APARCH,..., processes. We notably exhibit the advantages (moment order and robustness) of this estimator compared to the classical Gaussian QMLE. Numerical simulations confirms the accuracy of this estimator.
We provide a unified approach to a method of estimation of the regression parameter in balanced linear models with a structured covariance matrix that combines a high breakdown point and bounded influence with high asymptotic efficiency at models with multivariate normal errors. Of main interest are linear mixed effects models, but our approach also includes several other standard multivariate models, such as multiple regression, multivariate regression, and multivariate location and scatter. We provide sufficient conditions for the existence of the estimators and corresponding functionals, es…
We consider a robust version of the classical Wald test statistics for testing simple and composite null hypotheses for general parametric models. These test statistics are based on the minimum density power divergence estimators instead of the maximum likelihood estimators. An extensive study of their robustness properties is given though the influence functions as well as the chi-square inflation factors. It is theoretically established that the level and power of these robust tests are stable against outliers, whereas the classical Wald test breaks down. Some numerical examples confirm the …
Stabilized Higher-Order Influence Functions: Statistical Theory of a Class of Bilinear Forms
School of Mathematical Sciences, Shanghai Jiao Tong University（上海交通大学数学科学学院）；Department of Statistics, University of Virginia（弗吉尼亚大学统计学系）；Department of Statistics and Data Science, Tsinghua University（清华大学统计与数据科学系）；Institute of Natural Sciences, MOE–LSC, CMA–Shanghai, SJTU–Yale Joint Center for Biostatistics and Data Science, Shanghai Jiao Tong University（上海交通大学自然科学研究院、教育部数学天元基金管理中心-上海数学中心、上海交通大学-耶鲁大学生物统计与数据科学联合中心）
In this paper we propose a family of robust estimates for isotonic regression: isotonic M-estimators. We show that their asymptotic distribution is, up to an scalar factor, the same as that of Brunk's classical isotonic estimator. We also derive the influence function and the breakdown point of these estimates. Finally we perform a Monte Carlo study that shows that the proposed family includes estimators that are simultaneously highly efficient under gaussian errors and highly robust when the error distribution has heavy tails.
We consider a model $Y\_t=σ\_tη\_t$ in which $(σ\_t)$ is not independent of the noise process $(η\_t)$, but $σ\_t$ is independent of $η\_t$ for each $t$. We assume that $(σ\_t)$ is stationary and we propose an adaptive estimator of the density of $\ln(σ^2\_t)$ based on the observations $Y\_t$. Under various dependence structures, the rates of this nonparametric estimator coincide with the minimax rates obtained in the i.i.d. case when $(σ\_t)$ and $(η\_t)$ are independent, in all cases where these minimax rates are known. The results apply to various linear and non linear ARCH processes.
A broad class of smooth, possibly data-adaptive nonparametric copula estimators that contains empirical Bernstein copulas introduced by Sancetta and Satchell (and thus the empirical beta copula proposed by Segers, Sibuya and Tsukahara) is studied. Within this class, a subclass of estimators that depend on a scalar parameter determining the amount of marginal smoothing and a functional parameter controlling the shape of the smoothing region is specifically considered. Empirical investigations of the influence of these parameters suggest to focus on two particular data-adaptive smooth copula est…
Zhang (2019) presented a general estimation approach based on the Gaussian distribution for general parametric models where the likelihood of the data is difficult to obtain or unknown, but the mean and variance-covariance matrix are known. Castilla and Zografos (2021) extended the method to density power divergence-based estimators, which are more robust than the likelihood-based Gaussian estimator against data contamination. In this paper we introduce the restricted minimum density power divergence Gaussian estimator (MDPDGE) and study its main asymptotic properties. Also, we examine it robu…
A new robust and statistically efficient estimator for ARMA models called the bounded influence propagation (BIP) τ-estimator is proposed. The estimator incorporates an auxiliary model, which prevents the propagation of outliers. Strong consistency and asymptotic normality of the estimator for ARMA models that are driven by independently and identically distributed (iid) innovations with symmetric distributions are established. To analyze the infinitesimal effect of outliers on the estimator, the influence function is derived and computed explicitly for an AR(1) model with additive outliers. T…
Strong consistency and asymptotic normality of the Quasi-Maximum Likelihood Estimator (QMLE) are given for a general class of multidimensional causal processes. For particular cases already studied in the literature (for instance univariate or multivariate GARCH, ARCH, ARMA-GARCH processes) the assumptions required for establishing these results are often weaker than existing conditions. The QMLE asymptotic behavior is also given for numerous new examples of univariate or multivariate processes (for instance TARCH or NLARCH processes).
This paper studies nonparametric regression with long memory (LRD) errors and predictors. First, we formulate general conditions which guarantee the standard rate of convergence for a nonparametric kernel estimator. Second, we calculate the Mean Integrated Squared Error (MISE). In particular, we show that LRD of errors may influence MISE. On the other hand, an estimator for a shape function is typically not influenced by LRD in errors. Finally, we investigate properties of a data-driven bandwidth choice. We show that Averaged Squared Error (ASE) is a good approximation of MISE, however, this i…
The subject of robust estimation in time series is widely discussed in literature. One of the approaches is to use GM-estimation. This method incorporates a broad class of nonparametric estimators which under suitable conditions includes estimators robust to outliers in data. For the linear models the sensitivity of GM-estimators to outliers have been studied in the work by Martin and Yohai [5], and influence functionals for this estimator were derived. In this paper we follow this direction and examine the asymptotical properties of the class of M-estimators, which is narrower than the class …
Since the two seminal papers by Fisher (1915, 1921) were published, the test under a fixed value correlation coefficient null hypothesis for the bivariate normal distribution constitutes an important statistical problem. In the framework of asymptotic robust statistics, it remains being a topic of great interest to be investigated. For this and other tests, focused on paired correlated normal random samples, Rényi's pseudodistance estimators are proposed, their asymptotic distribution is established and an iterative algorithm is provided for their computation. From them the Wald-type test stat…
Coordinated Science Laboratory, University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校协调科学实验室）；ECE Department, University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校电子与计算机工程系）
We consider the problem of identifying parameters of a particular class of Markov chains, called Bernoulli Autoregressive (BAR) processes. The structure of any BAR model is encoded by a directed graph. Incoming edges to a node in the graph indicate that the state of the node at a particular time instant is influenced by the states of the corresponding parental nodes in the previous time instant. The associated edge weights determine the corresponding level of influence from each parental node. In the simplest setup, the Bernoulli parameter of a particular node's state variable is a convex comb…
The problem of estimating the directed information rate between two discrete processes $\{X_n\}$ and $\{Y_n\}$ via the plug-in (or maximum-likelihood) estimator is considered. When the joint process $\{(X_n,Y_n)\}$ is a Markov chain of a given memory length, the plug-in estimator is shown to be asymptotically Gaussian and to converge at the optimal rate $O(1/\sqrt{n})$ under appropriate conditions; this is the first estimator that has been shown to achieve this rate. An important connection is drawn between the problem of estimating the directed information rate and that of performing a hypoth…
Many mathematical models involve input parameters, which are not precisely known. Global sensitivity analysis aims to identify the parameters whose uncertainty has the largest impact on the variability of a quantity of interest (output of the model). One of the statistical tools used to quantify the influence of each input variable on the output is the Sobol sensitivity index. We consider the statistical estimation of this index from a finite sample of model outputs: we present two estimators and state a central limit theorem for each. We show that one of these estimators has an optimal asympt…
This paper deals with optimally-robust parameter estimation in generalized Pareto distributions (GPDs). These arise naturally in many situations where one is interested in the behavior of extreme events as motivated by the Pickands-Balkema-de Haan extreme value theorem (PBHT). The application we have in mind is calculation of the regulatory capital required by Basel II for a bank to cover operational risk. In this context the tail behavior of the underlying distribution is crucial. This is where extreme value theory enters, suggesting to estimate these high quantiles parameterically using, e.g…
The minimum divergence estimators have proved to be useful tools in the area of robust inference. The robustness of such estimators are measured using the classical Influence functions. However, in many complex situations like testing a composite hypothesis using divergence require the estimators to be restricted into some subspace of the parameter space. The robustness of these restricted minimum divergence estimators are very important in order to have overall robust inference. In this paper we provide a comprehensive description of the robustness of such restricted estimators in terms of th…
This paper studies the generalization of the targeted minimum loss-based estimation (TMLE) framework to estimation of effects of time-varying interventions in settings where both interventions, covariates, and outcome can happen at subject-specific time-points on an arbitrarily fine time-scale. TMLE is a general template for constructing asymptotically linear substitution estimators for smooth low-dimensional parameters in infinite-dimensional models. Existing longitudinal TMLE methods are developed for data where observations are made on a discrete time-grid. We consider a continuous-time cou…
In multiple testing, the family-wise error rate can be bounded under some conditions by the copula of the test statistics. Assuming that this copula is Archimedean, we consider two non-parametric Archimedean generator estimators. More specifically, we use the non-parametric estimator from Genest et al. (2011) and a slight modification thereof. In simulations, we compare the resulting multiple tests with the Bonferroni test and the multiple test derived from the true generator as baselines.
We present a new finite-sample analysis of M-estimators of locations in $\mathbb{R}^d$ using the tool of the influence function. In particular, we show that the deviations of an M-estimator can be controlled thanks to its influence function (or its score function) and then, we use concentration inequality on M-estimators to investigate the robust estimation of the mean in high dimension in a corrupted setting (adversarial corruption setting) for bounded and unbounded score functions. For a sample of size $n$ and covariance matrix $Σ$, we attain the minimax speed $\sqrt{Tr(Σ)/n}+\sqrt{\|Σ\|_{op…
A new robust correlation estimator based on the spatial sign covariance matrix (SSCM) is proposed. We derive its asymptotic distribution and influence function at elliptical distributions. Finite sample and robustness properties are studied and compared to other robust correlation estimators by means of numerical simulations.
Recently, many estimators for network treatment effects have been proposed. But, their optimality properties in terms of semiparametric efficiency have yet to be resolved. We present a simple, yet flexible asymptotic framework to derive the efficient influence function and the semiparametric efficiency lower bound for a family of network causal effects under partial interference. An important corollary of our results is that one of the existing estimators by Liu et al. (2019) is locally efficient. We also present other estimators that are efficient and discuss results on adaptive estimation. W…
By deriving influence functions related to multiple-set linear canonical analysis (MSLCA) we show that the classical version of this analysis, based on empirical covariance operators, is not robust. Then, we introduce a robust version of MSLCA by using the MCD estimator of the covariance operator of the involved random vector. The related influence functions are then derived and are shown to be bounded. Asymptotic properties of the introduced robust MSLCA are obtained and permit to propose a robust test for mutual non-correlation. This test is shown to be robust by studying the related second …
Approximate statistical inference via determination of the asymptotic distribution of a statistic is routinely used for inference in applied medical statistics (e.g. to estimate the standard error of the marginal or conditional risk ratio). One method for variance estimation is the classical Delta-method but there is a knowledge gap as this method is not routinely included in training for applied medical statistics and its uses are not widely understood. Given that a smooth function of an asymptotically normal estimator is also asymptotically normally distributed, the Delta-method allows appro…
Evaluation of treatment effects and more general estimands is typically achieved via parametric modelling, which is unsatisfactory since model misspecification is likely. Data-adaptive model building (e.g. statistical/machine learning) is commonly employed to reduce the risk of misspecification. Naive use of such methods, however, delivers estimators whose bias may shrink too slowly with sample size for inferential methods to perform well, including those based on the bootstrap. Bias arises because standard data-adaptive methods are tuned towards minimal prediction error as opposed to e.g. min…
This paper derives a new family of estimators, namely the minimum density power divergence estimators, as a robust generalization of the maximum likelihood estimator for the polytomous logistic regression model. Based on these estimators, a family of Wald-type test statistics for linear hypotheses is introduced. Robustness properties of both the proposed estimators and the test statistics are theoretically studied through the classical influence function analysis. Appropriate real life examples are presented to justify the requirement of suitable robust statistical procedures in place of the l…
Parameter estimation in empirical fields is usually undertaken using parametric models, and such models readily facilitate statistical inference. Unfortunately, they are unlikely to be sufficiently flexible to be able to adequately model real-world phenomena, and may yield biased estimates. Conversely, non-parametric approaches are flexible but do not readily facilitate statistical inference and may still exhibit residual bias. We explore the potential for Influence Functions (IFs) to (a) improve initial estimators without needing more data (b) increase model robustness and (c) facilitate stat…
We introduce a new family of estimators for unnormalized statistical models. Our family of estimators is parameterized by two nonlinear functions and uses a single sample from an auxiliary distribution, generalizing Maximum Likelihood Monte Carlo estimation of Geyer and Thompson (1992). The family is such that we can estimate the partition function like any other parameter in the model. The estimation is done by optimizing an algebraically simple, well defined objective function, which allows for the use of dedicated optimization methods. We establish consistency of the estimator family and gi…
The precision matrix that encodes conditional linear dependency relations among a set of variables forms an important object of interest in multivariate analysis. Sparse estimation procedures for precision matrices such as the graphical lasso (Glasso) gained popularity as they facilitate interpretability, thereby separating pairs of variables that are conditionally dependent from those that are independent (given all other variables). Glasso lacks, however, robustness to outliers. To overcome this problem, one typically applies a robust plug-in procedure where the Glasso is computed from a rob…
Estimators based on influence functions (IFs) have been shown to be effective in many settings, especially when combined with machine learning techniques. By focusing on estimating a specific target of interest (e.g., the average effect of a treatment), rather than on estimating the full underlying data generating distribution, IF-based estimators are often able to achieve asymptotically optimal mean-squared error. Still, many researchers find IF-based estimators to be opaque or overly technical, which makes their use less prevalent and their benefits less available. To help foster understandi…
Ratios of sample percentiles or of quantiles based on a single sample are often published for skewed income data to illustrate aspects of income inequality, but distribution-free confidence intervals for such ratios are to our knowledge not in the literature. Here we derive and compare two large-sample methods for obtaining such intervals. They both require good distribution-free estimates of the quantile density at the quantiles of interest, and such estimates have recently become available. Simulation studies for various sample sizes are carried out for Pareto, lognormal and exponential dist…
This paper aims to provide a tutorial for upper level undergraduate and graduate students in statistics, biostatistics and epidemiology on deriving influence functions for non-parametric and semi-parametric models. The author will build on previously known efficiency theory and provide a useful identity and formulaic technique only relying on the basics of integration which, are self-contained in this tutorial and can be used in most any setting one might encounter in practice. The paper provides many examples of such derivations for well-known influence functions as well as for new parameters…
Higher-Order Influence Functions (HOIFs) provide a unified theory for constructing rate-optimal estimators for a large class of low-dimensional (smooth) statistical functionals/parameters (and sometimes even infinite-dimensional functions) that arise in substantive fields including epidemiology, economics, and the social sciences. Since the introduction of HOIFs by Robins et al. (2008), they have been viewed mostly as a theoretical benchmark rather than a useful tool for statistical practice. Works aimed to flip the script are scant, but a few recent papers Liu et al. (2017, 2021b) make some p…
We restrict our attention to space-time point pattern data for which we have a single realisation within a finite region. Second-order characteristics are used to analyse the spatio-temporal structure of the underlying point process. In particular, the space-time inhomogeneous pair correlation function and $K$-function measure the spatio-temporal clustering/regularity and the spatio-temporal interaction, and thus help with model choice. Non-parametric estimators of the second-order characteristics require information from outside the study region, resulting to the so-called edge effects which …
We generalize the { ${\rm M}$-estimator} put forward by Catoni in his seminal paper [C12] to the case in which samples can have finite $α$-th moment with $α\in (1,2)$ rather than finite variance, our approach is by slightly modifying the influence function $φ$ therein. The choice of the new influence function is inspired by the Taylor-like expansion developed in [C-N-X]. We obtain a deviation bound of the estimator, as $α\rightarrow 2$, this bound is the same as that in [C12]. Experiment shows that our generalized ${\rm M}$-estimator performs better than the empirical mean estimator, the small…
In this paper we introduce an influence measure based on second order expansion of the RV and GCD measures for the comparison between unperturbed and perturbed eigenvectors of a symmetric matrix estimator. Example estimators are considered to highlight how this measure compliments recent influence analysis. Importantly, we also show how a sample based version of this measure can be used to accurately and efficiently detect influential observations in practice.
