The Classical Tukey-Huber Contamination Model (CCM) is a usual framework to describe the mechanism of outliers generation in robust statistics. In a data set with $n$ observations and $p$ variables, under the CCM, an outlier is a unit, even if only one or few values are corrupted. Classical robust procedures were designed to cope with this setting and the impact of observations were limited whenever necessary. Recently, a different mechanism of outliers generation, namely Independent Contamination Model (ICM), was introduced. In this new setting each cell of the data matrix might be corrupted or not with a probability independent on the status of the other cells. ICM poses new challenge to robust statistics since the percentage of contaminated rows dramatically increase with $p$, often reaching more than $50\%$. When this situation appears, classical affine equivariant robust procedures do not work since their breakdown point is $50\%$. For this contamination model we propose a new type of robust methods namely composite robust procedures which are inspired on the idea of composite likelihood, where low dimension likelihood, very often the likelihood of pairs, are aggregate together in order to obtain an approximation of the full likelihood which is more tractable. Our composite robust procedures are build over pairs of observations in order to gain robustness in the independent contamination model. We propose composite S and $τ$-estimators for linear mixed models. Composite $τ$-estimators are proved to have an high breakdown point both in the CCM and ICM. A Monte Carlo study shows that our estimators compare favorably with respect to classical S-estimators under the CCM and outperform them under the ICM. One example based on a real data set illustrates the new robust procedure.
Composite likelihoods are a class of alternatives to the full likelihood which are widely used in many situations in which the likelihood itself is intractable. A composite likelihood may be computed without the need to specify the full distribution of the response, which means that in some situations the resulting estimator will be more robust to model misspecification than the maximum likelihood estimator. The purpose of this note is to show that such increased robustness is not guaranteed. An example is given in which various marginal composite likelihood estimators are inconsistent under m…
The maximum composite likelihood estimator for parametric models of determinantal point processes (DPPs) is discussed. Since the joint intensities of these point processes are given by determinant of positive definite kernels, we have the explicit form of the joint intensities for every order. This fact enables us to consider the generalized maximum composite likelihood estimator for any order. This paper introduces the two step generalized composite likelihood estimator and shows the moment convergence of the estimator under a stationarity. Moreover, our results can yield information criteria…
Likelihood inference for max-stable random fields is in general impossible because their finite-dimen\-sional probability density functions are unknown or cannot be computed efficiently. The weighted composite likelihood approach that utilizes lower dimensional marginal likelihoods (typically pairs or triples of sites that are not too distant) is rather favored. In this paper, we consider the family of spatial max-stable Brown-Resnick random fields associated with isotropic fractional Brownian fields. We assume that the sites are given by only one realization of a homogeneous Poisson point pro…
We study the problem of estimating the covariance parameters of a one-dimensional Gaussian process with exponential covariance function under fixed-domain asymptotics. We show that the weighted pairwise maximum likelihood estimator of the microergodic parameter can be consistent or inconsistent. This depends on the range of admissible parameter values in the likelihood optimization. On the other hand, the weighted pairwise conditional maximum likelihood estimator is always consistent. Both estimators are also asymptotically Gaussian when they are consistent. Their asymptotic variances are larg…
Properties of mixed fractional Brownian motion has been discussed by Cheridito (2001) and Zili (2006). We have proposed an estimator of volatility parameter for a model driven by MFBM. In our article we have shown that the estimator has some desirable asymptotic properties.
Asymptotic properties of maximum composite likelihood estimators for max-stable Brown-Resnick random fields over a fixed-domain
Université du Littoral Côte d’Opale（加来海峡海岸大学）；Université de Lyon, Université Lyon 1, Institut de Science Financière et d’Assurances（里昂大学，里昂第一大学，金融与保险科学研究所）；Laboratory in Finance and Insurance - LFA CREST - Center for Research in Economics and Statistics, ENSAE（金融与保险实验室 - LFA CREST - 经济与统计研究中心，国立经济统计学院）
The goal of mediation analysis is to study the effect of exposure on an outcome interceded by a mediator. Two simple hypotheses are tested: the effect of the exposure on the mediator, and the effect of the mediator on the outcome. When either of these hypotheses is true, a predetermined significance level can be assured. When both nulls are true, the same test becomes conservative. Adaptively finding the correct scenario enables customizing the tests and consequently enlarges their efficiency, which is most important in a multiple testing framework. In this work, we link between adaptive two-s…
Generalized linear mixed models are powerful tools for analyzing clustered data, where the unknown parameters are classically (and most commonly) estimated by the maximum likelihood and restricted maximum likelihood procedures. However, since the likelihood based procedures are known to be highly sensitive to outliers, M-estimators have become popular as a means to obtain robust estimates under possible data contamination. In this paper, we prove that, for sufficiently smooth general loss functions defining the M-estimators in generalized linear mixed models, the tail probability of the deviat…
We investigate the modeling and the numerical solution of machine learning problems with prediction functions which are linear combinations of elements of a possibly infinite-dimensional dictionary. We propose a novel flexible composite regularization model, which makes it possible to incorporate various priors on the coefficients of the prediction function, including sparsity and hard constraints. We show that the estimators obtained by minimizing the regularized empirical risk are consistent in a statistical sense, and we design an error-tolerant composite proximal thresholding algorithm for…
Composite likelihoods are increasingly used in applications where the full likelihood is analytically unknown or computationally prohibitive. Although the maximum composite likelihood estimator has frequentist properties akin to those of the usual maximum likelihood estimator, Bayesian inference based on composite likelihoods has yet to be explored. In this paper we investigate the use of the Metropolis--Hastings algorithm to compute a pseudo-posterior distribution based on the composite likelihood. Two methodologies for adjusting the algorithm are presented and their performance on approximat…
In statistical analysis, measuring a score of predictive performance is an important task. In many scientific fields, appropriate scores were tailored to tackle the problems at hand. A proper score is a popular tool to obtain statistically consistent forecasts. Furthermore, a mathematical characterization of the proper score was studied. As a result, it was revealed that the proper score corresponds to a Bregman divergence, which is an extension of the squared distance over the set of probability distributions. In the present paper, we introduce composite scores as an extension of the typical …
In this article we study a class of parameters with the so-called `mixed bias property'. For parameters with this property, the bias of the semiparametric efficient one step estimator is equal to the mean of the product of the estimation errors of two nuisance functions. In non-parametric models, parameters with the mixed bias property admit so-called rate doubly robust estimators, i.e. estimators that are consistent and asymptotically normal when one succeeds in estimating both nuisance functions at sufficiently fast rates, with the possibility of trading off slower rates of convergence for t…
We study behavior of the restricted maximum likelihood (REML) estimator under a misspecified linear mixed model (LMM) that has received much attention in recent gnome-wide association studies. The asymptotic analysis establishes consistency of the REML estimator of the variance of the errors in the LMM, and convergence in probability of the REML estimator of the variance of the random effects in the LMM to a certain limit, which is equal to the true variance of the random effects multiplied by the limiting proportion of the nonzero random effects present in the LMM. The aymptotic results also …
Motivated by better modeling of intra-individual variability in longitudinal data, we propose a class of location-scale mixed effects models, in which the data of each individual is modeled by a parameter-varying generalized hyperbolic distribution. We first study the local maximum-likelihood asymptotics and reveal the instability in the numerical optimization of the log-likelihood. Then, we construct an asymptotically efficient estimator based on the Newton-Raphson method based on the original log-likelihood function with the initial estimator being naive least-squares-type. Numerical experim…
Motivated by applications in genetic fields, we propose to estimate the heritability in high dimensional sparse linear mixed models. The heritability determines how the variance is shared between the different random components of a linear mixed model. The main novelty of our approach is to consider that the random effects can be sparse, that is may contain null components, but we do not know neither their proportion nor their positions. The estimator that we consider is strongly inspired by the one proposed by Pirinen et al. (2013), and is based on a maximum likelihood approach. We also study…
The issues of bias-correction and robustness are crucial in the strategy of divide-and-conquer (DC), especially for asymmetric nonparametric models with massive data. It is known that quantile-based methods can achieve the robustness, but the quantile estimation for nonparametric regression has non-ignorable bias when the error distribution is asymmetric. This paper explores a global bias-corrected DC by quantile-matched composite for nonparametric regressions with general error distributions. The proposed strategies can achieve the bias-correction and robustness, simultaneously. Unlike common…
Optimization under uncertainty and risk is indispensable in many practical situations. Our paper addresses stability of optimization problems using composite risk functionals which are subjected to measure perturbations. Our main focus is the asymptotic behavior of data-driven formulations with empirical or smoothing estimators such as kernels or wavelets applied to some or to all functions of the compositions. We analyze the properties of the new estimators and we establish strong law of large numbers, consistency, and bias reduction potential under fairly general assumptions. Our results are…
We present new results for consistency of maximum likelihood estimators with a focus on multivariate mixed models. Our theory builds on the idea of using subsets of the full data to establish consistency of estimators based on the full data. It requires neither that the data consist of independent observations, nor that the observations can be modeled as a stationary stochastic process. Compared to existing asymptotic theory using the idea of subsets we substantially weaken the assumptions, bringing them closer to what suffices in classical settings. We apply our theory in two multivariate mix…
Wasserman et al. (2020, PNAS, vol. 117, pp. 16880-16890) constructed estimator agnostic and finite-sample valid confidence sets and hypothesis tests, using split-data likelihood ratio-based statistics. We demonstrate that the same approach extends to the use of split-data composite likelihood ratios as well, and thus establish universal methods for conducting multivariate inference when the data generating process is only known up to marginal and conditional relationships between the coordinates. Always-valid sequential inference is also considered.
We extend a recently established asymptotic normality theorem for generalized linear mixed models to include the dispersion parameter. The new results show that the maximum likelihood estimators of all model parameters have asymptotically normal distributions with asymptotic mutual independence between fixed effects, random effects covariance and dispersion parameters. The dispersion parameter maximum likelihood estimator has a particularly simple asymptotic distribution which enables straightforward valid likelihood-based inference.
Does the asymptotic variance of the maximum composite likelihood estimator of a parameter of interest always decrease when the nuisance parameters are known? Will a composite likelihood necessarily become more efficient by incorporating addi- tional independent component likelihoods, or by using component likelihoods with higher dimension? In this note we show through illustrative examples that the an- swer to both questions is no, and indeed the opposite direction might be observed. The role of information bias is highlighted to understand the occurrence of these paradoxical phenomenon.
In the linear random effects model, when distributional assumptions such as normality of the error variables cannot be justified, moments may serve as alternatives to describe relevant distributions in neighborhoods of their means. Generally, estimators may be obtained as solutions of estimating equations. It turns out that there may be several equations, each of them leading to consistent estimators, in which case finding the efficient estimator becomes a crucial problem. In this paper, we systematically study estimation of moments of the errors and random effects in linear mixed models.
We study the problem of nonparametric estimation of a multivariate function $g:\mathbb {R}^d\to\mathbb{R}$ that can be represented as a composition of two unknown smooth functions $f:\mathbb{R}\to\mathbb{R}$ and $G:\mathbb{R}^d\to \mathbb{R}$. We suppose that $f$ and $G$ belong to known smoothness classes of functions, with smoothness $γ$ and $β$, respectively. We obtain the full description of minimax rates of estimation of $g$ in terms of $γ$ and $β$, and propose rate-optimal estimators for the sup-norm loss. For the construction of such estimators, we first prove an approximation result for…
Asymptotic Theory for Restricted Maximum Likelihood Estimators in Generalized Linear Mixed Models
School of Systems and Computing, University of New South Wales（新南威尔士大学系统计算学院）；UNSW Data Science Hub（新南威尔士大学数据科学中心）；School of Mathematics and Statistics, University of New South Wales（新南威尔士大学数学与统计学院）
We consider three problems in high-dimensional Gaussian linear mixed models. Without any assumptions on the design for the fixed effects, we construct an asymptotic $F$-statistic for testing whether a collection of random effects is zero, derive an asymptotic confidence interval for a single random effect at the parametric rate $\sqrt{n}$, and propose an empirical Bayes estimator for a part of the mean vector in ANOVA type models that performs asymptotically as well as the oracle Bayes estimator. We support our results with numerical simulations and provide comparisons with oracle estimators. …
We prove the local asymptotic mixed normality (LAMN) property for a family of probability measures defined by parametrized diffusion processes with nonsynchronous observations. We assume that observation times of processes are independent of processes and we will study asymptotics when the maximum length of observation intervals goes to zero in probability. We also prove that the quasi-maximum likelihood estimator and the Bayes-type estimator proposed in Ogihara and Yoshida (Stochastic Process. Appl. 124 (2014) 2954-3008) are asymptotically efficient.
We present a robust test for composite null hypothesis based on the general $S$-divergence family. This requires a non-trivial extension of the results of Ghosh et al.~(2015). We derive the asymptotic and theoretical robustness properties of the resulting test along with the properties of the minimum $S$-divergence estimators under parameter restrictions imposed by the null hypothesis. An illustration in the context of the normal model is also presented.
Likelihood-based inference for birth-death processes with composite birth mechanisms
Network Science Institute, Northeastern University London（伦敦东北大学网络科学研究所）；Department of Mathematics, University of Sussex（萨塞克斯大学数学系）；Department of Mathematics, Northeastern University（东北大学数学系）
The mixed fractional Vasicek model, which is an extended model of the traditional Vasicek model, has been widely used in modelling volatility, interest rate and exchange rate. Obviously, if some phenomenon are modeled by the mixed fractional Vasicek model, statistical inference for this process is of great interest. Based on continuous time observations, this paper considers the problem of estimating the drift parameters in the mixed fractional Vasicek model. We will propose the maximum likelihood estimators of the drift parameters in the mixed fractional Vasicek model with the Radon-Nikodym d…
Fields like public health, public policy, and social science often want to quantify the degree of dependence between variables whose relationships take on unknown functional forms. Typically, in fact, researchers in these fields are attempting to evaluate causal theories, and so want to quantify dependence after conditioning on other variables that might explain, mediate or confound causal relations. One reason conditional mutual information is not more widely used for these tasks is the lack of estimators which can handle combinations of continuous and discrete random variables, common in app…
The quasi-likelihood estimator and the Bayesian type estimator of the volatility parameter are in general asymptotically mixed normal. In case the limit is normal, the asymptotic expansion was derived in Yoshida (1997) as an application of the martingale expansion. The expansion for the asymptotically mixed normal distribution is then indispensable to develop the higher-order approximation and inference for the volatility. The classical approaches in limit theorems, where the limit is a process with independent increments or a simple mixture, do not work. We present asymptotic expansion of a m…
This paper considers the problem of robust hypothesis testing under non-identically distributed data. We propose Wald-type tests for both simple and composite hypothesis for independent but non-homogeneous observations based on the robust minimum density power divergence estimator of the common underlying parameter. Asymptotic and theoretical robustness properties of the proposed tests have been discussed. Application to the problem of testing the general linear hypothesis in a generalized linear model with fixed-design has been considered in detail with specific illustrations for its special …
London School of Economics and Political Science（伦敦政治经济学院）；University of Minnesota（明尼苏达大学）
Matrix completion is a class of machine learning methods that concerns the prediction of missing entries in a partially observed matrix. This paper studies matrix completion for mixed data, i.e., data involving mixed types of variables (e.g., continuous, binary, ordinal). We formulate it as a low-rank matrix estimation problem under a general family of non-linear factor models and then propose entrywise consistent estimators for estimating the low-rank matrix. Tight probabilistic error bounds are derived for the proposed estimators. The proposed methods are evaluated by simulation studies and …
We consider the matrix completion problem of recovering a structured low rank matrix with partially observed entries with mixed data types. Vast majority of the solutions have proposed computationally feasible estimators with strong statistical guarantees for the case where the underlying distribution of data in the matrix is continuous. A few recent approaches have extended using similar ideas these estimators to the case where the underlying distributions belongs to the exponential family. Most of these approaches assume that there is only one underlying distribution and the low rank constra…
This paper introduces a high-dimensional linear IV regression for the data sampled at mixed frequencies. We show that the high-dimensional slope parameter of a high-frequency covariate can be identified and accurately estimated leveraging on a low-frequency instrumental variable. The distinguishing feature of the model is that it allows handing high-dimensional datasets without imposing the approximate sparsity restrictions. We propose a Tikhonov-regularized estimator and derive the convergence rate of its mean-integrated squared error for time series data. The estimator has a closed-form expr…
