We consider in this paper the semiparametric mixture of two distributions equal up to a shift parameter. The model is said to be semiparametric in the sense that the mixed distribution is not supposed to belong to a parametric family. In order to insure the identifiability of the model it is assumed that the mixed distribution is symmetric, the model being then defined by the mixing proportion, two location parameters, and the probability density function of the mixed distribution. We propose a new class of M-estimators of these parameters based on a Fourier approach, and prove that they are square root consistent under mild regularity conditions. Their finite-sample properties are illustrated by a Monte Carlo study and a benchmark real dataset is also studied with our method.
In this article, we propose two classes of semiparametric mixture regression models with single-index for model based clustering. Unlike many semiparametric/nonparametric mixture regression models that can only be applied to low dimensional predictors, the new semiparametric models can easily incorporate high dimensional predictors into the nonparametric components. The proposed models are very general, and many of the recently proposed semiparametric/nonparametric mixture regression models are indeed special cases of the new models. Backfitting estimates and the corresponding modified EM algo…
To avoid specification of the error distribution in a regression model, we propose a general nonparametric scale mixture model for the error distribution. For fitting such mixtures, the predictive recursion method is a simple and computationally efficient alternative to existing methods. We define a predictive recursion-based marginal likelihood function, and estimation of the regression parameters proceeds by maximizing this function. A hybrid predictive recursion--EM algorithm is proposed for this purpose. The method's performance is compared with that of existing methods in simulations and …
There is a difficulty in finding an estimate of variance of the profile likelihood estimator in the joint model of longitudinal and survival data. We solve the difficulty by introducing the ``statistical generalized derivative''. The derivative is used to show the asymptotic normality of the estimator without assuming the second derivative of the density function in the model exists.
In this paper, we consider nonparametric multidimensional finite mixture models and we are interested in the semiparametric estimation of the population weights. Here, the i.i.d. observations are assumed to have at least three components which are independent given the population. We approximate the semiparametric model by projecting the conditional distributions on step functions associated to some partition. Our first main result is that if we refine the partition slowly enough, the associated sequence of maximum likelihood estimators of the weights is asymptotically efficient, and the poste…
Motivated by the analysis of a Positron Emission Tomography (PET) imaging data considered in Bowen et al. (2012), we introduce a semiparametric topographical mixture model able to capture the characteristics of dichotomous shifted response-type experiments. We propose a local estimation procedure, based on the symmetry of the local noise, for the proportion and locations functions involved in the proposed model. We establish under mild conditions the minimax properties and asymptotic normality of our estimators when Monte Carlo simulations are conducted to examine their finite sample performan…
An important and yet difficult problem in fitting multivariate mixture models is determining the mixture complexity. We develop theory and a unified framework for finding the nonparametric maximum likelihood estimator of a multivariate mixing distribution and consequently estimating the mixture complexity. Multivariate mixtures provide a flexible approach to fitting high-dimensional data while offering data reduction through the number, location and shape of the component densities. The central principle of our method is to cast the mixture maximization problem in the concave optimization fram…
We consider a semiparametric mixture of two univariate density functions where one of them is known while the weight and the other function are unknown. Such mixtures have a history of application to the problem of detecting differentially expressed genes under two or more conditions in microarray data. Until now, some additional knowledge about the unknown component (e.g. the fact that it belongs to a location family) has been assumed. As opposed to this approach, we do not assume any additional structure on the unknown density function. For this mixture model, we derive a new sufficient iden…
Suppose that univariate data are drawn from a mixture of two distributions that are equal up to a shift parameter. Such a model is known to be nonidentifiable from a nonparametric viewpoint. However, if we assume that the unknown mixed distribution is symmetric, we obtain the identifiability of this model, which is then defined by four unknown parameters: the mixing proportion, two location parameters and the cumulative distribution function of the symmetric mixed distribution. We propose estimators for these four parameters when no training data is available. Our estimators are shown to be st…
For extreme value estimation we propose to use a model with a Dirichlet process mixture of gamma densities in the center and generalized Pareto densities for the tails. Due to the randomness in the center and a heavy tailed density in the tails density estimation and posterior inference for high quantiles are possible. The approach can be used in a "default" manner on the positive reals because it works when prior information is unavailable. The proposed model can be easy to implement and a sensitivity analysis is provided. We applied the proposed model for simulated and real data sets.
We consider X 1 ,. .. , X n a sample of data on the circle S 1 , whose distribution is a twocomponent mixture. Denoting R and Q two rotations on S 1 , the density of the X i 's is assumed to be g(x) = pf (R --1 x) + (1 -- p)f (Q --1 x), where p $\in$ (0, 1) and f is an unknown density on the circle. In this paper we estimate both the parametric part $θ$ = (p, R, Q) and the nonparametric part f. The specific problems of identifiability on the circle are studied. A consistent estimator of $θ$ is introduced and its asymptotic normality is proved. We propose a Fourier-based estimator of f with a p…
Predictive recursion is an accurate and computationally efficient algorithm for nonparametric estimation of mixing densities in mixture models. In semiparametric mixture models, however, the algorithm fails to account for any uncertainty in the additional unknown structural parameter. As an alternative to existing profile likelihood methods, we treat predictive recursion as a filter approximation to fitting a fully Bayes model, whereby an approximate marginal likelihood of the structural parameter emerges and can be used for inference. We call this the predictive recursion marginal likelihood.…
We propose Adaptive Incremental Mixture Markov chain Monte Carlo (AIMM), a novel approach to sample from challenging probability distributions defined on a general state-space. While adaptive MCMC methods usually update a parametric proposal kernel with a global rule, AIMM locally adapts a semiparametric kernel. AIMM is based on an independent Metropolis-Hastings proposal distribution which takes the form of a finite mixture of Gaussian distributions. Central to this approach is the idea that the proposal distribution adapts to the target by locally adding a mixture component when the discrepa…
In a smooth semiparametric model, the marginal posterior distribution of the finite dimensional parameter of interest is expected to be asymptotically equivalent to the sampling distribution of frequentist's efficient estimators. This is the assertion of the so-called Bernstein-von Mises theorem, and recently, it has been proved in many interesting semiparametric models. In this thesis, we consider the semiparametric Bernstein-von Mises theorem in some models which have symmetric errors. The simplest example of these models is the symmetric location model that has 1-dimensional location parame…
In unsupervised classification, Hidden Markov Models (HMM) are used to account for a neighborhood structure between observations. The emission distributions are often supposed to belong to some parametric family. In this paper, a semiparametric modeling where the emission distributions are a mixture of parametric distributions is proposed to get a higher flexibility. We show that the classical EM algorithm can be adapted to infer the model parameters. For the initialisation step, starting from a large number of components, a hierarchical method to combine them into the hidden states is propose…
In this paper, we propose new semiparametric procedures for making inference on linear functionals and their functions of two semicontinuous populations. The distribution of each population is usually characterized by a mixture of a discrete point mass at zero and a continuous skewed positive component, and hence such distribution is semicontinuous in the nature. To utilize the information from both populations, we model the positive components of the two mixture distributions via a semiparametric density ratio model. Under this model setup, we construct the maximum empirical likelihood estima…
In the framework of censored data modeling, the classical linear regression model that assumes normally distributed random errors has received increasing attention in recent years, mainly for mathematical and computational convenience. However, practical studies have often criticized this linear regression model due to its sensitivity to departure from the normality and from the partial nonlinearity. This paper proposes to solve these potential issues simultaneously in the context of the partial linear regression model by assuming that the random errors follow a scale-mixture of normal (SMN) f…
We consider the problem of estimation of a shift parameter of an unknown symmetric function in Gaussian white noise. We introduce a notion of semiparametric second-order efficiency and propose estimators that are semiparametrically efficient and second-order efficient in our model. These estimators are of a penalized maximum likelihood type with an appropriately chosen penalty. We argue that second-order efficiency is crucial in semiparametric problems since only the second-order terms in asymptotic expansion for the risk account for the behavior of the ``nonparametric component'' of a semipar…
Technion - Israel Institute of Technology（以色列理工学院）；Massachusetts Institute of Technology（麻省理工学院）
This paper studies the problem of estimating the means $\pmθ_{*}\in\mathbb{R}^{d}$ of a symmetric two-component Gaussian mixture $δ_{*}\cdot N(θ_{*},I)+(1-δ_{*})\cdot N(-θ_{*},I)$ where the weights $δ_{*}$ and $1-δ_{*}$ are unequal. Assuming that $δ_{*}$ is known, we show that the population version of the EM algorithm globally converges if the initial estimate has non-negative inner product with the mean of the larger weight component. This can be achieved by the trivial initialization $θ_{0}=0$. For the empirical iteration based on $n$ samples, we show that when initialized at $θ_{0}=0$, the…
Robust and semiparametric statistics are of the same historical origin and largely employ the same locally asymptotically normal framework. In our talk, we consider he following more intrinsic connections of both fields: 1) Robust influence curves for semiparametric models with infinite dimensional nuisance parameter; for example, for semiparametric regression (Cox), and mixture models (Neyman--Scott). 2) Adaptiveness in the sense of Stein's necessary condition of robust neighborhood models and estimators with respect to a finite dimensional nuisance parameter; for example, location, linear re…
This paper studies the sparse normal mean models under the empirical Bayes framework. We focus on the mixture priors with an atom at zero and a density component centered at a data driven location determined by maximizing the marginal likelihood or minimizing the Stein Unbiased Risk Estimate. We study the properties of the corresponding posterior median and posterior mean. In particular, the posterior median is a thresholding rule and enjoys the multi-direction shrinkage property that shrinks the observation toward either the origin or the data-driven location. The idea is extended by consider…
A method that uses order statistics to construct multivariate distributions with fixed marginals and which utilizes a representation of the Bernstein copula in terms of a finite mixture distribution is proposed. Expectation-maximization (EM) algorithms to estimate the Bernstein copula are proposed, and a local convergence property is proved. Moreover, asymptotic properties of the proposed semiparametric estimators are provided. Illustrative examples are presented using three real data sets and a 3-dimensional simulated data set. These studies show that the Bernstein copula is able to represent…
The Gini index is a popular inequality measure with many applications in social and economic studies. This paper studies semiparametric inference on the Gini indices of two semicontinuous populations. We characterize the distribution of each semicontinuous population by a mixture of a discrete point mass at zero and a continuous skewed positive component. A semiparametric density ratio model is then employed to link the positive components of the two distributions. We propose the maximum empirical likelihood estimators of the two Gini indices and their difference, and further investigate the a…
The inverse Gaussian (IG) is one of the most famous and considered distributions with positive support. We propose a convenient mode-based parameterization yielding the reparametrized IG (rIG) distribution; it allows/simplifies the use of the IG distribution in various statistical fields, and we give some examples in nonparametric statistics, robust statistics, and model-based clustering. In nonparametric statistics, we define a smoother based on rIG kernels. By construction, the estimator is well-defined and free of boundary bias. We adopt likelihood cross-validation to select the smoothing p…
We consider in this paper a contamined regression model where the distribution of the contaminating component is known when the Eu- clidean parameters of the regression model, the noise distribution, the contamination ratio and the distribution of the design data are un- known. Our model is said to be semiparametric in the sense that the probability density function (pdf) of the noise involved in the regression model is not supposed to belong to a parametric density family. When the pdf's of the noise and the contaminating phenomenon are supposed to be symmetric about zero, we propose an estim…
We consider the estimation of the location of the pole and memory parameter, λ^0 and α, respectively, of covariance stationary linear processes whose spectral density function f(λ) satisfies f(λ)\sim C| λ-λ^0| ^{-α} in a neighborhood of λ^0. We define a consistent estimator of λ^0 and derive its limit distribution Z_{λ^0}. As in related optimization problems, when the true parameter value can lie on the boundary of the parameter space, we show that Z_{λ^0} is distributed as a normal random variable when λ^0\in (0,π), whereas for λ^0=0 or π, Z_{λ^0} is a mixture of discrete and continuous rando…
This article discusses the problem of estimation of parameters in finite mixtures when the mixture components are assumed to be symmetric and to come from the same location family. We refer to these mixtures as semi-parametric because no additional assumptions other than symmetry are made regarding the parametric form of the component distributions. Because the class of symmetric distributions is so broad, identifiability of parameters is a major issue in these mixtures. We develop a notion of identifiability of finite mixture models, which we call k-identifiability, where k denotes the number…
In this article, we present some specific aspects of symmetric Gamma process mixtures for use in regression models. We propose a new Gibbs sampler for simulating the posterior and we establish adaptive posterior rates of convergence related to the Gaussian mean regression problem.
Consistency for constrained maximum likelihood estimation and clustering based on mixtures of elliptically-symmetric distributions under general data generating processes
In observational studies designed to estimate the effects of interventions or exposures, such as cigarette smoking, it is desirable to try to control background differences between the treated group (e.g., current smokers) and the control group (e.g., never smokers) on covariates $X$ (e.g., age, education). Matched sampling attempts to effect this control by selecting subsets of the treated and control groups with similar distributions of such covariates. This paper examines the consequences of matching using affinely invariant methods when the covariate distributions are ``discriminant mixtur…
In this article, we revisit the problem of estimating the unknown zero-symmetric distribution in a two-component location mixture model, considered in previous works, now under the assumption that the zero-symmetric distribution has a log-concave density. When consistent estimators for the shift locations and mixing probability are used, we show that the nonparametric log-concave Maximum Likelihood estimator (MLE) of both the mixed density and that of the unknown zero-symmetric component are consistent in the Hellinger distance. In case the estimators for the shift locations and mixing probabi…
