The aim of this paper is to study different estimation procedures based on $φ-$divergences. The dual representation of $φ-$divergences based on the Fenchel-Legendre duality is the main interest of this study. It provides a way to estimate $φ-$divergences by a simple plug-in of the empirical distribution without any smoothing technique. Resulting estimators are thoroughly studied theoretically and with simulations showing that the so called minimum $φ-$divergence estimator (MD$φ$DE) is generally non robust and behaves similarly to the maximum likelihood estimator. We give some arguments supporting the non robustness property, and give insights on how to modify the classical approach. An alternative class of $φ-$divergences robust estimators based on the dual representation is presented. We study consistency and robustness properties from an influence function point of view of the new estimators. In a second part, we invoke the Basu-Lindsay approach for approximating $φ-$divergences and provide a comparison between these approaches. The so called dual $φ-$divergence is also discussed and compared to our new estimator. A full simulation study of all these approaches is given in order to compare efficiency and robustness of all mentioned estimators against the so-called minimum density power divergence, showing encouraging results in favor of our new class of minimum dual $φ-$divergences.
Empirical phi-divergence test-statistics have demostrated to be a useful technique for the simple null hypothesis to improve the finite sample behaviour of the classical likelihood ratio test-statistic, as well asfor model misspecification problems, in both cases for the one population problem. This paper introduces this methodology for two sample problems. A simulation study illustrates situations in which the new test-statistics become a competitive tool with respect to the classical z-test and the likelihood ratio test-statistic.
In this paper we apply divergence measures to empirical likelihood applied to logistic regression models. We define a family of empirical test statistics based on divergence measures, called empirical phi-divergence test statistics, extending the empirical likelihood ratio test. We study the asymptotic distribution of these empirical test statistics, showing that it is the same for all the test statistics in this family, and the same as the classical empirical likelihood ratio test. Next, we study the power function for the members in this family, showing that the empirical phi-divergence test…
The main purpose of this paper is to introduce first a new family of empirical test statistics for testing a simple null hypothesis when the vector of parameters of interest are defined through a specific set of unbiased estimating functions. This family of test statistics is based on a distance between two probability vectors, with the first probability vector obtained by maximizing the empirical likelihood on the vector of parameters, and the second vector defined from the fixed vector of parameters under the simple null hypothesis. The distance considered for this purpose is the phi-diverge…
In this paper we are interested in testing whether there are any signals hidden in high dimensional noise data. Therefore we study the family of goodness-of-fit tests based on $Φ$-divergences including the test of Berk and Jones as well as Tukey's higher criticism test. The optimality of this family is already known for the heterogeneous normal mixture model. We now present a technique to transfer this optimality to more general models. For illustration we apply our results to dense signal and sparse signal models including the exponential-$χ^2$ mixture model and general exponential families a…
A new family of minimum distance estimators for binary logistic regression models based on $ϕ$-divergence measures is introduced. The so called "pseudo minimum phi-divergence estimator"(PM$ϕ$E) family is presented as an extension of "minimum phi-divergence estimator" (M$ϕ$E) for general sample survey designs and contains, as a particular case, the pseudo maximum likelihood estimator (PMLE) considered in Roberts et al. \cite{r}. Through a simulation study it is shown that some PM$ϕ$Es have a better behaviour, in terms of efficiency, than the PMLE.
In information geometry, generalized exponential families and statistical manifolds with curvature are under active investigation in recent years. In this paper we consider the statistical manifold induced by a logarithmic $L^{(α)}$-divergence which generalizes the Bregman divergence. It is known that such a manifold is dually projectively flat with constant negative sectional curvature, and is closely related to the $\mathcal{F}^{(α)}$-family, a generalized exponential family introduced by the second author. Our main result constructs a dual foliation of the statistical manifold, i.e., an ort…
This article develops the theoretical framework needed to study the multinomial logistic regression model for complex sample design with pseudo minimum phi-divergence estimators. Through a numerical example and simulation study new estimators are proposed for the parameter of the logistic regression model with overdispersed multinomial distributions for the response variables, the pseudo minimum Cressie-Read divergence estimators, as well as new estimators for the intra-cluster correlation coefficient. The results show that the Binder's method for the intra-cluster correlation coefficient exhi…
In the field of statistics, many kind of divergence functions have been studied as an amount which measures the discrepancy between two probability distributions. In the differential geometrical approach in statistics (information geometry), dually flat spaces play a key role. In a dually flat space, there exist dual affine coordinate systems and strictly convex functions called potential and a canonical divergence is naturally introduced as a function of the affine coordinates and potentials. The canonical divergence satisfies a relational expression called triangular relation. This can be re…
In Econometrics, imposing restrictions without assuming underlying distributions to modelize complex realities is a valuable methodological tool. However, if a subset of restrictions were not correctly specified, the usual test-statistics for correctly specified models tend to reject erronously a simple null hypothesis. In this setting, we may say that the model suffers from misspecification. We study the behavior of empirical phi-divergence test-statistics, introduced in Balakrishnan et al. (2015), by using the exponential tilted empirical likelihood estimators of Schennach (2007), as a good …
We derive independence tests by means of dependence measures thresholding in a semiparametric context. Precisely, estimates of phi-mutual informations, associated to phi-divergences between a joint distribution and the product distribution of its margins, are derived through the dual representation of phi-divergences. The asymptotic properties of the proposed estimates are established, including consistency, asymptotic distributions and large deviations principle. The obtained tests of independence are compared via their relative asymptotic Bahadur efficiency and numerical simulations. It foll…
The study of mixture models constitutes a large domain of research in statistics. In the first part of this work, we present phi-divergences and the existing methods which produce robust estimators. We are more particularly interested in the so-called dual formula of phi-divergences. We build a new robust estimator based on this formula. We study its asymptotic properties and give a numerical comparison with existing methods on simulated data. We also introduce a proximal-point algorithm whose aim is to calculate divergence-based estimators. We give some of the convergence properties of this a…
Phi-Divergence test statistics for testing the validity of latent class models for binary data
本文的主要目的是提出新的检验统计量族，用于研究某些数据对二进制数据的潜在类模型的拟合优度问题。引入的检验统计量族是基于φ-散度测度的，φ-散度测度是极大似然的自然扩展。我们还处理的问题，测试嵌套序列的二进制数据的潜在类模型。对于这些统计量，我们得到了它们的渐近分布。我们将考虑Felipe et al（2014）中介绍的用于解决估计问题的相合估计量。最后，进行了模拟研究，以比较的效率，在这个意义上的水平和权力，本文考虑的新的统计量的样本量是不够大，适用于渐近结果。
We study the logarithmic $L^{(α)}$-divergence which extrapolates the Bregman divergence and corresponds to solutions to novel optimal transport problems. We show that this logarithmic divergence is equivalent to a conformal transformation of the Bregman divergence, and, via an explicit affine immersion, is equivalent to Kurose's geometric divergence. In particular, the $L^{(α)}$-divergence is a canonical divergence of a statistical manifold with constant sectional curvature $-α$. For such a manifold, we give a geometric interpretation of its sectional curvature in terms of how the divergence b…
Optimal transport and information geometry both study geometric structures on spaces of probability distributions. Optimal transport characterizes the cost-minimizing movement from one distribution to another, while information geometry originates from coordinate-invariant properties of statistical inference. Their connections and applications in statistics and machine learning have started to gain more attention. In this paper we give a new differential geometric connection between the two fields. Namely, the pseudo-Riemannian framework of Kim and McCann, a geometric perspective on the fundam…
This article proposes copula-based dependence quantification between multiple groups of random variables of possibly different sizes via the family of $Phi$-divergences. An axiomatic framework for this purpose is provided, after which we focus on the absolutely continuous setting assuming copula densities exist. We consider parametric and semi-parametric frameworks, discuss estimation procedures, and report on asymptotic properties of the proposed estimators. In particular, we first concentrate on a Gaussian copula approach yielding explicit and attractive dependence coefficients for specific …
