The main purpose of this paper is to introduce and study the behavior of minimum ϕ-divergence estimators as an alternative to the maximum likelihood estimator in latent class models for binary items. As it will become clear below, minimum ϕ-divergence estimators are a natural extension of the maximum likelihood estimator. The asymptotic properties of minimum ϕ-divergence estimators for latent class models for binary data are developed. Finally, to compare the efficiency and robustness of these new estimators with those obtained through maximum likelihood when the sample size is not big enough to apply the asymptotic results, we have carried out a simulation study.
Phi-Divergence test statistics for testing the validity of latent class models for binary data
本文的主要目的是提出新的检验统计量族，用于研究某些数据对二进制数据的潜在类模型的拟合优度问题。引入的检验统计量族是基于φ-散度测度的，φ-散度测度是极大似然的自然扩展。我们还处理的问题，测试嵌套序列的二进制数据的潜在类模型。对于这些统计量，我们得到了它们的渐近分布。我们将考虑Felipe et al（2014）中介绍的用于解决估计问题的相合估计量。最后，进行了模拟研究，以比较的效率，在这个意义上的水平和权力，本文考虑的新的统计量的样本量是不够大，适用于渐近结果。
This paper presents objective priors for robust Bayesian estimation against outliers based on divergences. The minimum $γ$-divergence estimator is well-known to work well estimation against heavy contamination. The robust Bayesian methods by using quasi-posterior distributions based on divergences have been also proposed in recent years. In objective Bayesian framework, the selection of default prior distributions under such quasi-posterior distributions is an important problem. In this study, we provide some properties of reference and moment matching priors under the quasi-posterior distribu…
The learning and evaluation of energy-based latent variable models (EBLVMs) without any structural assumptions are highly challenging, because the true posteriors and the partition functions in such models are generally intractable. This paper presents variational estimates of the score function and its gradient with respect to the model parameters in a general EBLVM, referred to as VaES and VaGES respectively. The variational posterior is trained to minimize a certain divergence to the true model posterior and the bias in both estimates can be bounded by the divergence theoretically. With a m…
Developing deep generative models that flexibly incorporate diverse measures of probability distance is an important area of research. Here we develop an unified mathematical framework of f-divergence generative model, f-GM, that incorporates both VAE and f-GAN, and enables tractable learning with general f-divergences. f-GM allows the experimenter to flexibly design the f-divergence function without changing the structure of the networks or the learning procedure. f-GM jointly models three components: a generator, a inference network and a density estimator. Therefore it simultaneously enable…
MAZIN Inc.（MAZIN公司）；Nomura Asset Management Co., Ltd.（野村资产管理股份有限公司）
Optimal asset allocation is a key topic in modern finance theory. To realize the optimal asset allocation on investor's risk aversion, various portfolio construction methods have been proposed. Recently, the applications of machine learning are rapidly growing in the area of finance. In this article, we propose the Student's $t$-process latent variable model (TPLVM) to describe non-Gaussian fluctuations of financial timeseries by lower dimensional latent variables. Subsequently, we apply the TPLVM to minimum-variance portfolio as an alternative of existing nonlinear factor models. To test the …
Tsinghua University（清华大学）；Princeton University（普林斯顿大学）；Google（谷歌）；University of Toronto（多伦多大学）
Standard variational lower bounds used to train latent variable models produce biased estimates of most quantities of interest. We introduce an unbiased estimator of the log marginal likelihood and its gradients for latent variable models based on randomized truncation of infinite series. If parameterized by an encoder-decoder architecture, the parameters of the encoder can be optimized to minimize its variance of this estimator. We show that models trained using our estimator give better test-set likelihoods than a standard importance-sampling based approach for the same average computational…
Density-based minimum divergence procedures represent popular techniques in parametric statistical inference. They combine strong robustness properties with high (sometimes full) asymptotic efficiency. Among density-based minimum distance procedures, the methods based on the Bregman-divergence have the attractive property that the empirical formulation of the divergence does not require the use of any non-parametric smoothing technique such as kernel density estimation. The methods based on the density power divergence (DPD) represent the current standard in this area of research. In this pape…
Asymptotic Properties of Minimum S-Divergence Estimator for Discrete Models
基于最小化统计偏差的鲁棒推理已被证明是基于极大似然和相关方法的经典技术的有用替代方案。最近Ghosh et al.（2013）提出了一类广义的散度测度，即S-Divergence族，并通过一些数值例子讨论了它在稳健参数估计中的有用性。在本文中，我们研究了离散模型下所提出的最小S-散度估计量的渐近性质。
In this paper a new family of minimum divergence estimators based on the Bregman divergence is proposed. The popular density power divergence (DPD) class of estimators is a sub-class of Bregman divergences. We propose and study a new sub-class of Bregman divergences called the exponentially weighted divergence (EWD). Like the minimum DPD estimator, the minimum EWD estimator is recognised as an M-estimator. This characterisation is useful while discussing the asymptotic behaviour as well as the robustness properties of this class of estimators. Performances of the two classes are compared -- bo…
The purpose of this paper is to introduce two semiparametric methods for the estimation of copula parameter. These methods are based on minimum Alpha-Divergence between a non-parametric estimation of copula density using local likelihood probit transformation method and a true copula density function. A Monte Carlo study is performed to measure the performance of these methods based on Hellinger distance and Neyman divergence as special cases of Alpha-Divergence. Simulation results are compared to the Maximum Pseudo-Likelihood (MPL) estimation as a conventional estimation method in well-known …
This paper aims at presenting a new application of information geometry to reinforcement learning focusing on dynamic treatment resumes. In a standard framework of reinforcement learning, a Q-function is defined as the conditional expectation of a reward given a state and an action for a single-stage situation. We introduce an equivalence relation, called the policy equivalence, in the space of all the Q-functions. A class of information divergence is defined in the Q-function space for every stage. The main objective is to propose an estimator of the optimal policy function by a method of min…
We consider the problem of robust inference under the generalized linear model (GLM) with stochastic covariates. We derive the properties of the minimum density power divergence estimator of the parameters in GLM with random design and use this estimator to propose robust Wald-type tests for testing any general composite null hypothesis about the GLM. The asymptotic and robustness properties of the proposed tests are also examined for the GLM with random design. Application of the proposed robust inference procedures to the popular Poisson regression model for analyzing count data is discussed…
In this paper a new family of minimum divergence estimators based on the Bregman divergence is proposed, where the defining convex function has an exponential nature. These estimators avoid the necessity of using an intermediate kernel density and many of them also have strong robustness properties. It is further demonstrated that the proposed approach can be extended to construct a class of generalized estimating equations, where the pool of the resultant estimators encompass a large variety of minimum divergence estimators and range from highly robust to fully efficient based on the choice o…
Adaptive LASSO Penalized Minimum Density Power Divergence Estimation through Least Squares Approximation: Application to Bone Mineral Density Data from the SWAN Study
Indian Institute of Technology (Indian School of Mines)（印度理工学院（印度矿业学院））
The panel data regression models have become one of the most widely applied statistical approaches in different fields of research, including social, behavioral, environmental sciences, and econometrics. However, traditional least-squares-based techniques frequently used for panel data models are vulnerable to the adverse effects of the data contamination or outlying observations that may result in biased and inefficient estimates and misleading statistical inference. In this study, we propose a minimum density power divergence estimation procedure for panel data regression models with random …
