Second-order variational type equations for spatial point processes are established. In case of log linear parametric models for pair correlation functions, it is demonstrated that the variational equations can be applied to construct estimating equations with closed form solutions for the parameter estimates. This result is used to fit orthogonal series expansions of log pair correlation functions of general form.
We propose a new variational Bayes estimator for high-dimensional copulas with discrete, or a combination of discrete and continuous, margins. The method is based on a variational approximation to a tractable augmented posterior, and is faster than previous likelihood-based approaches. We use it to estimate drawable vine copulas for univariate and multivariate Markov ordinal and mixed time series. These have dimension $rT$, where $T$ is the number of observations and $r$ is the number of series, and are difficult to estimate using previous methods. The vine pair-copulas are carefully selected …
We introduce a new variational estimator for the intensity function of an inhomogeneous spatial point process with points in the $d$-dimensional Euclidean space and observed within a bounded region. The variational estimator applies in a simple and general setting when the intensity function is assumed to be of log-linear form $β+{θ}^{\top}z(u)$ where $z$ is a spatial covariate function and the focus is on estimating ${θ}$. The variational estimator is very simple to implement and quicker than alternative estimation procedures. We establish its strong consistency and asymptotic normality. We a…
Non-linear hierarchical models are commonly used in many disciplines. However, inference in the presence of non-nested effects and on large datasets is challenging and computationally burdensome. This paper provides two contributions to scalable and accurate inference. First, I derive a new mean-field variational algorithm for estimating binomial logistic hierarchical models with an arbitrary number of non-nested random effects. Second, I propose "marginally augmented variational Bayes" (MAVB) that further improves the initial approximation through a step of Bayesian post-processing. I prove t…
Ising models originated in statistical physics and are widely used in modeling spatial data and computer vision problems. However, statistical inference of this model remains challenging due to intractable nature of the normalizing constant in the likelihood. Here, we use a pseudo-likelihood instead to study the Bayesian estimation of two-parameter, inverse temperature, and magnetization, Ising model with a fully specified coupling matrix. We develop a computationally efficient variational Bayes procedure for model estimation. Under the Gaussian mean-field variational family, we derive posteri…
We develop approximate estimation methods for exponential random graph models (ERGMs), whose likelihood is proportional to an intractable normalizing constant. The usual approach approximates this constant with Monte Carlo simulations, however convergence may be exponentially slow. We propose a deterministic method, based on a variational mean-field approximation of the ERGM's normalizing constant. We compute lower and upper bounds for the approximation error for any network size, adapting nonlinear large deviations results. This translates into bounds on the distance between true likelihood a…
We present a unifying view on various statistical estimation techniques including penalization, variational and thresholding methods. These estimators will be analyzed in the context of statistical linear inverse problems including nonparametric and change point regression, and high dimensional linear models as examples. Our approach reveals many seemingly unrelated estimation schemes as special instances of a general class of variational multiscale estimators, named MIND (MultIscale Nemirovskii--Dantzig). These estimators result from minimizing certain regularization functionals under convex …
Efficient Online Variational Estimation via Monte Carlo Sampling
Télécom Paris, Institut Polytechnique de Paris（巴黎电信学院，巴黎理工学院）；Université Bretagne Sud, UMR CNRS 6205, LMBA（南布列塔尼大学，法国国家科学研究中心联合研究单位6205，数学与建模实验室）；Sorbonne Université, Université Paris Cité, CNRS, Laboratoire de Probabilités, Statistique et Modélisation, LPSM（索邦大学，巴黎西岱大学，法国国家科学研究中心，概率、统计与建模实验室）
Causal Effect Estimation using identifiable Variational AutoEncoder with Latent Confounders and Post-Treatment Variables
College of Informatics, Huazhong Agricultural University（华中农业大学信息学院）；Data 61, Commonwealth Scientific and Industrial Research Organisation (CSIRO)（联邦科学与工业研究组织数据61分部）；STEM, University of South Australia（南澳大学 STEM 学院）
Mixture model-based clustering has become an increasingly popular data analysis technique since its introduction over fifty years ago, and is now commonly utilized within a family setting. Families of mixture models arise when the component parameters, usually the component covariance (or scale) matrices, are decomposed and a number of constraints are imposed. Within the family setting, model selection involves choosing the member of the family, i.e., the appropriate covariance structure, in addition to the number of mixture components. To date, the Bayesian information criterion (BIC) has pro…
Linear regression with measurement error in the covariates is a heavily studied topic, however, the statistics/econometrics literature is almost silent to estimating a multi-equation model with measurement error. This paper considers a seemingly unrelated regression model with measurement error in the covariates and introduces two novel estimation methods: a pure Bayesian algorithm (based on Markov chain Monte Carlo techniques) and its mean field variational Bayes (MFVB) approximation. The MFVB method has the added advantage of being computationally fast and can handle big data. An issue perti…
Mixture models are widely used in Bayesian statistics and machine learning, in particular in computational biology, natural language processing and many other fields. Variational inference, a technique for approximating intractable posteriors thanks to optimization algorithms, is extremely popular in practice when dealing with complex models such as mixtures. The contribution of this paper is two-fold. First, we study the concentration of variational approximations of posteriors, which is still an open problem for general mixtures, and we derive consistency and rates of convergence. We also ta…
Dynamic factor models are often estimated by point-estimation methods, disregarding parameter uncertainty. We propose a method accounting for parameter uncertainty by means of posterior approximation, using variational inference. Our approach allows for any arbitrary pattern of missing data, including different sample sizes and mixed frequencies. It also yields a straight-forward estimation algorithm absent of time-consuming simulation techniques. In empirical examples using both small and large models, we compare our method to full Bayesian estimation from MCMC-simulations. Generally, the app…
