MaxEntropy Pursuit Variational Inference

Variational approximation for heteroscedastic linear models and matching pursuit algorithms

Modern statistical applications involving large data sets have focused attention on statistical methodologies which are both efficient computationally and able to deal with the screening of large numbers of different candidate models. Here we consider computationally efficient variational Bayes approaches to inference in high-dimensional heteroscedastic linear regression, where both the mean and variance are described in terms of linear functions of the predictors and where the number of predictors can be larger than the sample size. We derive a closed form variational lower bound on the log m…

面向可解释预测的结合大语言与多模态模型的变分信息追踪

Variational Information Pursuit with Large Language and Multimodal Models for Interpretable Predictions

本研究针对变分信息追踪（V-IP）依赖人工密集概念标注、仅适用于小规模任务的局限，结合基础模型提出FM+V-IP框架，通过LLM生成可解释概念集、多模态模型做语义标注，无需概念过滤即可用更少概念达到有竞争力的可解释预测性能。

用于可解释预测的变分信息追踪

Variational Information Pursuit for Interpretable Predictions

本文提出变分信息追踪（V-IP），通过最小化期望交叉熵直接学习最优查询选择策略和分类器，无需生成模型，在视觉、NLP和医学诊断任务上实现了更快推理、更短查询链和更优性能。

Neural Network Renormalization Group

We present a variational renormalization group (RG) approach using a deep generative model based on normalizing flows. The model performs hierarchical change-of-variables transformations from the physical space to a latent space with reduced mutual information. Conversely, the neural net directly maps independent Gaussian noises to physical configurations following the inverse RG flow. The model has an exact and tractable likelihood, which allows unbiased training and direct access to the renormalized energy function of the latent variables. To train the model, we employ probability density di…

Solving Bayesian Inverse Problems via Variational Autoencoders

In recent years, the field of machine learning has made phenomenal progress in the pursuit of simulating real-world data generation processes. One notable example of such success is the variational autoencoder (VAE). In this work, with a small shift in perspective, we leverage and adapt VAEs for a different purpose: uncertainty quantification in scientific inverse problems. We introduce UQ-VAE: a flexible, adaptive, hybrid data/model-informed framework for training neural networks capable of rapid modelling of the posterior distribution representing the unknown parameter of interest. Specifica…

Dispersed Exponential Family Mixture VAEs for Interpretable Text Generation

Deep generative models are commonly used for generating images and text. Interpretability of these models is one important pursuit, other than the generation quality. Variational auto-encoder (VAE) with Gaussian distribution as prior has been successfully applied in text generation, but it is hard to interpret the meaning of the latent variable. To enhance the controllability and interpretability, one can replace the Gaussian prior with a mixture of Gaussian distributions (GM-VAE), whose mixture components could be related to hidden semantic aspects of data. In this paper, we generalize the pr…