使用非参数贝叶斯字典学习的联合时空风场数据外推与不确定性量化

贝叶斯非参数因子分析：通过边缘化狄利克雷过程列聚类与尖峰- slab 稀疏性

Bayesian Nonparametric Factor Analysis via Marginalized Dirichlet Process Column Clustering with Spike-and-Slab Sparsity

提出一种边缘化狄利克雷过程列聚类与尖峰- slab稀疏性的贝叶斯非参数因子模型，自适应推断因子数并合并冗余字典，实现最优收缩率，在模拟和乳腺癌数据中优于基线。

The Human Kernel

Bayesian nonparametric models, such as Gaussian processes, provide a compelling framework for automatic statistical modelling: these models have a high degree of flexibility, and automatically calibrated complexity. However, automating human expertise remains elusive; for example, Gaussian processes with standard kernels struggle on function extrapolation problems that are trivial for human learners. In this paper, we create function extrapolation problems and acquire human responses, and then design a kernel learning framework to reverse engineer the inductive biases of human learners across …

外推感知的非参数统计推断

Extrapolation-Aware Nonparametric Statistical Inference

本文扩展非参数统计模型，提出基于观测支撑集内方向导数极值的外推假设与一致界估计方法，用于预测和不确定性量化。

Tensor-Dictionary Learning with Deep Kruskal-Factor Analysis

A multi-way factor analysis model is introduced for tensor-variate data of any order. Each data item is represented as a (sparse) sum of Kruskal decompositions, a Kruskal-factor analysis (KFA). KFA is nonparametric and can infer both the tensor-rank of each dictionary atom and the number of dictionary atoms. The model is adapted for online learning, which allows dictionary learning on large data sets. After KFA is introduced, the model is extended to a deep convolutional tensor-factor analysis, supervised by a Bayesian SVM. The experiments section demonstrates the improvement of KFA over vecto…

Physics Informed Deep Kernel Learning

Deep kernel learning is a promising combination of deep neural networks and nonparametric function learning. However, as a data driven approach, the performance of deep kernel learning can still be restricted by scarce or insufficient data, especially in extrapolation tasks. To address these limitations, we propose Physics Informed Deep Kernel Learning (PI-DKL) that exploits physics knowledge represented by differential equations with latent sources. Specifically, we use the posterior function sample of the Gaussian process as the surrogate for the solution of the differential equation, and co…

Besov regularity of functions with sparse random wavelet coefficients

This paper addresses the problem of regularity properties of functions represented as an expansion in a wavelet basis with random coefficients in terms of finiteness of their Besov norm with probability 1. Such representations are used to specify a prior measure in Bayesian nonparametric wavelet regression. Investigating regularity of such functions is an important problem since the support of the posterior measure does not include functions that are not in the support of the prior measure, and therefore determines the functions that are possible to estimate using specified Bayesian model. We …

Multiscale Dictionary Learning for Estimating Conditional Distributions

Nonparametric estimation of the conditional distribution of a response given high-dimensional features is a challenging problem. It is important to allow not only the mean but also the variance and shape of the response density to change flexibly with features, which are massive-dimensional. We propose a multiscale dictionary learning model, which expresses the conditional response density as a convex combination of dictionary densities, with the densities used and their weights dependent on the path through a tree decomposition of the feature space. A fast graph partitioning algorithm is appl…

Incremental Ensemble Gaussian Processes

Belonging to the family of Bayesian nonparametrics, Gaussian process (GP) based approaches have well-documented merits not only in learning over a rich class of nonlinear functions, but also in quantifying the associated uncertainty. However, most GP methods rely on a single preselected kernel function, which may fall short in characterizing data samples that arrive sequentially in time-critical applications. To enable {\it online} kernel adaptation, the present work advocates an incremental ensemble (IE-) GP framework, where an EGP meta-learner employs an {\it ensemble} of GP learners, each h…

Bayesian Mean-parameterized Nonnegative Binary Matrix Factorization

Binary data matrices can represent many types of data such as social networks, votes, or gene expression. In some cases, the analysis of binary matrices can be tackled with nonnegative matrix factorization (NMF), where the observed data matrix is approximated by the product of two smaller nonnegative matrices. In this context, probabilistic NMF assumes a generative model where the data is usually Bernoulli-distributed. Often, a link function is used to map the factorization to the $[0,1]$ range, ensuring a valid Bernoulli mean parameter. However, link functions have the potential disadvantage …

Adaptive Bayesian credible sets in regression with a Gaussian process prior

We investigate two empirical Bayes methods and a hierarchical Bayes method for adapting the scale of a Gaussian process prior in a nonparametric regression model. We show that all methods lead to a posterior contraction rate that adapts to the smoothness of the true regression function. Furthermore, we show that the corresponding credible sets cover the true regression function whenever this function satisfies a certain extrapolation condition. This condition depends on the specific method, but is implied by a condition of self-similarity. The latter condition is shown to be satisfied with pro…

Structure Discovery in Nonparametric Regression through Compositional Kernel Search

Despite its importance, choosing the structural form of the kernel in nonparametric regression remains a black art. We define a space of kernel structures which are built compositionally by adding and multiplying a small number of base kernels. We present a method for searching over this space of structures which mirrors the scientific discovery process. The learned structures can often decompose functions into interpretable components and enable long-range extrapolation on time-series datasets. Our structure search method outperforms many widely used kernels and kernel combination methods on …