Random discrete distributions, say $F,$ known as species sampling models, represent a rich class of models for classification and clustering, in Bayesian statistics and machine learning. They also arise in various areas of probability and its applications. Jim Pitman, within the species sampling context, shows that mixed Poisson processes may be interpreted as the sample size up till a given time or in terms of waiting times of appearance of individuals to be classified. He notes connections to some recent work in the Bayesian statistic/machine learning literature, with some more classical results. We let $F:=F_{α,θ},$ be a Pitman-Yor process for $α\in (0,1),$ and $θ>-α,$ with $α$-diversity equivalent in distribution to $S^{-α}_{α,θ},$ and let $(N_{S_{α,θ}}(λ),λ\ge 0)$ denote a mixed Poisson process with rate $S_{α,θ}.$ In this paper we derive explicit stick-breaking representations of $F_{α,θ}$ given $N_{S_{α,θ}}(λ)=m.$ More precisely, if $(P_{\ell})\sim \mathrm{PD}(α,θ)$, denotes a ranked sequence following the two parameter Poisson-Dirichlet distribution, we obtain explicit representations of the sized biased permutation of $(P_{\ell})|N_{S_{α,θ}}(λ)=m.$ Due to distributional results we shall develop in a more general context, it suffices to consider the stable case $F_{α,0}|N_{S_α}(λ)=m.$ Notably, it follows that $F_{α,0}|N_{S_α}(λ)=0,$ is equivalent in distribution to the popular normalized generalized gamma process. Hence, we obtain explicit stick-breaking representations for the generalized gamma class as a special case.
For a long time, the Dirichlet process has been the gold standard discrete random measure in Bayesian nonparametrics. The Pitman--Yor process provides a simple and mathematically tractable generalization, allowing for a very flexible control of the clustering behaviour. Two commonly used representations of the Pitman--Yor process are the stick-breaking process and the Chinese restaurant process. The former is a constructive representation of the process which turns out very handy for practical implementation, while the latter describes the partition distribution induced. However, the usual pro…
The Pitman-Yor process is a random discrete probability distribution of which the atoms can be used to model the relative abundance of species. The process is indexed by a type parameter $σ$, which controls the number of different species in a finite sample from a realization of the distribution. A random sample of size $n$ from the Pitman-Yor process of type $σ>0$ will contain of the order $n^σ$ distinct values (``species''). In this paper we consider the estimation of the type parameter by both empirical Bayes and full Bayes methods. We derive the asymptotic normality of the empirical Bayes …
We consider Bayesian nonparametric density estimation using a Pitman-Yor or a normalized inverse-Gaussian process kernel mixture as the prior distribution for a density. The procedure is studied from a frequentist perspective. Using the stick-breaking representation of the Pitman-Yor process or the expression of the finite-dimensional distributions for the normalized-inverse Gaussian process, we prove that, when the data are replicates from an infinitely smooth density, the posterior distribution concentrates on any shrinking $L^p$-norm ball, $1\leq p\leq\infty$, around the sampling density at…
The Pitman-Yor process is a random probability distribution, that can be used as a prior distribution in a nonparametric Bayesian analysis. The process is of species sampling type and generates discrete distributions, which yield of the order $n^σ$ different values ("species") in a random sample of size $n$, if the type $σ$ is positive. Thus this type parameter can be set to target true distributions of various levels of discreteness, making the Pitman-Yor process an interesting prior in this case. It was previously shown that the resulting posterior distribution is consistent if and only if t…
In many applications, a finite mixture is a natural model, but it can be difficult to choose an appropriate number of components. To circumvent this choice, investigators are increasingly turning to Dirichlet process mixtures (DPMs), and Pitman-Yor process mixtures (PYMs), more generally. While these models may be well-suited for Bayesian density estimation, many investigators are using them for inferences about the number of components, by considering the posterior on the number of components represented in the observed data. We show that this posterior is not consistent --- that is, on data …
In Bayesian nonparametrics there exists a rich variety of discrete priors, including the Dirichlet process and its generalizations, which are nowadays well-established tools. Despite the remarkable advances, few proposals are tailored for modeling observations lying on product spaces, such as $\mathbb{R}^p$. Indeed, for multivariate random measures, most available priors lack flexibility and do not allow for separate partition structures among the spaces. We introduce a discrete nonparametric prior, termed enriched Pitman-Yor process (EPY), aimed at addressing these issues. Theoretical propert…
We introduce the Pitman Yor Diffusion Tree (PYDT) for hierarchical clustering, a generalization of the Dirichlet Diffusion Tree (Neal, 2001) which removes the restriction to binary branching structure. The generative process is described and shown to result in an exchangeable distribution over data points. We prove some theoretical properties of the model and then present two inference methods: a collapsed MCMC sampler which allows us to model uncertainty over tree structures, and a computationally efficient greedy Bayesian EM search algorithm. Both algorithms use message passing on the tree s…
The Australian National University（澳大利亚国立大学）；Data61/NICTA；Monash University（莫纳什大学）；Duke University（杜克大学）
The Dirichlet process and its extension, the Pitman-Yor process, are stochastic processes that take probability distributions as a parameter. These processes can be stacked up to form a hierarchical nonparametric Bayesian model. In this article, we present efficient methods for the use of these processes in this hierarchical context, and apply them to latent variable models for text analytics. In particular, we propose a general framework for designing these Bayesian models, which are called topic models in the computer science community. We then propose a specific nonparametric Bayesian topic…
In this paper we consider approximations to the popular Pitman-Yor process obtained by truncating the stick-breaking representation. The truncation is determined by a random stopping rule that achieves an almost sure control on the approximation error in total variation distance. We derive the asymptotic distribution of the random truncation point as the approximation error epsilon goes to zero in terms of a polynomially tilted positive stable distribution. The practical usefulness and effectiveness of this theoretical result is demonstrated by devising a sampling algorithm to approximate func…
Discrete random probability measures and the exchangeable random partitions they induce are key tools for addressing a variety of estimation and prediction problems in Bayesian inference. Indeed, many popular nonparametric priors, such as the Dirichlet and the Pitman-Yor process priors, select discrete probability distributions almost surely and, therefore, automatically induce exchangeable random partitions. Here we focus on the family of Gibbs-type priors, a recent and elegant generalization of the Dirichlet and the Pitman-Yor process priors. These random probability measures share propertie…
Prior distributions play a crucial role in Bayesian approaches to clustering. Two commonly-used prior distributions are the Dirichlet and Pitman-Yor processes. In this paper, we investigate the predictive probabilities that underlie these processes, and the implicit "rich-get-richer" characteristic of the resulting partitions. We explore an alternative prior for nonparametric Bayesian clustering -- the uniform process -- for applications where the "rich-get-richer" property is undesirable. We also explore the cost of this process: partitions are no longer exchangeable with respect to the order…
This article constructs a class of random probability measures based on exponentially and polynomially tilting operated on the laws of completely random measures. The class is proved to be conjugate in that it covers both prior and posterior random probability measures in the Bayesian sense. Moreover, the class includes some common and widely used random probability measures, the normalized completely random measures (James (Poisson process partition calculus with applications to exchangeable models and Bayesian nonparametrics (2002) Preprint), Regazzini, Lijoi and Prünster (Ann. Statist. 31 (…
Algorithms based on spectral graph cut objectives such as normalized cuts, ratio cuts and ratio association have become popular in recent years because they are widely applicable and simple to implement via standard eigenvector computations. Despite strong performance for a number of clustering tasks, spectral graph cut algorithms still suffer from several limitations: first, they require the number of clusters to be known in advance, but this information is often unknown a priori; second, they tend to produce clusters with uniform sizes. In some cases, the true clusters exhibit a known size d…
This paper introduces a general class of hierarchical nonparametric prior distributions. The random probability measures are constructed by a hierarchy of generalized species sampling processes with possibly non-diffuse base measures. The proposed framework provides a general probabilistic foundation for hierarchical random measures with either atomic or mixed base measures and allows for studying their properties, such as the distribution of the marginal and total number of clusters. We show that hierarchical species sampling models have a Chinese Restaurants Franchise representation and can …
Recent advances in Bayesian models for random partitions have led to the formulation and exploration of Exchangeable Sequences of Clusters (ESC) models. Under ESC models, it is the cluster sizes that are exchangeable, rather than the observations themselves. This property is particularly useful for obtaining microclustering behavior, whereby cluster sizes grow sublinearly in the number of observations, as is common in applications such as record linkage, sparse networks and genomics. Unfortunately, the exchangeable clusters property comes at the cost of projectivity. As a consequence, in contr…
Gibbs partition models are the largest class of infinite exchangeable partitions of the positive integers generalizing the product form of the probability function of the two-parameter Poisson-Dirichlet family. Recently those models have been investigated in a Bayesian nonparametric approach to species sampling problems as alternatives to the Dirichlet and the Pitman-Yor process priors. Here we derive marginals of conditional and unconditional multivariate distributions arising from exchangeable Gibbs partitions to obtain explicit formulas for joint falling factorial moments of corresponding c…
Most generative models for clustering implicitly assume that the number of data points in each cluster grows linearly with the total number of data points. Finite mixture models, Dirichlet process mixture models, and Pitman--Yor process mixture models make this assumption, as do all other infinitely exchangeable clustering models. However, for some applications, this assumption is inappropriate. For example, when performing entity resolution, the size of each cluster should be unrelated to the size of the data set, and each cluster should contain a negligible fraction of the total number of da…
We investigate a class of feature allocation models that generalize the Indian buffet process and are parameterized by Gibbs-type random measures. Two existing classes are contained as special cases: the original two-parameter Indian buffet process, corresponding to the Dirichlet process, and the stable (or three-parameter) Indian buffet process, corresponding to the Pitman--Yor process. Asymptotic behavior of the Gibbs-type partitions, such as power laws holding for the number of latent clusters, translates into analogous characteristics for this class of Gibbs-type feature allocation models.…
We investigate the class of $σ$-stable Poisson-Kingman random probability measures (RPMs) in the context of Bayesian nonparametric mixture modeling. This is a large class of discrete RPMs which encompasses most of the the popular discrete RPMs used in Bayesian nonparametrics, such as the Dirichlet process, Pitman-Yor process, the normalized inverse Gaussian process and the normalized generalized Gamma process. We show how certain sampling properties and marginal characterizations of $σ$-stable Poisson-Kingman RPMs can be usefully exploited for devising a Markov chain Monte Carlo (MCMC) algorit…
Most generative models for clustering implicitly assume that the number of data points in each cluster grows linearly with the total number of data points. Finite mixture models, Dirichlet process mixture models, and Pitman--Yor process mixture models make this assumption, as do all other infinitely exchangeable clustering models. However, for some tasks, this assumption is undesirable. For example, when performing entity resolution, the size of each cluster is often unrelated to the size of the data set. Consequently, each cluster contains a negligible fraction of the total number of data poi…
This paper develops some general calculus for GGC and Dirichlet process means functionals. It then proceeds via an investigation of positive Linnik random variables, and more generally random variables derived from compositions of a stable subordinator with GGC subordinators, to establish various distributional equivalences between these models and phenomena connected to local times and occupation times of what are defined as randomly skewed Bessel processes and bridges. This yields a host of interesting identities and explicit density formula for these models. Randomly skewed Bessel processes…
There is a very rich literature proposing Bayesian approaches for clustering starting with a prior probability distribution on partitions. Most approaches assume exchangeability, leading to simple representations in terms of Exchangeable Partition Probability Functions (EPPF). Gibbs-type priors encompass a broad class of such cases, including Dirichlet and Pitman-Yor processes. Even though there have been some proposals to relax the exchangeability assumption, allowing covariate-dependence and partial exchangeability, limited consideration has been given on how to include concrete prior knowle…
Imperial College London（帝国理工学院）；Cyprus University of Technology（塞浦路斯理工大学）
Generalized autoregressive conditional heteroscedasticity (GARCH) models have long been considered as one of the most successful families of approaches for volatility modeling in financial return series. In this paper, we propose an alternative approach based on methodologies widely used in the field of statistical machine learning. Specifically, we propose a novel nonparametric Bayesian mixture of Gaussian process regression models, each component of which models the noise variance process that contaminates the observed data as a separate latent Gaussian process driven by the observed data. T…
In this paper, we propose a new proof of the Jensen formula in 1895. We also derive some formulas similar to those in Pitman and Yor, 2003. Besides, a new formula of the generalized Bernoulli function is also derived. At the end of the paper, the probability density functions of sinh and tanh are studied briefly for general cases.
Most of applied statistics involves regression analysis of data. This paper presents a stand-alone and menu-driven software package, Bayesian Regression: Nonparametric and Parametric Models. Currently, this package gives the user a choice from 83 Bayesian models for data analysis. They include 47 Bayesian nonparametric (BNP) infinite-mixture regression models; 5 BNP infinite-mixture models for density estimation; and 31 normal random effects models (HLMs), including normal linear models. Each of the 78 regression models handles either a continuous, binary, or ordinal dependent variable, and ca…
We introduce a new class of nonparametric prior distributions on the space of continuously varying densities, induced by Dirichlet process mixtures which diffuse in time. These select time-indexed random functions without jumps, whose sections are continuous or discrete distributions depending on the choice of kernel. The construction exploits the widely used stick-breaking representation of the Dirichlet process and induces the time dependence by replacing the stick-breaking components with one-dimensional Wright-Fisher diffusions. These features combine appealing properties of the model, inh…
We describe the combinatorial stochastic process underlying a sequence of conditionally independent Bernoulli processes with a shared beta process hazard measure. As shown by Thibaux and Jordan [TJ07], in the special case when the underlying beta process has a constant concentration function and a finite and nonatomic mean, the combinatorial structure is that of the Indian buffet process (IBP) introduced by Griffiths and Ghahramani [GG05]. By reinterpreting the beta process introduced by Hjort [Hjo90] as a measurable family of Dirichlet processes, we obtain a simple predictive rule for the gen…
