Free exponential families have been previously introduced as a special case of the q-exponential family. We show that free exponential families arise also from a procedure analogous to the definition of exponential families by using the Cauchy-Stieltjes kernel instead of the exponential kernel. We use this approach to re-derive several known results and to study further similarities with exponential families and reproductive exponential models.
This paper continues the study of a kernel family which uses the Cauchy-Stieltjes kernel in place of the celebrated exponential kernel of the exponential families theory. We extend the theory to cover generating measures with support that is unbounded on one side. We illustrate the need for such an extension by showing that cubic pseudo-variance functions correspond to free-infinitely divisible laws without the first moment. We also determine the domain of means, advancing the understanding of Cauchy-Stieltjes kernel families also for compactly supported generating measures.
RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）；The University of Tokyo（东京大学）
In this paper, we give a method to construct "good" exponential families systematically by representation theory. More precisely, we consider a homogeneous space $G/H$ as a sample space and construct an exponential family invariant under the transformation group $G$ by using a representation of $G$. The method generates widely used exponential families such as normal, gamma, Bernoulli, categorical, Wishart, von Mises, Fisher-Bingham and hyperboloid distributions.
We explore properties of Cauchy-Stieltjes families that have no counterpart in exponential families. We relate the variance function of the iterated Cauchy-Stieltjes family to the pseudo-variance function of the initial Cauchy-Stieltjes family. We also investigate when the domain of means can be extended beyond the "natural domain".
Let G = An be the graph corresponding to the graphical model of nearest neighbour interaction in a Gaussian character. We study Natural Exponential Families( NEF) ofWishart distributions on convex cones QG and PG, where PG is the cone of positive definite real symmetric matrices with obligatory zeros prescribed by G, and QG is the dual cone of PG. The Wishart NEF that we construct include Wishart distributions considered earlier by Lauritzen (1996) and Letac and Massam (2007) for models based on decomposable graphs. Our approach is however different and allows us to study the basic objects ofW…
In this paper we specify some facts about the sequence of polynomials associated to a \CSK family and we prove that quadratic variance function is characterized by the property of orthogonality of these polynomials.
We prove the large deviation principle (LDP) for posterior distributions arising from subfamilies of full exponential families, allowing misspecification of the model. Moreover, motivated by the so-called inverse Sanov Theorem (see e.g. Ganesh and O'Connell 1999 and 2000), we prove the LDP for the corresponding maximum likelihood estimator, and we study the relationship between rate functions. In our setting, even in the non misspecified case, it is not true in general that the rate functions for posterior distributions and for maximum likelihood estimators are Kullback-Leibler divergences wit…
We characterize the existence of the maximum likelihood estimator for discrete exponential families. Our criterion is simple to apply as we show in various settings, most notably for exponential models of random graphs. As an application, we point out the size of independent identically distributed samples for which the maximum likelihood estimator exists with high probability.
In this paper, we determine the effect of the free multiplicative convolution on the pseudo-variance function of a Cauchy-Stieltjes kernel family. We then use the machinery of variance functions to establish some limit theorems related to this type of convolution and involving the free additive convolution and the boolean additive convolution.
Consider a measure $μ$ on $\R^n$ generating a natural exponential family $F(μ)$ with variance function $V_{F(μ)}(m)$ and Laplace transform $$ \exp(\ell_μ(s))=\int_{\R^n} \exp(-\<s,x\>)μ(dx).$$ A dual measure $μ^*$ satisfies $-\ell'_{μ^*}(-\ell'_μ(s))=s.$ Such a dual measure does not always exist. One important property is $\ell"_{μ^*}(m)=(V_{F(μ)}(m))^{-1},$ leading to the notion of duality among exponential families (or rather among the extended notion of T exponential families $T\hskip-2pt F$ obtained by considering all translations of a given exponential family $F$).
Chentsov's theorem characterizes the Fisher information metric on statistical models as essentially the only Riemannian metric that is invariant under sufficient statistics. This implies that each statistical model is naturally equipped with a geometry, so Chentsov's theorem explains why many statistical properties can be described in geometric terms. However, despite being one of the foundational theorems of statistics, Chentsov's theorem has only been proved previously in very restricted settings or under relatively strong regularity and invariance assumptions. We therefore prove a version o…
For a natural exponential family (NEF), one can associate in a natural way two standard families of conjugate priors, one on the natural parameter and the other on the mean parameter. These families of conjugate priors have been used to establish some remarkable properties and characterization results of the quadratic NEF's. In the present paper, we show that for a NEF, we can associate a class of NEF's, and for each one of these NEF's, we define a family of conjugate priors on the natural parameter and a family of conjugate priors on the mean parameter which are different of the standard ones…
This paper studies variance functions of Cauchy-Stieltjes Kernel families generated by compactly supported centered probability measures. We describe several operations that allow us to construct additional variance functions from known ones. We construct a class of examples which exhausts all cubic variance functions, and provide examples of polynomial variance functions of arbitrary degree. We also relate Cauchy-Stieltjes Kernel families with polynomial variance functions to generalized orthogonality. Our main results are stated solely in terms of classical probability; some proofs rely on a…
This paper studies a class of exponential family models whose canonical parameters are specified as linear functionals of an unknown infinite-dimensional slope function. The optimal minimax rates of convergence for slope function estimation are established. The estimators that achieve the optimal rates are constructed by constrained maximum likelihood estimation with parameters whose dimension grows with sample size. A change-of-measure argument, inspired by Le Cam's theory of asymptotic equivalence, is used to eliminate the bias caused by the non-linearity of exponential family models.
This paper studies a class of exponential family models whose canonical parameters are specified as linear functionals of an unknown infinite-dimensional slope function. The optimal minimax rates of convergence for slope function estimation are established. The estimators that achieve the optimal rates are constructed by constrained maximum likelihood estimation with parameters whose dimension grows with sample size. A change-of-measure argument, inspired by Le Cam's theory of asymptotic equivalence, is used to eliminate the bias caused by the nonlinearity of exponential family models.
Let $P$ a probability on the real line generating a natural exponential family $(P_t)_{t\in \R}$. We show that $t$ is a median of $P_t$ for all $t$ only if $P$ is the standard Gaussian law $N(0.1).$ The proof is based on the Choquet Deny equation.
In this paper we present various new inequalities for tail proabilities for distributions that are elements of the most improtant exponential families. These families include the Poisson distributions, the Gamma distributions, the binomial distributions, the negative binomial distributions and the inverse Gaussian distributions. All these exponential families have simple variance functions and the variance functions play an important role in the exposition. All the inequalities presented in this paper are formulated in terms of the signed log-likelihood. The inequalities are of a qualitative n…
This paper introduces a notion of 2-orthogonality for a sequence of polynomials to give extended versions of the Meixner and Feinsilver characterization results based on orthogonal polynomials. These new versions subsume the Letac-Mora characterization of the real natural exponential families having cubic variance function.
We consider natural and general exponential families $(Q_m)_{m\in M}$ on $\mathbb{R}^d$ parametrized by the means. We study the submodels $(Q_{θm_1+(1-θ)m_2})_{θ\in[0,1]}$ parametrized by a segment in the means domain, mainly from the point of view of the Fisher information. Such a parametrization allows for a parsimonious model and is particularly useful in practical situations when hesitating between two parameters $m_1$ and $m_2$. The most interesting examples are obtained when $\mathbb{R}^d$ is a linear space of matrices, in particular for Gaussian and Wishart models.
The variation distance closure of an exponential family with a convex set of canonical parameters is described, assuming no regularity conditions. The tools are the concepts of convex core of a measure and extension of an exponential family, introduced previously by the authors, and a new concept of accessible faces of a convex set. Two other closures related to the information divergence are also characterized.
In this paper, we give three equivalent properties of the class of multivariate simple cubic natural exponential families (NEF's). The first property says that the cumulant function of any basis of the family is a solution of some Monge-Ampére equation, the second is that the variance function satisfies a differential equation, and the third is characterized by the equality between two families of prior distributions related to the NEF. These properties represent the extensions to this class of the properties stated in $\cite{Casalis(1996)}$ and satisfied by the Wishart and the simple quadrati…
This work studies the large sample properties of the posterior-based inference in the curved exponential family under increasing dimension. The curved structure arises from the imposition of various restrictions on the model, such as moment restrictions, and plays a fundamental role in econometrics and others branches of data analysis. We establish conditions under which the posterior distribution is approximately normal, which in turn implies various good properties of estimation and inference procedures based on the posterior. In the process we also revisit and improve upon previous results …
In this paper, we introduce, for a multiplier $χ$, a notion of generalized power function $x\mapsto Δ_χ(x),$ defined on the homogeneous cone ${\mathcal{P}}$ of a Vinberg algebra ${\mathcal{A}}$. We then extend to ${\mathcal{A}}$ the famous Gindikin result, that is we determine the set of multipliers $χ$ such that the map $θ\mapsto Δ_χ(θ^{-1})$, defined on ${\mathcal{P}}^{\ast}$, is the Laplace transform of a positive measure $R_χ$. We also determine the set of $χ$ such that $R_χ$ generates an exponential family, and we calculate the variance function of this family
Asymptotics of the normalizing constant is computed for a class of one parameter exponential families on permutations which includes Mallows model with Spearmans's Footrule and Spearman's Rank Correlation Statistic. The MLE, and a computable approximation of the MLE are shown to be consistent. The pseudo-likelihood estimator of Besag is shown to be $\sqrt{n}$-consistent. An iterative algorithm (IPFP) is proved to converge to the limiting normalizing constant. The Mallows model with Kendall's Tau is also analyzed to demonstrate flexibility of the tools of this paper.
In this paper, we give a simple criterion to verify that functions of the form $e^g$ are in the Hayman class when $g$ is a power series with nonnegative coefficients. Thus, using the Hayman and Báez-Duarte formulas, we obtain asymptotics for the coefficients of generating functions that arise in many examples of set construction in analytic combinatorics. This new criterion greatly simplifies that obtained previously by the authors.
This paper mainly contributes to a classification of statistical Einstein manifolds, namely statistical manifolds at the same time are Einstein manifolds. A statistical manifold is a Riemannian manifold, each of whose points is a probability distribution. With the Fisher information metric as a Riemannian metric, information geometry was developed to understand the intrinsic properties of statistical models, which play important roles in statistical inference, etc. Among all these models, exponential families is one of the most important kinds, whose geometric structures are fully determined b…
The purpose of this paper is to present a recursive algorithm and its implementation in Maple and Mathematica for the computation of joint moments and cumulants of Hawkes processes with exponential kernels. Numerical results and computation times are also discussed. Obtaining closed form expressions can be computationally intensive, as joint fifth cumulant and moment formulas can be respectively expanded into up to 3,288 and 27,116 summands.
We generalize the exponential family of probability distributions. In our approach, the exponential function is replaced by a $φ$-function, resulting in a $φ$-family of probability distributions. We show how $φ$-families are constructed. In a $φ$-family, the analogue of the cumulant-generating function is a normalizing function. We define the $φ$-divergence as the Bregman divergence associated to the normalizing function, providing a generalization of the Kullback-Leibler divergence. A formula for the $φ$-divergence where the $φ$-function is the Kaniadakis' $κ$-exponential function is derived.
Due to its clustering and self-exciting properties, the Hawkes process has been used extensively in numerous fields ranging from sismology to finance. Since data is often aquired on regular time intervals, we propose a piece-wise constant model based on a Discrete-Time Hawkes Process (DTHP). We prove that this discrete-time model converges to the usual continuous-time Hawkes process as the time-step tends to zero.
All one-condition generalized inverses of the Markovian kernel I - P, where P is the transition matrix of a finite irreducible Markov chain, can be uniquely specified in terms of the stationary probabilities and the mean first passage times of the underlying Markov chain. Special sub-families include the group inverse of I - P, Kemeny and Snell's fundamental matrix of the Markov chain and the Moore-Penrose g-inverse. The elements of some sub-families of the generalized inverses can also be re-expressed involving the second moments of the recurrence time variables. Some applications to Kemeny's…
