Group-invariant probability distributions appear in many data-generative models in machine learning, such as graphs, point clouds, and images. In practice, one often needs to estimate divergences between such distributions. In this work, we study how the inherent invariances, with respect to any smooth action of a Lie group on a manifold, improve sample complexity when estimating the 1-Wasserstein distance, the Sobolev Integral Probability Metrics (Sobolev IPMs), the Maximum Mean Discrepancy (MMD), and also the complexity of the density estimation problem (in the $L^2$ and $L^\infty$ distance). Our results indicate a two-fold gain: (1) reducing the sample complexity by a multiplicative factor corresponding to the group size (for finite groups) or the normalized volume of the quotient space (for groups of positive dimension); (2) improving the exponent in the convergence rate (for groups of positive dimension). These results are completely new for groups of positive dimension and extend recent bounds for finite group actions.
Deep Generative Sampling in the Dual Divergence Space: A Data-efficient & Interpretative Approach for Generative AI
Morgan Stanley(摩根士丹利);NIST(美国国家标准与技术研究院);Georgetown University(乔治城大学);IBM Research(IBM研究院);Mila - Quebec AI Institute(米拉-魁北克人工智能研究所);Université de Montréal(蒙特利尔大学)
Unbiased Estimating Equation on Inverse Divergence and Its Conditions
Media Center Toyohashi University of Technology Toyohashi, Aichi, Japan Email;Department of Computer Science;Engineering Toyohashi University of Technology Toyohashi, Aichi, Japan Email
Risk Bounds for Mixture Density Estimation on Compact Domains via the $h$-Lifted Kullback--Leibler Divergence
School of Mathematics and Physics(数学与物理学院);The University of Queensland(昆士兰大学);Institute of Mathematics for Industry(产业数学研究所);Kyushu University(九州大学);School of Computing, Engineering, and Mathematical Sciences(计算、工程与数学科学学院);La Trobe University(拉筹伯大学)
我们考虑的问题,估计概率密度函数的样本数据的基础上,使用有限的混合密度从一些组件类。为此,我们引入了$h$ -提升Kullback-Leibler(KL)分歧作为标准库尔贝克-莱布勒散度的推广和进行风险最小化的标准。在紧支撑假设下,我们证明了当使用$h$ -提升的库尔贝克-莱布勒散度时,期望估计误差的$\mathcal{O}(1/{\sqrt{n}})$界,推广了Rakhlin等人的结果.(2005年,ESAIM:概率与统计,第9卷)和李和巴伦(1999,Advances in Neural Information ProcessingSystems,Vol.12),以允许不是严格正的密度函数的风险边界。我们开发了一个程序的计算相应的最大$h$ -提升似然估计($h$ -MLLE)使用的优化最大化框架,并提供实验结果支持我们的理论界。