具有各向异性尾自适应流的胖尾变分推断
Fat-Tailed Variational Inference with Anisotropic Tail Adaptive Flows
摘要:
虽然厚尾密度通常在稳健模型和尺度混合中作为后验和边缘分布出现,但当基于高斯的变分推断无法准确捕获尾部衰减时,它们会带来挑战。首先,我们改进了以往的理论的尾部Lipschitz流的尾部量化如何影响尾部衰减的速度,并通过扩大理论的非Lipschitz多项式流。然后,我们开发了一个替代理论的多元尾部参数是敏感的尾部各向异性。在这样做的过程中,我们揭示了困扰许多现有的基于流的方法的一个基本问题:它们只能对尾部各向同性分布(即,在每个方向上具有相同尾部参数的分布)。为了减轻这一点,使尾部各向异性目标的建模,我们提出了各向异性尾部自适应流(ATAF)。合成和现实世界的目标上的实验结果证实,ATAF是有竞争力的,与以前的工作,同时也表现出适当的尾部各向异性。
英文摘要:
While fat-tailed densities commonly arise as posterior and marginal distributions in robust models and scale mixtures, they present challenges when Gaussian-based variational inference fails to capture tail decay accurately. We first improve previous theory on tails of Lipschitz flows by quantifying how the tails affect the rate of tail decay and by expanding the theory to non-Lipschitz polynomial flows. Then, we develop an alternative theory for multivariate tail parameters which is sensitive to tail-anisotropy. In doing so, we unveil a fundamental problem which plagues many existing flow-based methods: they can only model tail-isotropic distributions (i.e., distributions having the same tail parameter in every direction). To mitigate this and enable modeling of tail-anisotropic targets, we propose anisotropic tail-adaptive flows (ATAF). Experimental results on both synthetic and real-world targets confirm that ATAF is competitive with prior work while also exhibiting appropriate tail-anisotropy.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。