arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.07987stat.MEstat.ML
收藏

通过Q-指数过程进行贝叶斯学习

Bayesian Learning via Q-Exponential Process

Shuyi Li, Michael O'Connor, Shiwei Lan

AI总结:

本文提出Q-指数过程(Q-EP),一种对应L_q正则化的随机过程,作为贝叶斯先验,比高斯过程具有更尖锐的惩罚,在函数建模、图像重建和逆问题中表现优于GP和Besov过程。

摘要:

正则化是优化、统计和机器学习中最基础的主题之一。为了在估计参数$u\in\mathbb{R}^d$时获得稀疏性,通常会在目标函数中添加一个$\ell_q$惩罚项$\Vert u\Vert_q$。与这种$\ell_q$惩罚相对应的概率分布是什么?当我们对函数$u\in L^q$进行建模时,与$\Vert u\Vert_q$相对应的正确随机过程是什么?这对于统计建模高维对象(例如图像)非常重要,通过惩罚来保留某些属性(例如图像中的边缘)。在这项工作中,我们将$q$-指数分布(密度正比于$\exp{(- \frac{1}{2}|u|^q)}$)推广到一个名为$Q$-指数(Q-EP)过程的随机过程,该过程对应于函数的$L_q$正则化。关键步骤是通过从一大类椭圆等高线分布中选择,来指定一致的多元$q$-指数分布。这项工作与通常由展开级数定义的Besov过程密切相关。Q-EP可以被视为Besov过程的一种定义,具有显式的概率公式和对相关长度的直接控制。从贝叶斯的角度来看,Q-EP为函数提供了一个灵活的先验,其惩罚比常用的高斯过程(GP)更尖锐($q<2$)。我们在建模函数数据、重建图像和解决逆问题方面比较了GP、Besov和Q-EP,并展示了我们提出的方法的优势。

英文摘要:

Regularization is one of the most fundamental topics in optimization, statistics and machine learning. To get sparsity in estimating a parameter $u\in\mathbb{R}^d$, an $\ell_q$ penalty term, $\Vert u\Vert_q$, is usually added to the objective function. What is the probabilistic distribution corresponding to such $\ell_q$ penalty? What is the correct stochastic process corresponding to $\Vert u\Vert_q$ when we model functions $u\in L^q$? This is important for statistically modeling large dimensional objects, e.g. images, with penalty to preserve certainty properties, e.g. edges in the image. In this work, we generalize the $q$-exponential distribution (with density proportional to) $\exp{(- \frac{1}{2}|u|^q)}$ to a stochastic process named $Q$-exponential (Q-EP) process that corresponds to the $L_q$ regularization of functions. The key step is to specify consistent multivariate $q$-exponential distributions by choosing from a large family of elliptic contour distributions. The work is closely related to Besov process which is usually defined by the expanded series. Q-EP can be regarded as a definition of Besov process with explicit probabilistic formulation and direct control on the correlation length. From the Bayesian perspective, Q-EP provides a flexible prior on functions with sharper penalty ($q<2$) than the commonly used Gaussian process (GP). We compare GP, Besov and Q-EP in modeling functional data, reconstructing images, and solving inverse problems and demonstrate the advantage of our proposed methodology.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑