arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2111.07941stat.MLcs.DScs.LGmath.STstat.MEstat.TH
收藏

近线性时间下的分布压缩

Distribution Compression in Near-linear Time

UC Berkeley(加州大学伯克利分校);Harvard University(哈佛大学);MIT(麻省理工学院);Microsoft Research New England(微软新英格兰研究院)

Abhishek Shetty, Raaz Dwivedi, Lester Mackey

摘要:

在分布压缩中,人们的目标是使用少量的代表点精确地概括概率分布$\mathbb{P}$。近似最优细化过程通过从马尔可夫链中采样$n$点并识别具有与$\mathbb{P}$的$\widetilde{\mathcal{O}}(1/\sqrt{n})$差异的$\sqrt{n}$点来实现该目标。不幸的是,这些算法在样本大小$n$中遭受二次或超二次运行时间。为了解决这一不足,我们介绍了压缩++,一个简单的元程序,用于加快任何细化算法,而遭受最多的$4$的错误因素。当与Dwivedi和Mackey(2021)的二次时间核减半和核细化算法相结合时,Compress++在$\mathcal{O}(n \log^3 n)$时间和$\mathcal{O}( \sqrt{n} \log^2 n )$空间中提供具有PDMATHAMR积分误差和优于Monte-Carlo最大平均差异的$\sqrt{n}$点。此外,Compress++在给定任何二次时间输入的情况下都享有相同的近线性运行时间,并将超二次算法的运行时间减少了一个平方根因子。在我们针对具有挑战性的微分方程后验的高维Monte Carlo样本和马尔可夫链的基准测试中,Compress++在数量级更少的时间内匹配或几乎匹配其输入算法的精度。

英文摘要:

In distribution compression, one aims to accurately summarize a probability distribution $\mathbb{P}$ using a small number of representative points. Near-optimal thinning procedures achieve this goal by sampling $n$ points from a Markov chain and identifying $\sqrt{n}$ points with $\widetilde{\mathcal{O}}(1/\sqrt{n})$ discrepancy to $\mathbb{P}$. Unfortunately, these algorithms suffer from quadratic or super-quadratic runtime in the sample size $n$. To address this deficiency, we introduce Compress++, a simple meta-procedure for speeding up any thinning algorithm while suffering at most a factor of $4$ in error. When combined with the quadratic-time kernel halving and kernel thinning algorithms of Dwivedi and Mackey (2021), Compress++ delivers $\sqrt{n}$ points with $\mathcal{O}(\sqrt{\log n/n})$ integration error and better-than-Monte-Carlo maximum mean discrepancy in $\mathcal{O}(n \log^3 n)$ time and $\mathcal{O}( \sqrt{n} \log^2 n )$ space. Moreover, Compress++ enjoys the same near-linear runtime given any quadratic-time input and reduces the runtime of super-quadratic algorithms by a square-root factor. In our benchmarks with high-dimensional Monte Carlo samples and Markov chains targeting challenging differential equation posteriors, Compress++ matches or nearly matches the accuracy of its input algorithm in orders of magnitude less time.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑