归一化常数估计的复杂性分析：从Jarzynski等式到退火重要性采样及其进一步发展
Complexity Analysis of Normalizing Constant Estimation: from Jarzynski Equality to Annealed Importance Sampling and beyond
AI总结：
本文研究了归一化常数估计问题，提出了一种非渐近分析方法，推导了退火重要性采样估计归一化常数的复杂度，并提出了一种新的算法以处理多模态问题。
摘要：
给定一个未归一化的概率密度π∝e^{-V}，估计其归一化常数Z=∫_{R^d}e^{-V(x)}dx或自由能F=-log Z是贝叶斯统计、统计力学和机器学习中的关键问题。尤其是在高维或π多模态时，这变得尤为具有挑战性。为了减轻传统重要性采样估计器的高方差，采用基于退火的方法如Jarzynski等式和退火重要性采样是常见的选择，但其定量复杂度保证仍很少被探索。我们朝着退火重要性采样的非渐近分析迈出第一步。特别是，我们推导出一个oracle复杂度为~O(dβ²A²/ε⁴)的复杂度，用于在高概率下估计Z的ε相对误差。其中，β是V的光滑度，A表示一个插值π和可处理参考分布的概率测度曲线的动作。我们的分析利用Girsanov定理和最优传输，不需要显式要求目标分布的等周假设。最后，为了处理广泛使用的几何插值的大动作，我们提出了一种基于反扩散采样器的新算法，建立了分析其复杂度的框架，并通过实验证明其在处理多模态问题中的效率。
英文摘要：
Given an unnormalized probability density $π\propto\mathrm{e}^{-V}$, estimating its normalizing constant $Z=\int_{\mathbb{R}^d}\mathrm{e}^{-V(x)}\mathrm{d}x$ or free energy $F=-\log Z$ is a crucial problem in Bayesian statistics, statistical mechanics, and machine learning. It is challenging especially in high dimensions or when $π$ is multimodal. To mitigate the high variance of conventional importance sampling estimators, annealing-based methods such as Jarzynski equality and annealed importance sampling are commonly adopted, yet their quantitative complexity guarantees remain largely unexplored. We take a first step toward a non-asymptotic analysis of annealed importance sampling. In particular, we derive an oracle complexity of $\widetilde{O}\left(\frac{dβ^2{\mathcal{A}}^2}{\varepsilon^4}\right)$ for estimating $Z$ within $\varepsilon$ relative error with high probability, where $β$ is the smoothness of $V$ and $\mathcal{A}$ denotes the action of a curve of probability measures interpolating $π$ and a tractable reference distribution. Our analysis, leveraging Girsanov's theorem and optimal transport, does not explicitly require isoperimetric assumptions on the target distribution. Finally, to tackle the large action of the widely used geometric interpolation, we propose a new algorithm based on reverse diffusion samplers, establish a framework for analyzing its complexity, and empirically demonstrate its efficiency in tackling multimodality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
