变分推断中的退火缓解模式崩溃：关于高斯混合物的理论研究
Annealing in variational inference mitigates mode collapse: A theoretical study on Gaussian mixtures
AI总结：
本文通过理论分析证明退火策略能有效缓解变分推断中的模式崩溃问题，并验证其在神经网络模型中的适用性。
摘要：
模式崩溃，即在目标多模分布时无法捕捉一个或多个模式，是现代变分推断中的核心挑战。在本工作中，我们提供了一种基于退火策略的数学分析，用于缓解模式崩溃，这在可处理的设置中是学习高斯混合物时已知会出现模式崩溃的。利用低维的总结统计描述，我们精确地刻画了初始温度和退火率之间的相互作用，并推导出模式崩溃概率的精确公式。我们的分析显示，适当选择的退火方案可以稳健地防止模式崩溃。最后，我们展示了这些理论权衡在基于神经网络的模型（如RealNVP归一化流）中定性地扩展，为设计缓解模式崩溃的退火策略提供指导。
英文摘要：
Mode collapse, the failure to capture one or more modes when targetting a multimodal distribution, is a central challenge in modern variational inference. In this work, we provide a mathematical analysis of annealing based strategies for mitigating mode collapse in a tractable setting: learning a Gaussian mixture, where mode collapse is known to arise. Leveraging a low dimensional summary statistics description, we precisely characterize the interplay between the initial temperature and the annealing rate, and derive a sharp formula for the probability of mode collapse. Our analysis shows that an appropriately chosen annealing scheme can robustly prevent mode collapse. Finally, we present numerical evidence that these theoretical tradeoffs qualitatively extend to neural network based models, RealNVP normalizing flows, providing guidance for designing annealing strategies mitigating mode collapse in practical variational inference pipelines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
