马尔可夫链样本流的距离度量
Distances for Markov chains from sample streams
AI总结:
本文提出一种基于样本流的随机优化方法,通过新的线性规划表述和随机原始-对偶算法估计马尔可夫链的双模拟度量,无需显式转移模型,并提供样本复杂度保证和实证验证。
摘要:
双模拟度量是衡量随机过程(特别是马尔可夫链)之间相似性的强大工具。近期进展揭示,双模拟度量实际上是最优传输距离,这促进了快速算法的发展,用于计算此类度量并具有可证明的准确性和运行时间保证。然而,这些近期方法以及所有先前已知的方法都假设完全了解转移动态。在大多数现实场景中,这一假设往往不切实际,因为通常只有样本轨迹可用。在本工作中,我们提出了一种随机优化方法,以解决这一局限性,并基于样本访问估计双模拟度量,无需显式转移模型。我们的方法源自双模拟度量的新线性规划(LP)表述,我们使用随机原始-对偶优化方法求解。我们提供了算法样本复杂度的理论保证,并通过一系列实证评估验证了其有效性。
英文摘要:
Bisimulation metrics are powerful tools for measuring similarities between stochastic processes, and specifically Markov chains. Recent advances have uncovered that bisimulation metrics are, in fact, optimal-transport distances, which has enabled the development of fast algorithms for computing such metrics with provable accuracy and runtime guarantees. However, these recent methods, as well as all previously known methods, assume full knowledge of the transition dynamics. This is often an impractical assumption in most real-world scenarios, where typically only sample trajectories are available. In this work, we propose a stochastic optimization method that addresses this limitation and estimates bisimulation metrics based on sample access, without requiring explicit transition models. Our approach is derived from a new linear programming (LP) formulation of bisimulation metrics, which we solve using a stochastic primal-dual optimization method. We provide theoretical guarantees on the sample complexity of the algorithm and validate its effectiveness through a series of empirical evaluations.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。