互模拟度量即最优传输距离，且可被高效计算
Bisimulation Metrics are Optimal Transport Distances, and Can be Computed Efficiently
AI总结：
本文提出折扣占用耦合与 LP 表述，将马尔可夫链互模拟度量统一为最优传输距离，并通过熵正则化的 SVI 实现快速高效计算。
摘要：
我们提出了一个用于构建马尔可夫链之间最优传输距离的新框架。此前已知的表述研究的是由这些链诱导的整个联合分布之间的耦合，并通过将问题归约为在适当定义的马尔可夫决策过程中进行动态规划（DP）来推导解。然而，这种表述迄今尚未产生特别高效的算法，因为计算相关的 DP 算子需要完整求解一个静态最优传输问题，而且在整个优化过程中需要多次应用这些算子。在本工作中，我们通过考察联合分布的扁平化版本之间的耦合（我们称之为折扣占用耦合）发展了另一种视角，并证明在完整联合分布空间中计算最优传输距离，可以等价表述为在这一约简空间中求解一个线性规划（LP）。这种 LP 表述使我们能够移植最优传输理论其他领域中的若干算法思想。特别地，我们的表述使得可以在优化问题中引入适当的熵正则化概念，进而使我们能够通过一种类 Sinkhorn 方法直接计算最优传输距离，我们将该方法称为 Sinkhorn Value Iteration（SVI）。我们在理论和实验上均表明，该方法会快速收敛到最优耦合，其计算成本本质上与在每一对状态上运行普通 Sinkhorn 相同。在此过程中，我们指出我们的最优传输距离与马尔可夫链之间常见的互模拟度量概念完全一致，因此我们的结果也适用于计算此类度量；事实上，我们的算法被证明显著高于迄今为此目的开发的最佳已知方法的效率。
英文摘要：
We propose a new framework for formulating optimal transport distances between Markov chains. Previously known formulations studied couplings between the entire joint distribution induced by the chains, and derived solutions via a reduction to dynamic programming (DP) in an appropriately defined Markov decision process. This formulation has, however, not led to particularly efficient algorithms so far, since computing the associated DP operators requires fully solving a static optimal transport problem, and these operators need to be applied numerous times during the overall optimization process. In this work, we develop an alternative perspective by considering couplings between a flattened version of the joint distributions that we call discounted occupancy couplings, and show that calculating optimal transport distances in the full space of joint distributions can be equivalently formulated as solving a linear program (LP) in this reduced space. This LP formulation allows us to port several algorithmic ideas from other areas of optimal transport theory. In particular, our formulation makes it possible to introduce an appropriate notion of entropy regularization into the optimization problem, which in turn enables us to directly calculate optimal transport distances via a Sinkhorn-like method we call Sinkhorn Value Iteration (SVI). We show both theoretically and empirically that this method converges quickly to an optimal coupling, essentially at the same computational cost of running vanilla Sinkhorn in each pair of states. Along the way, we point out that our optimal transport distance exactly matches the common notion of bisimulation metrics between Markov chains, and thus our results also apply to computing such metrics, and in fact our algorithm turns out to be significantly more efficient than the best known methods developed so far for this purpose.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
