arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2003.00722cs.LGcs.AIstat.ML
收藏

批量平稳分布估计

Batch Stationary Distribution Estimation

University of Alberta(阿尔伯塔大学);Google Research(谷歌研究院)

Junfeng Wen, Bo Dai, Lihong Li, Dale Schuurmans

摘要:

考虑了给定一组抽样转移的遍历马尔可夫链的平稳分布的近似问题。经典的基于模拟的方法假设访问底层过程,以便可以收集足够长度的轨迹来近似静态采样。相反,我们考虑一种替代设置,其中预先通过单独的,可能未知的过程收集了一组固定的转换。我们的目标仍然是估计平稳分布的属性,但没有额外的访问底层系统。我们提出了一个一致的估计,是基于恢复的校正比函数在给定的数据。特别是,我们开发了一个变分幂方法(VPM),在一般条件下提供可证明一致的估计。除了统一一些现有的方法,从不同的子领域,我们还发现,VPM产生显着更好的估计,在一系列的问题,包括建模,随机微分方程,后处理MCMC,和关闭政策评估。

英文摘要:

We consider the problem of approximating the stationary distribution of an ergodic Markov chain given a set of sampled transitions. Classical simulation-based approaches assume access to the underlying process so that trajectories of sufficient length can be gathered to approximate stationary sampling. Instead, we consider an alternative setting where a fixed set of transitions has been collected beforehand, by a separate, possibly unknown procedure. The goal is still to estimate properties of the stationary distribution, but without additional access to the underlying system. We propose a consistent estimator that is based on recovering a correction ratio function over the given data. In particular, we develop a variational power method (VPM) that provides provably consistent estimates under general conditions. In addition to unifying a number of existing approaches from different subfields, we also find that VPM yields significantly better estimates across a range of problems, including queueing, stochastic differential equations, post-processing MCMC, and off-policy evaluation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑