Monte Carlo algorithms simulate some prescribed number of samples, taking some random real time to complete the computations necessary. This work considers the converse: to impose a real-time budget on the computation, which results in the number of samples simulated being random. To complicate matters, the real time taken for each simulation may depend on the sample produced, so that the samples themselves are not independent of their number, and a length bias with respect to compute time is apparent. This is especially problematic when a Markov chain Monte Carlo (MCMC) algorithm is used and the final state of the Markov chain -- rather than an average over all states -- is required, which is the case in parallel tempering implementations of MCMC. The length bias does not diminish with the compute budget in this case. It also occurs in sequential Monte Carlo (SMC) algorithms, which is the focus of this paper. We propose an anytime framework to address the concern, using a continuous-time Markov jump process to study the progress of the computation in real time. We first show that for any MCMC algorithm, the length bias of the final state's distribution due to the imposed real-time computing budget can be eliminated by using a multiple chain construction. The utility of this construction is then demonstrated on a large-scale SMC^2 implementation, using four billion particles distributed across a cluster of 128 graphics processing units on the Amazon EC2 service. The anytime framework imposes a real-time budget on the MCMC move steps within the SMC$^{2}$ algorithm, ensuring that all processors are simultaneously ready for the resampling step, demonstrably reducing idleness to due waiting times and providing substantial control over the total compute budget.
Optimization Monte Carlo: Efficient and Embarrassingly Parallel Likelihood-Free Inference
Informatics Institute（信息学研究所）；University of Amsterdam（阿姆斯特丹大学）；Donald Bren School of Information and Computer Sciences（唐纳德·布伦信息与计算机科学学院）；University of California, Irvine（加州大学欧文分校）；Canadian Institute for Advanced Research（加拿大高等研究院）
开发有效的MCMC算法是贝叶斯推断中必不可少的。在并行回火中，多个相互作用的MCMC链运行以更有效地探索状态空间并提高性能。多个链通过局部移动独立地前进，并且性能增强步骤是交换移动，其中链暂停以在彼此之间交换它们的当前样本。为了加速独立的本地移动，它们可以在多个处理器上同时执行。然后遇到另一个问题：根据MCMC实现和推理问题，局部移动可能需要不同的随机时间才能完成。也可能存在基础设施引起的变化，例如在云计算中出现的相同处理器上的竞争作业。在交换发生之前，所有的链都必须完成它们所参与的局部移动，以避免引入潜在的实质性偏差（命题2.1）。为了解决多处理器并行回火中随机变化的本地移动完成时间的问题，我们采用Murray et al.（2016）的Anytime蒙特卡罗框架：我们对并行本地移动施加实时截止日期，并在这些截止日期执行交换，而没有任何处理器空闲。我们展示了我们的方法，在实时截止日期的交流不会引入偏见，并导致显着的性能增强，在天真的方法，闲置，直到每个处理器的本地移动完成。该方法，然后应用在ABC设置，任何时候ABC并行回火算法推导出的Lotka-Volterra捕食模型的参数估计的困难任务，并观察到类似的效率提高。
Identifying the Optimal Integration Time in Hamiltonian Monte Carlo
通过利用光滑概率系统的自然几何结构，哈密顿蒙特卡罗产生计算效率高的马尔可夫链蒙特卡罗估计。至少在算法足够好的情况下。在本文中，我将展示如何几何基础的哈密顿蒙特卡罗隐含地确定这些参数的最佳选择，特别是积分时间。然后，我认为这些原则在现有的算法和一个新的实现称为\emph{Exhaustive Hamiltonian Monte Carlo}的实际后果，然后在一些说明性的例子证明这些想法的效用。
