Spectral subsampling MCMC was recently proposed to speed up Markov chain Monte Carlo (MCMC) for long stationary univariate time series by subsampling periodogram observations in the frequency domain. This article extends the approach to multivariate time series using a multivariate generalisation of the Whittle likelihood. To assess the computational gains from spectral subsampling in challenging problems, a multivariate generalisation of the autoregressive tempered fractionally integrated moving average model (ARTFIMA) is introduced and some of its properties derived. Bayesian inference based on the Whittle likelihood is demonstrated to be a fast and accurate alternative to the exact time domain likelihood. Spectral subsampling is shown to provide up to two orders of magnitude additional speed-up, while retaining MCMC sampling efficiency and accuracy, compared to spectral methods using the full dataset.
Keywords: Bayesian, Markov chain Monte Carlo, Semi-long memory, Spectral analysis, Whittle likelihood.
Spectral subsampling MCMC for Lévy-driven continuous-time ARMA models with expensive likelihood contributions
Human Technology Institute, University of Technology Sydney(悉尼科技大学人类技术研究所);School of Mathematical and Physical Sciences, University of Technology Sydney(悉尼科技大学数理科学学院);School of Economics, University of New South Wales(新南威尔士大学经济管理学院);Guanghua School of Management, Peking University(北京大学光华管理学院)
Stabilised weighted data subsampling for accelerated Bayesian inference in models with recursive likelihoods
University of Technology Sydney(悉尼科技大学);University of New South Wales(新南威尔士大学);Human Technology Institute, University of Technology Sydney(悉尼科技大学人类技术研究所)
Improving Sampling Accuracy of Stochastic Gradient MCMC Methods via Non-uniform Subsampling of Gradients
Georgia Institute of Technology(佐治亚理工学院);Google Inc.(谷歌公司);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))
许多马尔可夫链蒙特卡罗(MCMC)方法利用目标分布的势函数的梯度信息来有效地探索样本空间。然而,计算梯度对于大规模应用(诸如当代机器学习中的那些应用)而言通常在计算上是昂贵的。随机梯度(SG-)MCMC方法通常通过均匀子采样的数据点通过随机梯度来近似梯度,并且实现了提高的计算效率,但是以引入采样误差为代价。我们提出了一种非均匀子采样方案,以提高采样精度。提出的指数加权随机梯度(EWSG)的设计,使一个非均匀SG-MCMC方法模仿的批量梯度MCMC方法的统计行为,因此,由于SG近似的不准确性减少。EWSG不同于经典的方差缩减(VR)技术,因为它关注的是整个分布而不仅仅是方差;尽管如此,它的局部方差也被证明了。EWSG也可以被看作是重要性采样思想的扩展,成功的随机梯度为基础的优化,采样任务。在EWSG的实际实现中,非均匀子采样通过数据索引上的Metropolis-Hastings链有效地执行,该链与MCMC算法耦合。数值实验不仅证明了EWSG的有效性,而且还指导超参数的选择,并验证我们的\emph{non-asymptotic global error bound},尽管在实现中的近似。值得注意的是,虽然统计精度得到了提高,但收敛速度可以与统一版本相媲美,这使得EWSG成为VR的实用替代方案(但EWSG和VR也可以组合)。