arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1801.05007stat.MEstat.ML
收藏

大规模复杂数据的分割和重组:使用MCMC建模似然函数

Divide and Recombine for Large and Complex Data: Model Likelihood Functions using MCMC

Qi Liu, Anindya Bhadra, William S. Cleveland

摘要:

在Divide & Recombine(D&R)中,大数据被划分为子集,每种分析方法都被应用于子集,并重新组合输出。这使得深入的分析和实际的计算性能。提出了一种新的D\&R方法来计算大数据的数据模型(DM)参数的似然函数。似然模型(LM)是DM参数的参数概率密度函数。通过将密度拟合到由每个子集的DM似然函数绘制的MCMC中来估计密度参数,然后将拟合的密度重新组合。该程序说明使用正常和偏正态LM的逻辑斯谛回归DM。

英文摘要:

In Divide & Recombine (D&R), big data are divided into subsets, each analytic method is applied to subsets, and the outputs are recombined. This enables deep analysis and practical computational performance. An innovate D\&R procedure is proposed to compute likelihood functions of data-model (DM) parameters for big data. The likelihood-model (LM) is a parametric probability density function of the DM parameters. The density parameters are estimated by fitting the density to MCMC draws from each subset DM likelihood function, and then the fitted densities are recombined. The procedure is illustrated using normal and skew-normal LMs for the logistic regression DM.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑