大数据分析中的信息分割
Information Splitting for Big Data Analytics
AI总结：
本文提出信息分割技术，简化大数据集的计算，使线性混合模型适用于大规模数据集，通过矩阵代数变换实现更高效的计算。
摘要：
许多统计模型需要估计未知（协）方差参数，通常通过最大化对数似然函数来获得，涉及对数行列式项。理论上，需要观测信息（负Hessian矩阵或对数似然的二阶导数）以获得准确的最大似然估计。当使用Fisher信息（观测信息的期望值）时，可以得到比牛顿法更简单的Fisher评分算法。随着生物科学、推荐系统和社会网络中高通量技术的发展，数据集的规模以及相应的统计模型突然增加了多个数量级。观测信息和Fisher信息对于这些大数据集难以获得。本文引入信息分割技术以简化计算。在分割观测信息和Fisher信息的均值后，可以得到一个更简单的对数似然的近似Hessian矩阵。这种近似Hessian矩阵可以显著减少计算量，并使线性混合模型适用于大数据集。这种分割和更简单的公式严重依赖于矩阵代数变换，并适用于大规模育种模型和基因组广义关联分析。
英文摘要：
Many statistical models require an estimation of unknown (co)-variance parameter(s) in a model. The estimation usually obtained by maximizing a log-likelihood which involves log determinant terms. In principle, one requires the \emph{observed information}--the negative Hessian matrix or the second derivative of the log-likelihood---to obtain an accurate maximum likelihood estimator according to the Newton method. When one uses the \emph{Fisher information}, the expect value of the observed information, a simpler algorithm than the Newton method is obtained as the Fisher scoring algorithm. With the advance in high-throughput technologies in the biological sciences, recommendation systems and social networks, the sizes of data sets---and the corresponding statistical models---have suddenly increased by several orders of magnitude. Neither the observed information nor the Fisher information is easy to obtained for these big data sets. This paper introduces an information splitting technique to simplify the computation. After splitting the mean of the observed information and the Fisher information, an simpler approximate Hessian matrix for the log-likelihood can be obtained. This approximated Hessian matrix can significantly reduce computations, and makes the linear mixed model applicable for big data sets. Such a spitting and simpler formulas heavily depends on matrix algebra transforms, and applicable to large scale breeding model, genetics wide association analysis.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
