所有距离草图,再访:用于海量图分析的HIP估计器
All-Distances Sketches, Revisited: HIP Estimators for Massive Graphs Analysis
摘要:
具有数十亿条边的图数据集(如社交和Web图)非常普遍,可扩展计算至关重要。全距离草图(ADS)[Cohen 1997],是一个强大的工具,可扩展的近似统计。该草图是强调较近节点的节点的距离关系的小尺寸样本。所有节点的草图使用近似线性计算来计算,并且将估计器应用于节点的草图以估计其属性。我们提供,第一次,ADS算法和应用程序的统一阐述。我们提出了历史逆概率(HIP)估计,适用于一个节点的ADS估计一个大的自然类的统计。对于重要的特殊情况下的邻域基数(在一些查询距离内的节点数)和接近中心,HIP估计最多有一半的方差以前的估计,我们表明,这基本上是最优的。此外,HIP得到了多项式的改进,更一般的统计量和估计是简单的,灵活的,无偏的,和优雅的。对于数据流的近似不同计数,HIP优于HyperLogLog MinHash草图(Flajolet et et al. 2007)的原始估计器,为这种最先进的实用算法获得了显着提高的估计质量。
英文摘要:
Graph datasets with billions of edges, such as social and Web graphs, are prevalent, and scalable computation is critical. All-distances sketches (ADS) [Cohen 1997], are a powerful tool for scalable approximation of statistics. The sketch is a small size sample of the distance relation of a node which emphasizes closer nodes. Sketches for all nodes are computed using a nearly linear computation and estimators are applied to sketches of nodes to estimate their properties. We provide, for the first time, a unified exposition of ADS algorithms and applications. We present the Historic Inverse Probability (HIP) estimators which are applied to the ADS of a node to estimate a large natural class of statistics. For the important special cases of neighborhood cardinalities (the number of nodes within some query distance) and closeness centralities, HIP estimators have at most half the variance of previous estimators and we show that this is essentially optimal. Moreover, HIP obtains a polynomial improvement for more general statistics and the estimators are simple, flexible, unbiased, and elegant. For approximate distinct counting on data streams, HIP outperforms the original estimators for the HyperLogLog MinHash sketches (Flajolet et al. 2007), obtaining significantly improved estimation quality for this state-of-the-art practical algorithm.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。