arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1301.5121cs.DBcs.DC
收藏

图数据库分区的定量评价

Partitioning Graph Databases - A Quantitative Evaluation

Alex Averbuch, Martin Neumann

摘要:

电子数据在规模和连通性方面都在以越来越快的速度增长:数据之间的关系越来越多,人们对数据之间的关系越来越感兴趣。一个例子是Twitter社交网络图。由于这种增长,对可以处理这些数据的技术的需求正在增加。目前,关系数据库是主要的技术,但它们不太适合处理连接的数据,因为它们针对索引密集型操作进行了优化。相反,图数据库针对图计算进行了优化。它们通过直接引用来链接记录,避免了索引查找,并且能够在恒定的时间内检索相邻的元素,而不管图的大小。然而,随着数据量的增加,这些数据库的增长超出了一台计算机的资源,数据分区变得必要。我们评估的可行性,使用图分区算法分区图数据库。开发了一个原型分区数据库。三个分区算法探讨和一个实施。使用了三个图形数据集:两个真实的和一个合成生成的。它们以各种方式进行分区,并测量对数据库性能的影响。我们为每个数据集定义了一个合成访问模式,并在分区数据集上执行。评估在模拟环境中进行,确保可重复性,并允许测量网络流量和负载平衡等指标。结果表明,与随机划分相比,划分算法减少了40- 90%的流量。在使用期间间歇地执行该算法保持了分区质量,同时只需要初始分区计算的1%。强相关性的理论质量指标和非均匀访问模式下产生的网络流量。

英文摘要:

Electronic data is growing at increasing rates, in both size and connectivity: the increasing presence of, and interest in, relationships between data. An example is the Twitter social network graph. Due to this growth demand is increasing for technologies that can process such data. Currently relational databases are the predominant technology, but they are poorly suited to processing connected data as they are optimized for index-intensive operations. Conversely, graph databases are optimized for graph computation. They link records by direct references, avoiding index lookups, and enabling retrieval of adjacent elements in constant time, regardless of graph size. However, as data volume increases these databases outgrow the resources of one computer and data partitioning becomes necessary. We evaluate the viability of using graph partitioning algorithms to partition graph databases. A prototype partitioned database was developed. Three partitioning algorithms explored and one implemented. Three graph datasets were used: two real and one synthetically generated. These were partitioned in various ways and the impact on database performance measured. We defined one synthetic access pattern per dataset and executed each on the partitioned datasets. Evaluation took place in a simulation environment, ensuring repeatability and allowing measurement of metrics like network traffic and load balance. Results show that compared to random partitioning the partitioning algorithm reduced traffic by 40-90%. Executing the algorithm intermittently during usage maintained partition quality, while requiring only 1% the computation of initial partitioning. Strong correlations were found between theoretic quality metrics and generated network traffic under non-uniform access patterns.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑