快速鲁棒的分布式子图枚举
Fast and Robust Distributed Subgraph Enumeration
摘要:
研究了分布式环境下经典的子图计数问题。现有的解决方案要么遭受严重的内存危机,要么依赖于大型索引,这使得它们对非常大的图不切实际。它们中的大多数遵循同步模型,其中性能通常由性能最差的机器来检查。基于此,本文提出了一个健壮的异步分布式子图枚举系统RADS。RADS首先识别可以使用单机算法找到的结果。该策略不仅提高了整体性能,而且降低了网络通信和内存开销。此外,RADS采用了一种新的区域分组多轮扩展验证和过滤框架,该框架不需要洗牌和交换中间结果,也不需要在每台机器中复制大部分数据图。这一特性不仅降低了网络通信成本和内存使用量,还允许我们采用简单的策略进行内存控制和负载均衡,使其更加健壮。在RADS中还使用了几种算法来进一步提高性能。我们的实验验证了RADS国家的最先进的子图枚举方法的优越性。
英文摘要:
We study the classic subgraph enumeration problem under distributed settings. Existing solutions either suffer from severe memory crisis or rely on large indexes, which makes them impractical for very large graphs. Most of them follow a synchronous model where the performance is often bottlenecked by the machine with the worst performance. Motivated by this, in this paper, we propose RADS, a Robust Asynchronous Distributed Subgraph enumeration system. RADS first identifies results that can be found using single-machine algorithms. This strategy not only improves the overall performance but also reduces network communication and memory cost. Moreover, RADS employs a novel region-grouped multi-round expand verify & filter framework which does not need to shuffle and exchange the intermediate results, nor does it need to replicate a large part of the data graph in each machine. This feature not only reduces network communication cost and memory usage, but also allows us to adopt simple strategies for memory control and load balancing, making it more robust. Several heuristics are also used in RADS to further improve the performance. Our experiments verified the superiority of RADS to state-of-the-art subgraph enumeration approaches.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。