HUGE:一个高效可扩展的子图枚举系统
HUGE: An Efficient and Scalable Subgraph Enumeration System
摘要:
子图枚举是图分析中的一个基本问题,其目的是在大型数据图上找到给定查询图的所有实例。在本文中,我们提出了一个系统,称为巨大的有效地处理子图枚举的规模在分布式上下文中。HUGE的特点是:1)一个优化器,可以在不受现有工作限制的情况下计算高级执行计划; 2)一个混合通信层,支持推送和拉取通信; 3)一个新颖的两阶段执行模式,具有无锁和零拷贝缓存设计,4)一个BFS/DFS自适应调度器,以限制内存消耗,以及5)两层机器内和机器间负载均衡。HUGE是通用的,所有现有的分布式子图枚举算法都可以插入,以享受自动加速和有限内存执行。
英文摘要:
Subgraph enumeration is a fundamental problem in graph analytics, which aims to find all instances of a given query graph on a large data graph. In this paper, we propose a system called HUGE to efficiently process subgraph enumeration at scale in the distributed context. HUGE features 1) an optimiser to compute an advanced execution plan without the constraints of existing works; 2) a hybrid communication layer that supports both pushing and pulling communication; 3) a novel two-stage execution mode with a lock-free and zero-copy cache design, 4) a BFS/DFS-adaptive scheduler to bound memory consumption, and 5) two-layer intra- and inter-machine load balancing. HUGE is generic such that all existing distributed subgraph enumeration algorithms can be plugged in to enjoy automatic speed up and bounded-memory execution.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。