可扩展的最大准团挖掘：一种并行系统协同设计方法
Scalable Mining of Maximal Quasi-Cliques: An Algorithm-System Codesign Approach
摘要：
给定用户指定的最小度阈值$γ$，$γ$准团是子图$g=(V_g,E_g)$，其中每个顶点$v\in V_g$连接到其他顶点的至少$γ$部分（即，$g$中的$\lceil γ\cdot(|V_g|-1)\rceil$顶点）。准集团是稠密结构的最自然的定义之一，可用于发现社交网络中的社区和发现重要的生物分子结构和途径。然而，挖掘最大准集团是出了名的昂贵。在本文中，我们设计了并行算法挖掘最大准集团的G思想家，最近的分布式框架，针对分治图挖掘算法，挖掘到计算密集型任务，充分利用CPU核心。然而，我们发现直接使用G-thinker会导致掉队者问题，这是由于（i）不同任务之间的负载不平衡以及（ii）难以预测任务运行时间和时间增长与任务子图大小。我们通过重新设计G-thinker的执行引擎来解决这些挑战，以优先考虑长期运行的任务进行挖掘，并利用一种新的超时策略来有效地分解长期运行任务的挖掘工作负载，以提高负载均衡。虽然这个系统的重新设计适用于许多其他昂贵的密集子图挖掘问题，本文验证了这一想法，适应国家的最先进的准集团算法，快速，我们重新设计的G思想。我们通过集成新的修剪规则来改进Quick，并修复一些可能导致遗漏结果的遗漏边界情况。大量的实验验证了我们的新解决方案可以很好地扩展CPU核心的数量，实现201 $\times$运行时的加速时，挖掘一个图与3.77 M顶点和16.5 M边在16个节点的集群。
英文摘要：
Given a user-specified minimum degree threshold $γ$, a $γ$-quasi-clique is a subgraph $g=(V_g,E_g)$ where each vertex $v\in V_g$ connects to at least $γ$ fraction of the other vertices (i.e., $\lceil γ\cdot(|V_g|-1)\rceil$ vertices) in $g$. Quasi-clique is one of the most natural definitions for dense structures useful in finding communities in social networks and discovering significant biomolecule structures and pathways. However, mining maximal quasi-cliques is notoriously expensive. In this paper, we design parallel algorithms for mining maximal quasi-cliques on G-thinker, a recent distributed framework targeting divide-and-conquer graph mining algorithms that decomposes the mining into compute-intensive tasks to fully utilize CPU cores. However, we found that directly using G-thinker results in the straggler problem due to (i) the drastic load imbalance among different tasks and (ii) the difficulty of predicting the task running time and the time growth with task-subgraph size. We address these challenges by redesigning G-thinker's execution engine to prioritize long-running tasks for mining, and by utilizing a novel timeout strategy to effectively decompose the mining workloads of long-running tasks to improve load balancing. While this system redesign applies to many other expensive dense subgraph mining problems, this paper verifies the idea by adapting the state-of-the-art quasi-clique algorithm, Quick, to our redesigned G-thinker. We improve Quick by integrating new pruning rules, and fixing some missed boundary cases that could lead to missed results. Extensive experiments verify that our new solution scales well with the number of CPU cores, achieving 201$\times$ runtime speedup when mining a graph with 3.77M vertices and 16.5M edges in a 16-node cluster.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
