Graphyti：一个用于FlashGraph的半外部存储图形库
Graphyti: A Semi-External Memory Graph Library for FlashGraph
摘要：
图形数据集超过了大多数独立机器的内存容量。传统上，图框架通过横向扩展、分布式计算克服了内存限制。新兴的框架避免了分布式数据的网络瓶颈，半外部存储器（SEM）使用单个多核节点并在大于内存的图形上操作。在SEM中，对于具有$n$顶点和$m$边的图，$\mathcal{O}(m)$数据驻留在磁盘上，而$\mathcal{O}(n)$数据驻留在内存中。对于开发人员来说，这增加了复杂性，因为他们必须在应用程序中显式编码I/O。我们提出的原则是至关重要的应用程序开发人员采用，以实现最先进的性能，同时最大限度地减少I/O和内存的算法在SEM。我们在Graphyti中展示了它们，Graphyti是一个构建在FlashGraph上的可扩展并行SEM图形库，可以通过pip在Python中使用。在SEM中，Graphyti实现了80%的内存执行性能，并保留了FlashGraph的性能，这优于分布式引擎，如PowerGraph和Galois。
英文摘要：
Graph datasets exceed the in-memory capacity of most standalone machines. Traditionally, graph frameworks have overcome memory limitations through scale-out, distributing computing. Emerging frameworks avoid the network bottleneck of distributed data with Semi-External Memory (SEM) that uses a single multicore node and operates on graphs larger than memory. In SEM, $\mathcal{O}(m)$ data resides on disk and $\mathcal{O}(n)$ data in memory, for a graph with $n$ vertices and $m$ edges. For developers, this adds complexity because they must explicitly encode I/O within applications. We present principles that are critical for application developers to adopt in order to achieve state-of-the-art performance, while minimizing I/O and memory for algorithms in SEM. We present them in Graphyti, an extensible parallel SEM graph library built on FlashGraph and available in Python via pip. In SEM, Graphyti achieves 80% of the performance of in-memory execution and retains the performance of FlashGraph, which outperforms distributed engines, such as PowerGraph and Galois.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
