We propose a new framework for computing the embeddings of large-scale graphs on a single machine. A graph embedding is a fixed length vector representation for each node (and/or edge-type) in a graph and has emerged as the de-facto approach to apply modern machine learning on graphs. We identify that current systems for learning the embeddings of large-scale graphs are bottlenecked by data movement, which results in poor resource utilization and inefficient training. These limitations require state-of-the-art systems to distribute training across multiple machines. We propose Marius, a system for efficient training of graph embeddings that leverages partition caching and buffer-aware data orderings to minimize disk access and interleaves data movement with computation to maximize utilization. We compare Marius against two state-of-the-art industrial systems on a diverse array of benchmarks. We demonstrate that Marius achieves the same level of accuracy but is up to one order of magnitude faster. We also show that Marius can scale training to datasets an order of magnitude beyond a single machine's GPU and CPU memory capacity, enabling training of configurations with more than a billion edges and 550 GB of total parameters on a single machine with 16 GB of GPU memory and 64 GB of CPU memory. Marius is open-sourced at www.marius-project.org.
SkMTEB: Slovak Massive Text Embedding Benchmark and Model Adaptation
Comenius University in Bratislava（布拉迪斯拉发夸美纽斯大学）；Cisco Systems（思科系统）；Technical University of Košice（科希策技术大学）；Kempelen Institute of Intelligent Technologies（肯佩伦智能技术研究所）
Harvard University（哈佛大学）；SaluteDevices；MIRAI；Zendesk；Shanghai University of Finance and Economics（上海财经大学）；Google LLC；Salesforce；Cornell University（康奈尔大学）；Astera Institute（Astera研究院）；Independent Contributor（独立贡献者）；Indian Institute of Technology, Kharagpur（印度理工学院，克拉格浦分校）；Barclays（巴克莱银行）；Aarhus University（奥胡斯大学）；Stanford University（斯坦福大学）
Carleton University（卡尔顿大学）；Durham University（杜伦大学）；Stanford University（斯坦福大学）；Aarhus University（奥胡斯大学）；Indian Institute of Technology, Kharagpur（印度理工学院，卡里格普）；Harvard University（哈佛大学）
