Paving the way for Distributed Non-Blocking Algorithms and Data Structures in the Partitioned Global Address Space

CausalSpartanX: Causal Consistency and Non-Blocking Read-Only Transactions

Causal consistency is an intermediate consistency model that can be achieved together with high availability and performance requirements even in presence of network partitions. In the context of partitioned data stores, it has been shown that implicit dependency tracking using timestamps is more efficient than explicit dependency tracking. Existing time-based solutions depend on monotonic psychical clocks that are closely synchronized. These requirements make current protocols vulnerable to clock anomalies. In this paper, we propose a new time-based algorithm, CausalSpartanX, that instead of …

通过合成中间标签进行分区神经网络训练

Partitioned Neural Network Training via Synthetic Intermediate Labels

本文提出通过将神经网络分区到多个GPU并利用随机生成的合成中间标签训练各分段，以降低内存和通信开销，实验表明该方法在扩展MNIST上可保持与传统训练相近的准确率。

可扩展的森林-八叉树与任意分布式并行网格之间的一方向网格重叠计算

Scalable overset computation between a forest-of-octrees- and an arbitrary distributed parallel mesh

本文提出了一种可扩展的网格重叠算法，用于在森林-八叉树与任意分布式并行网格之间进行高效的数据查询与通信，适用于高并发计算场景。

Optimization and parallelization of B-spline based orbital evaluations in QMC on multi/many-core shared memory processors

B-spline based orbital representations are widely used in Quantum Monte Carlo (QMC) simulations of solids, historically taking as much as 50% of the total run time. Random accesses to a large four-dimensional array make it challenging to efficiently utilize caches and wide vector units of modern CPUs. We present node-level optimizations of B-spline evaluations on multi/many-core shared memory processors. To increase SIMD efficiency and bandwidth utilization, we first apply data layout transformation from array-of-structures to structure-of-arrays (SoA). Then by blocking SoA objects, we optimiz…

Scalable Graph Convolutional Network Training on Distributed-Memory Systems

Graph Convolutional Networks (GCNs) are extensively utilized for deep learning on graphs. The large data sizes of graphs and their vertex features make scalable training algorithms and distributed memory systems necessary. Since the convolution operation on graphs induces irregular memory access patterns, designing a memory- and communication-efficient parallel algorithm for GCN training poses unique challenges. We propose a highly parallel training algorithm that scales to large processor counts. In our solution, the large adjacency and vertex-feature matrices are partitioned among processors…