All-pairs compute problems apply a user-defined function to each combination of two items of a given data set. Although these problems present an abundance of parallelism, data reuse must be exploited to achieve good performance. Several researchers considered this problem, either resorting to partial replication with static work distribution or dynamic scheduling with full replication. In contrast, we present a solution that relies on hierarchical multi-level software-based caches to maximize data reuse at each level in the distributed memory hierarchy, combined with a divide-and-conquer approach to exploit data locality, hierarchical work-stealing to dynamically balance the workload, and asynchronous processing to maximize resource utilization. We evaluate our solution using three real-world applications (from digital forensics, localization microscopy, and bioinformatics) on different platforms (from a desktop machine to a supercomputer). Results shows excellent efficiency and scalability when scaling to 96 GPUs, even obtaining super-linear speedups due to a distributed cache.
A heterogeneous many-core platform for experiments on scalable custom interconnects and management of fault and critical events, applied to many-process applications: Vol. II, 2012 technical report
Distributed computing for physics-based data-driven reduced modeling at scale: Application to a rotating detonation rocket engine
Department of Mathematics, Virginia Tech(弗吉尼亚理工学院数学系);Oden Institute for Computational Engineering and Sciences, The University of Texas at Austin(德克萨斯大学奥登工程与科学研究院);Amentum, Edwards Air Force Base(艾门敦,爱德华兹空军基地);Air Force Research Laboratory, Edwards Air Force Base(空军研究实验室,爱德华兹空军基地)
Hetu v2: A General and Scalable Deep Learning System with Hierarchical and Heterogeneous Single Program Multiple Data Annotations
Beijing Key Laboratory of Software and Hardware Cooperative Artificial Intelligence Systems(软硬件协同智能系统北京市重点实验室);Institute of Computational Social Science, Peking University (Qingdao)(北京大学(青岛)计算社会科学研究院)
S2CE: A Hybrid Cloud and Edge Orchestrator for Mining Exascale Distributed Streams
物联网和其他设备等分布式和异构节点生成的数据量、速度、种类和准确性的爆炸性增长,不断挑战着大数据处理平台和挖掘技术的最新水平。因此,它揭示了迫切需要解决这种预期的艾级数据生成和从这些数据中提取见解之间不断增长的差距。为了满足这一需求,本文提出了Stream to Cloud & Edge(S2CE),这是同类产品中第一个优化的多云和边缘编排器,易于配置,可扩展和可扩展。S2CE将支持在混合云和边缘环境上运行的大量异构数据流的机器和深度学习,同时提供实用和可扩展处理的必要功能:数据融合和预处理,采样和合成流生成,云和边缘智能资源管理以及分布式处理。
SWARM+: Scalable and Resilient Multi-Agent Consensus for Decentralized Data-Aware Workload Management
Renaissance Computing Institute, University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校文艺复兴计算研究所);Argonne National Laboratory(阿贡国家实验室);Information Sciences Institute, University of Southern California(南加州大学信息科学研究所)
CoFormer: Collaborating with Heterogeneous Edge Devices for Scalable Transformer Inference
School of Information and Electrionics, Beijing Institute of Technology(信息与电子学院,北京理工大学);School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University(网络安全科学与技术学院,中山大学深圳校区);School of Computer Science, National Engineering Research Center for Multimedia Software, Wuhan University(计算机学院,多媒体软件国家工程研究中心,武汉大学);Information Technology Service Center of People’s Court, Beijing(人民法院信息技术服务中心,北京);College of Computing and Data Science, Nanyang Technological University(计算与数据科学学院,南洋理工大学)