Growing deployment of power and energy efficient throughput accelerators (GPU) in data centers demands enhancement of power-performance co-optimization capabilities of GPUs. Realization of exascale computing using accelerators requires further improvements in power efficiency. With hardwired kernel concurrency enablement in accelerators, inter- and intra-workload simultaneous kernels computation predicts increased throughput at lower energy budget. To improve Performance-per-Watt metric of the architectures, a systematic empirical study of real-world throughput workloads (with concurrent kernel execution) is required. To this end, we propose a multi-kernel throughput workload generation framework that will facilitate aggressive energy and performance management of exascale data centers and will stimulate synergistic power-performance co-optimization of throughput architectures. Also, we demonstrate a multi-kernel throughput benchmark suite based on the framework that encapsulates symmetric, asymmetric and co-existing (often appears together) kernel based workloads. On average, our analysis reveals that spatial and temporal concurrency within kernel execution in throughput architectures saves energy consumption by 32%, 26% and 33% in GTX470, Tesla M2050 and Tesla K20 across 12 benchmarks. Concurrency and enhanced utilization are often correlated but do not imply significant deviation in power dissipation. Diversity analysis of proposed multi-kernels confirms characteristic variation and power-profile diversity within the suite. Besides, we explain several findings regarding power-performance co-optimization of concurrent throughput workloads.
Kernelet: High-Throughput GPU Kernel Executions with Dynamic Slicing and Scheduling
图形处理器(GPU)最近被广泛用作集群和云等共享环境中的加速器。在这样的共享环境中,许多内核从不同的用户提交给GPU,吞吐量是性能和总拥有成本的重要指标。尽管最近改进了对并发GPU内核执行的运行时支持,但GPU可能严重未得到充分利用,导致次优吞吐量。在本文中,我们提出了Kernelet,一个运行时系统与动态切片和调度技术,以提高吞吐量的并发内核执行GPU上。通过切片,Kernelet将GPU内核划分为多个子内核(即切片)。每个切片都具有可调的占用率,以允许与其他切片协同调度并充分利用GPU资源。我们开发了一种新的和有效的基于马尔可夫链的性能模型来指导调度决策。我们的实验结果表明,在NVIDIA Tesla C2050和GTX 680 GPU上,性能分别提高了31.1%和23.4%。
High-Throughput Transaction Executions on Graphics Processors
联机事务处理(OLTP)是各种传统和新兴在线服务中的重要业务系统部分。由于用户数量的不断增加,OLTP系统需要高吞吐量,以便在短时间内执行数万个事务。受GPGPU(通用计算图形处理器)最近的成功,我们提出了GPUTx,一个OLTP引擎执行高吞吐量的事务执行的GPU内存中的数据库。与现有的GPGPU研究通常优化单个任务相比,事务执行需要同时处理许多小任务。具体来说,我们提出了批量执行模型,将多个事务分组为批量,并将批量作为单个任务在GPU上执行。批量内的事务在GPU上并发执行。本文研究了三种基本的执行策略(一种带锁,另两种无锁),并结合GPU对原子操作的硬件支持、大规模线程并行性和SPMD(Single Program Multiple Data)执行等特性对它们进行了优化。我们评估GPUTx在最新的NVIDIA GPU上与其在四核CPU上的对应物进行比较。我们的实验结果表明,GPUTx上的优化显着提高吞吐量,优化后的GPUTx达到4-10倍的吞吐量比其基于CPU的同行在公共事务处理基准测试。
EfficientAgent: What Makes KV Cache Offloading Work for Concurrent Agents?
The Hong Kong University of Science and Technology(香港科技大学);Huawei Technologies Ltd.(华为技术有限公司);The University of Hong Kong(香港大学);Sun Yat-sen University(中山大学)
B$^3$-PWL: GPU-Batched Branch-and-Bound for Piecewise-Linear Optimization with SOS2 Constraints
Johns Hopkins University(约翰斯·霍普金斯大学);University of North Carolina, Chapel Hill(北卡罗来纳大学教堂山分校);City University of Hong Kong(香港城市大学);The Hong Kong Institute of AI for Science(香港人工智能科学研究所)
TorchOpt: An Efficient Library for Differentiable Optimization
University of Edinburgh(爱丁堡大学);University College London(伦敦大学学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Institute for AI, Peking University(北京大学人工智能研究院)