arXiv 1606.044732016-06-14cs.DC

Multi-Tenant Virtual GPUs for Optimising Performance of a Financial Risk Application

Javier Prades, Blesson Varghese, Carlos Reano, Federico Silla

备注：: Accepted to the Journal of Parallel and Distributed Computing (JPDC), 10 June 2016

英文摘要：

Graphics Processing Units (GPUs) are becoming popular accelerators in modern High-Performance Computing (HPC) clusters. Installing GPUs on each node of the cluster is not efficient resulting in high costs and power consumption as well as underutilisation of the accelerator. The research reported in this paper is motivated towards the use of few physical GPUs by providing cluster nodes access to remote GPUs on-demand for a financial risk application. We hypothesise that sharing GPUs between several nodes, referred to as multi-tenancy, reduces the execution time and energy consumed by an application. Two data transfer modes between the CPU and the GPUs, namely concurrent and sequential, are explored. The key result from the experiments is that multi-tenancy with few physical GPUs using sequential data transfers lowers the execution time and the energy consumed, thereby improving the overall performance of the application.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

A Simulation Platform for Multi-tenant Machine Learning Services on Thousands of GPUs

University of Toronto（多伦多大学）；National University of Singapore（新加坡国立大学）；SenseTime（商汤科技）

Multi-tenant machine learning services have become emerging data-intensive workloads in data centers with heavy usage of GPU resources. Due to the large scale, many tuning parameters and heavy resource usage, it is usually impractical to evaluate and benchmark those machine learning services on real clusters. In this demonstration, we present AnalySIM, a cluster simulator that allows efficient design explorations for multi-tenant machine learning services. Specifically, by trace-driven cluster workload simulation, AnalySIM can easily test and analyze various scheduling policies in a number of …

2022-01-10cs.AIcs.DC

Analysis of Large-Scale Multi-Tenant GPU Clusters for DNN Training Workloads

With widespread advances in machine learning, a number of large enterprises are beginning to incorporate machine learning models across a number of products. These models are typically trained on shared, multi-tenant GPU clusters. Similar to existing cluster computing workloads, scheduling frameworks aim to provide features like high efficiency, resource isolation, fair sharing across users, etc. However Deep Neural Network (DNN) based workloads, predominantly trained on GPUs, differ in two significant ways from traditional big data analytics workloads. First, from a cluster utilization perspe…

2019-08-08cs.DC

A Survey of Multi-Tenant Deep Learning Inference on GPU

Deep Learning (DL) models have achieved superior performance. Meanwhile, computing hardware like NVIDIA GPUs also demonstrated strong computing scaling trends with 2x throughput and memory bandwidth for each generation. With such strong computing scaling of GPUs, multi-tenant deep learning inference by co-locating multiple DL models onto the same GPU becomes widely deployed to improve resource utilization, enhance serving throughput, reduce energy cost, etc. However, achieving efficient multi-tenant DL inference is challenging which requires thorough full-stack system optimization. This survey…

2022-05-24cs.ARcs.DC

面向多租户MIG云的在线碎片感知GPU调度器

An Online Fragmentation-Aware GPU Scheduler for Multi-Tenant MIG-based Clouds

本文提出了一种面向多租户MIG云的在线碎片感知GPU调度器，通过引入碎片度量指标和贪心算法，有效减少GPU碎片化，提高工作负载调度效率。

2025-11-24cs.DCcs.NI

Guardian：多租户环境中的安全GPU共享

Guardian: Safe GPU Sharing in Multi-Tenant Environments

针对多租户GPU共享中的内存安全隐患，本文提出PTX级边界检查方法Guardian，通过地址空间分区、底层调用拦截和PTX级内核插桩实现动态空间共享与内存隔离，在Caffe和PyTorch等框架上平均开销仅9%。

2024-10-16cs.DC

Elastic deep learning in multi-tenant GPU cluster

We study how to support elasticity, i.e., the ability to dynamically adjust the parallelism (number of GPUs), for deep neural network (DNN) training. Elasticity can benefit multi-tenant GPU cluster management in many ways, e.g., achieving various scheduling objectives (e.g., job throughput, job completion time, GPU efficiency) according to cluster load variations, maximizing the use of transient idle resources, performance profiling, job migration, and straggler mitigation. However, existing parallelism adjustment strategies incur high overheads, which hinder many applications from making effe…

2019-11-30cs.DC

更多相关论文收起更多论文

Perseus: Characterizing Performance and Cost of Multi-Tenant Serving for CNN Models

Worcester Polytechnic Institute（伍斯特理工学院）

Deep learning models are increasingly used for end-user applications, supporting both novel features such as facial recognition, and traditional features, e.g. web search. To accommodate high inference throughput, it is common to host a single pre-trained Convolutional Neural Network (CNN) in dedicated cloud-based servers with hardware accelerators such as Graphics Processing Units (GPUs). However, GPUs can be orders of magnitude more expensive than traditional Central Processing Unit (CPU) servers. These resources could also be under-utilized facing dynamic workloads, which may result in infl…

2020-03-30cs.DCcs.LGcs.PF

Synergy: Resource Sensitive DNN Scheduling in Multi-Tenant Clusters

Microsoft Research（微软研究院）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；VMware Research（威睿研究院）

Training Deep Neural Networks (DNNs) is a widely popular workload in both enterprises and cloud data centers. Existing schedulers for DNN training consider GPU as the dominant resource, and allocate other resources such as CPU and memory proportional to the number of GPUs requested by the job. Unfortunately, these schedulers do not consider the impact of a job's sensitivity to allocation of CPU, memory, and storage resources. In this work, we propose Synergy, a resource-sensitive scheduler for shared GPU clusters. Synergy infers the sensitivity of DNNs to different resources using optimistic p…

2022-08-24cs.DCcs.LG

MISO: Exploiting Multi-Instance GPU Capability on Multi-Tenant Systems for Machine Learning

GPU technology has been improving at an expedited pace in terms of size and performance, empowering HPC and AI/ML researchers to advance the scientific discovery process. However, this also leads to inefficient resource usage, as most GPU workloads, including complicated AI/ML models, are not able to utilize the GPU resources to their fullest extent -- encouraging support for GPU multi-tenancy. We propose MISO, a technique to exploit the Multi-Instance GPU (MIG) capability on the latest NVIDIA datacenter GPUs (e.g., A100, H100) to dynamically partition GPU resources among co-located jobs. MISO…

2022-10-06cs.ARcs.DC

LLMVisor：面向多租户大语言模型（LLM）服务的实时延迟归因模型

LLMVisor: A Real-Time Latency Attribution Model for Multi-Tenant LLM Serving

University of Michigan（密歇根大学）；ByteDance（字节跳动）

针对多租户LLM服务中联合批处理导致的延迟归因难题，提出基于屋顶线模型的LLMVisor实时延迟归因模型，在微秒级高效运行，相比基线方法大幅降低预填充和解码阶段的相对误差

2026-08-11cs.AIcs.DC

EvalStop：利用世界反馈检测和纠正多租户RLHF平台中的奖励过度优化

EvalStop: Using World Feedback to Detect and Correct Reward Overoptimization in Multi-Tenant RLHF Platforms

DeepMind, London, UK（深度Mind, 英国伦敦）；University of Cambridge, UK（英国剑桥大学）；University of Washington, USA（美国华盛顿大学）

提出EvalStop调度原语，通过检测评估分数连续下降来终止作业、释放GPU并保留最佳检查点，以纠正奖励过度优化，在RLHF负载上实现高精度检测并提升JCT。

2026-06-16cs.AIcs.DCcs.LG

DriftSched: 多租户GPU推理中运行时令牌漂移下的自适应QoS感知调度

DriftSched: Adaptive QoS-Aware Scheduling under Runtime Token Drift for Multi-Tenant GPU Inference

University of California, Berkeley（加州大学伯克利分校）

提出DriftSched框架，通过运行时反馈驱动的漂移补偿和自适应偏差校正，解决多租户LLM推理中令牌漂移导致的调度问题，在NVIDIA L4 GPU上实现平均38.8%的估计误差降低和42%的中位延迟改善。

2026-06-23cs.DCcs.LGcs.PF

通过智能资源共享在多租户GPU集群中调度深度学习作业

Scheduling Deep Learning Jobs in Multi-Tenant GPU Clusters via Wise Resource Sharing

提出SJF-BSBF调度算法，通过智能GPU共享与梯度累积优化多租户集群DL作业调度，无需抢占即可降低平均完成时间27-33%，并优于激进共享策略17%。

2024-07-18cs.DC

面向高效多租户模型服务的全局仿真引导型动态算子调度

Global Simulation-Guided Dynamic Operator Scheduling for Efficient Multi-Tenant Model Serving

Xiamen University（厦门大学）；Shanghai Innovation Institute（上海创新研究院）；Shanghai Jiao Tong University（上海交通大学）；East China Normal University（华东师范大学）；Tsinghua University（清华大学）

本文提出SliceScheduler系统，通过全局映射图、全局仿真器等组件实现算子级调度，在维持SLA违规率低于9%的同时，将多租户LLM服务的令牌吞吐量提升1.10-2.29倍，有效提高GPU利用率。

2026-08-18cs.DCcs.LGcs.OS

SAKURAONE：一种基于开放以太网的AI HPC系统及其在单租户LLM开发环境中的观测工作负载动态

SAKURAONE: An Open Ethernet-Based AI HPC System and Its Observed Workload Dynamics in a Single-Tenant LLM Development Environment

Research Center, SAKURA internet Inc.（SAKURA internet 公司研究中心）

SAKURAONE是一种基于开放以太网的AI HPC系统，通过观测单租户LLM开发环境中的工作负载动态，揭示了GPU集群在统一项目负载下的实际利用情况。

2026-04-16cs.DCcs.NI

Beaver：机器学习与延迟关键型vRAN工作负载之间的弹性GPU共享

Beaver: Elastic GPU Sharing between ML and Latency-Critical vRAN Workloads

Duke University（杜克大学）

Beaver通过动态调整SM分配和重写ML内核，在共享GPU上保护vRAN延迟关键型工作负载的截止时间，同时保留74%的LLM服务吞吐量。

2026-10-05cs.DCcs.NI

比特致瓦：一种利用GPU工作负载跨电力和计算基础设施的网络物理漏洞

Bit2Watt: A Cyber-Physical Vulnerability Exploiting GPU Workloads Across Power and Computing Infrastructures

Zhejiang University（浙江大学）

研究揭示利用GPU工作负载的网络物理漏洞Bit2Watt，攻击者借此引发高频功率调制破坏电力及计算设施。通过多种实验验证，此漏洞会致电力质量下降、引发连锁故障等，还分析了反馈路径风险，强调跨层防御的迫切性。

2026-07-08cs.ARcs.CRcs.DC