数据中心中基于空间多任务GPU的QoS-Aware和资源高效GPU微服务
Towards QoS-Aware and Resource-Efficient GPU Microservices Based on Spatial Multitasking GPUs In Datacenters
摘要：
虽然先前的研究集中在基于CPU的微服务上，但由于不同的竞争模式，它们不适用于基于GPU的微服务。在保证GPU微服务QoS的同时优化资源利用率是一项具有挑战性的工作。我们发现，开销是由微服务间通信，GPU资源争用和微服务管道内不平衡的吞吐量引起的。我们提出了Camelot，一个运行时系统，管理GPU micorservices考虑到上述因素。在Camelot中，基于内存的全局通信机制实现了现场数据共享，大大减少了用户查询的端到端延迟。我们还提出了两个竞争意识的资源分配政策，要么最大限度地提高峰值支持的服务负载或最小化的资源使用在低负载，同时确保所需的QoS。这两种策略在为微服务分配资源时考虑了微服务管道效应和运行时GPU资源争用。与最先进的工作相比，Camelot在有限的GPU下将支持的峰值负载提高了64.5%，并在低负载下减少了35%的资源使用，同时实现了所需的99%-ile延迟目标。
英文摘要：
While prior researches focus on CPU-based microservices, they are not applicable for GPU-based microservices due to the different contention patterns. It is challenging to optimize the resource utilization while guaranteeing the QoS for GPU microservices. We find that the overhead is caused by inter microservice communication, GPU resource contention and imbalanced throughput within microservice pipeline. We propose Camelot, a runtime system that manages GPU micorservices considering the above factors. In Camelot, a global memory-based communication mechanism enables onsite data sharing that significantly reduces the end-to-end latencies of user queries. We also propose two contention aware resource allocation policies that either maximize the peak supported service load or minimize the resource usage at low load while ensuring the required QoS. The two policies consider the microservice pipeline effect and the runtime GPU resource contention when allocating resources for the microservices. Compared with state-of-the-art work, Camelot increases the supported peak load by up to 64.5% with limited GPUs, and reduces 35% resource usage at low load while achieving the desired 99%-ile latency target.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
