arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.01565cs.LGcs.DC
收藏

TetriServe: 高效服务混合DiT工作负载

TetriServe: Efficiently Serving Mixed DiT Workloads

University of Michigan(密歇根大学);University of Wisconsin-Madison(威斯康星大学麦迪逊分校);Nanyang Technological University(南洋理工大学)

Runyu Lu, Shiqi He, Wenxuan Tan, Shenggui Li, Ruofan Wu, Jeff J. Ma, Ang Chen, Mosharaf Chowdhury

AI总结:

针对混合分辨率与截止时间的异构DiT工作负载,提出基于步骤级序列并行的TetriServe系统,通过轮次调度与自适应并行度,在保证图像质量下将SLO达成率提升32%。

摘要:

扩散Transformer(DiT)模型通过迭代去噪步骤生成高质量图像,但由于其高计算成本(尤其在大分辨率下),在严格服务级别目标(SLO)下服务这些模型具有挑战性。现有服务系统使用固定程度的序列并行,这对于具有混合分辨率和截止时间的异构工作负载效率低下,导致GPU利用率低和SLO达成率低。在本文中,我们提出步骤级序列并行,根据请求的截止时间动态调整单个请求的并行度。我们提出了TetriServe,一个实现此策略的DiT服务系统,用于高效图像生成。具体来说,TetriServe引入了一种新颖的基于轮次的调度机制,通过(1)将时间离散化为固定轮次以使截止时间感知调度可处理,(2)在步骤级别自适应并行度并最小化GPU小时消耗,以及(3)联合打包请求以最小化延迟完成,从而提高SLO达成率。对最先进的DiT模型进行的广泛评估表明,与现有解决方案相比,TetriServe在不降低图像质量的情况下实现了高达32%的SLO达成率提升。

英文摘要:

Diffusion Transformer (DiT) models excel at generating high-quality images through iterative denoising steps, but serving them under strict Service Level Objectives (SLOs) is challenging due to their high computational cost, particularly at larger resolutions. Existing serving systems use fixed-degree sequence parallelism, which is inefficient for heterogeneous workloads with mixed resolutions and deadlines, leading to poor GPU utilization and low SLO attainment. In this paper, we propose step-level sequence parallelism to dynamically adjust the degree of parallelism of individual requests according to their deadlines. We present TetriServe, a DiT serving system that implements this strategy for highly efficient image generation. Specifically, TetriServe introduces a novel round-based scheduling mechanism that improves SLO attainment by (1) discretizing time into fixed rounds to make deadline-aware scheduling tractable, (2) adapting parallelism at the step level and minimizing GPU hour consumption, and (3) jointly packing requests to minimize late completions. Extensive evaluation on state-of-the-art DiT models shows that TetriServe achieves up to 32% higher SLO attainment compared to existing solutions without degrading image quality.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑