arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

提交批量作业对等待时间的影响有多大?对Jean Zay工作负载轨迹的分析

How Much Does Submitting a Burst of Jobs Impact Waiting Time? An Analysis of the Jean Zay Workload Trace

Marius Garénaux, François Bodin

arXiv 2610.11703首次发表:更新:

AI 中文总结

本文分析Jean Zay超级计算机的工作负载轨迹,发现作业批量会影响等待时间,推导Slurm公平共享因子公式,经Welch t检验确定分区占用率和请求核心数对等待时间有显著影响。

AI 中文摘要

本文研究了IDRIS(科学信息学发展与资源研究所)的Jean Zay超级计算机上的作业等待时间,其研究动机是改进跨设施科学工作流的调度策略。基于大型工作负载轨迹,我们发现作业等待时间并非相互独立,而是通过作业批量(即一起提交的相同作业批次,也称为“flurries”)产生关联。我们提出了一种量化这些批量对等待时间影响的方法,发现等待时间与其在所属批量中的排名之间存在强皮尔逊相关性,超过40%的作业该相关性高于0.8。我们进一步基于卷积乘积推导了Slurm公平共享因子的精确公式,表明批量惩罚的时间尺度远短于公平共享本身,这指向一种专门针对批量提交的不同节流机制。在考虑批量效应后,我们使用Welch t检验发现,分区占用率和请求核心数对等待时间有显著影响,但并非所有分区的影响都显著。

英文摘要

This paper examines job waiting times on the Jean Zay supercomputer at IDRIS (Institut du D{é}veloppement et des Ressources en Informatique Scientifique http://www.idris.fr/), driven by the need to improve scheduling strategies for crossfacility scientific workflows. Drawing on large workload traces, we show that job waiting times are not independent of one another but are instead correlated through job bursts-batches of identical jobs submitted together, also called ``flurries''. We introduce a method for quantifying the effect of these bursts on waiting times, and find a strong Pearson correlation between waiting time and a job's rank within its burst. This correlation is above 0.8 for more than 40% of the jobs. We further derive an exact formula for the Slurm fairshare factor based on a convolution product, showing that the burst penalty operates on a much shorter timescale than fairshare itself-pointing to a distinct throttling mechanism specifically targeting bulk submissions. Once burst effects are accounted for, we use Welch's t-test to find that partition occupancy and number of requested cores have a significant effect on waiting time, though the effect is not significant for all partitions.

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑