arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.06194cs.DC
收藏

SuperBench:通过主动验证提升云AI基础设施可靠性

SuperBench: Improving Cloud AI Infrastructure Reliability with Proactive Validation

Yifan Xiong, Yuting Jiang, Ziyue Yang, Lei Qu, Guoshuai Zhao, Shuguang Liu, Dong Zhong, Boris Pinzur, Jie Zhang, Yang Wang, Jithin Jose, Hossein Pourreza, Jeff … 展开作者

Yifan Xiong, Yuting Jiang, Ziyue Yang, Lei Qu, Guoshuai Zhao, Shuguang Liu, Dong Zhong, Boris Pinzur, Jie Zhang, Yang Wang, Jithin Jose, Hossein Pourreza, Jeff Baxter, Kushal Datta, Prabhat Ram, Luke Melton, Joe Chau, Peng Cheng, Yongqiang Xiong, Lidong Zhou

AI总结:

SuperBench通过主动验证机制和基准测试套件,结合验证器与选择器,有效识别云AI基础设施中由硬件冗余引起的隐藏性能退化,并在Azure生产环境中将平均故障间隔时间提升最高22.61倍。

摘要:

云AI基础设施的可靠性对云服务提供商至关重要,这促使硬件冗余被广泛采用。然而,这些冗余可能无意中导致AI工作负载出现隐藏的性能退化,即所谓的“灰色故障”,严重影响端到端性能并掩盖性能问题,从而使故障和性能回退的根因分析变得更加复杂。我们提出了SuperBench,一个面向AI基础设施的主动验证系统,用于缓解由硬件冗余引起的隐藏退化并提升整体可靠性。SuperBench具备一套全面的基准测试套件,能够评估单个硬件组件,并能代表大多数真实AI工作负载。它包含一个验证器(Validator),通过学习基准测试标准来准确定位有缺陷的组件。此外,SuperBench还集成了一个选择器(Selector),用于在验证时间和问题相关代价之间取得平衡,从而以定制化的基准测试子集实现验证执行的最佳时机。通过测试平台评估和仿真,我们证明SuperBench可以将平均故障间隔时间提升最高达22.61倍。SuperBench已成功部署于Azure生产环境中,在过去两年中验证了数十万块GPU。

英文摘要:

Reliability in cloud AI infrastructure is crucial for cloud service providers, prompting the widespread use of hardware redundancies. However, these redundancies can inadvertently lead to hidden degradation, so called "gray failure", for AI workloads, significantly affecting end-to-end performance and concealing performance issues, which complicates root cause analysis for failures and regressions. We introduce SuperBench, a proactive validation system for AI infrastructure that mitigates hidden degradation caused by hardware redundancies and enhances overall reliability. SuperBench features a comprehensive benchmark suite, capable of evaluating individual hardware components and representing most real AI workloads. It comprises a Validator which learns benchmark criteria to clearly pinpoint defective components. Additionally, SuperBench incorporates a Selector to balance validation time and issue-related penalties, enabling optimal timing for validation execution with a tailored subset of benchmarks. Through testbed evaluation and simulation, we demonstrate that SuperBench can increase the mean time between incidents by up to 22.61x. SuperBench has been successfully deployed in Azure production, validating hundreds of thousands of GPUs over the last two years.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑