AdapTBF：通过自适应令牌借贷实现HPC存储的去中心化带宽控制
AdapTBF: Decentralized Bandwidth Control via Adaptive Token Borrowing for HPC Storage
AI总结：
AdapTBF通过自适应令牌借贷实现HPC存储的去中心化带宽控制，提升存储效率与公平性。
摘要：
现代高性能计算（HPC）应用在计算资源上运行，但共享全局存储系统。这种设计在应用消耗的存储带宽与其分配的计算资源不成比例时会引发问题。例如，一个在单个计算节点上运行的应用程序可以发出许多小随机写入，并从存储服务器消耗过多的I/O带宽。这会阻碍写入相同存储服务器的更大作业，这些作业分配了多个计算节点，导致显著的资源浪费。一个直接的解决方案是按应用分配的计算资源比例限制其在存储服务器上的I/O带宽。这种方法已在并行文件系统中使用令牌桶过滤器（TBF）实现。然而，严格的比例限制常常会降低整体I/O效率，因为HPC应用生成短而突发的I/O。限制带宽会浪费服务器容量，当应用程序空闲时，或阻止应用程序在突发阶段暂时使用更高的带宽。我们主张I/O控制应最大化每个应用程序的性能和整体存储效率，同时确保公平性（例如防止小作业阻碍大规模作业）。我们提出了AdapTBF，它在现代并行文件系统（如Lustre）中基于TBF，并引入了使用自适应借贷的去中心化带宽控制方法。我们详细说明了算法，将AdapTBF实现为Lustre，并使用合成工作负载进行评估，这些工作负载基于真实场景。结果表明，AdapTBF在极端条件下能够有效管理I/O带宽，同时保持高存储利用率。
英文摘要：
Modern high-performance computing (HPC) applications run on compute resources but share global storage systems. This design can cause problems when applications consume a disproportionate amount of storage bandwidth relative to their allocated compute resources. For example, an application running on a single compute node can issue many small, random writes and consume excessive I/O bandwidth from a storage server. This can hinder larger jobs that write to the same storage server and are allocated many compute nodes, resulting in significant resource waste. A straightforward solution is to limit each application's I/O bandwidth on storage servers in proportion to its allocated compute resources. This approach has been implemented in parallel file systems using Token Bucket Filter (TBF). However, strict proportional limits often reduce overall I/O efficiency because HPC applications generate short, bursty I/O. Limiting bandwidth can waste server capacity when applications are idle or prevent applications from temporarily using higher bandwidth during bursty phases. We argue that I/O control should maximize per-application performance and overall storage efficiency while ensuring fairness (e.g., preventing small jobs from blocking large-scale ones). We propose AdapTBF, which builds on TBF in modern parallel file systems (e.g., Lustre) and introduces a decentralized bandwidth control approach using adaptive borrowing and lending. We detail the algorithm, implement AdapTBF in Lustre, and evaluate it using synthetic workloads modeled after real-world scenarios. Results show that AdapTBF manages I/O bandwidth effectively while maintaining high storage utilization, even under extreme conditions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
