发表机构
Google; Drexel University（谷歌; 德雷塞尔大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对机器学习训练资源供需差距，提出基于市场的配额市场机制，通过允许用户表达工作负载价值及动态定价，实现帕累托效率和最大最小公平性，促进符合组织优先级的资源分配。
AI 中文摘要
近年来，机器学习（ML）训练资源需求不断攀升，导致高需求与可用供应之间存在巨大差距。有效分配这些稀缺且昂贵的资源对组织实现投资回报最大化至关重要。现有资源分配机制在动态用户需求场景下能保证帕累托效率和最大最小公平性，但在存在异质价值需求时无法保持这些关键属性。本文描述了配额市场（Quota Marketplace）的设计、实施、部署和理论分析，这是一种基于市场的机制，用于高效分配ML训练芯片（如GPU），明确解决了具有异质价值需求的场景。我们详细介绍了该机制在谷歌内部的实施情况，并展示了证明其影响的指标。我们还讨论了配额市场有效处理的许多关键业务需求，并记录了它所带来的收益和机会。我们从理论上证明了这种基于市场的方法如何通过允许用户表达其工作负载的价值并基于供需波动实现动态资源定价，从而实现帕累托效率和最大最小公平性的基本属性。最终，该市场促进了与组织优先级一致的资源分配。
英文摘要
The escalating demand for Machine Learning (ML) training resources in recent years has resulted in a substantial gap between the high demand and the available supply. Efficient allocation of these scarce and expensive resources is crucial for organizations to maximize their return on investment. Existing resource allocation mechanisms, like Karma [OSDI'23], are designed to guarantee Pareto efficiency and max-min fairness in settings with dynamic (time-varying) user demands, but fail to preserve these key properties in the presence of demands with heterogeneous values. Given the ubiquity and inevitability of heterogeneity in organizational values of different workloads, effective resource allocation policies must accommodate these variations. In this paper, we describe the design, implementation, deployment, and theoretical analysis of Quota Marketplace, a market-based mechanism to efficiently allocate ML training chips (like GPUs), explicitly addressing scenarios with demands of heterogeneous value. We detail the implementation of this mechanism within Google and present metrics that demonstrate its impact. We also discuss many business-critical requirements that the Quota Marketplace handles quite effectively, and document the gains and opportunities it has unlocked. We establish theoretically how this market-based approach achieves the essential properties of Pareto efficiency and max-min fairness by allowing the users to express the value of their workloads and enabling dynamic resource pricing based on supply and demand fluctuations. Ultimately, the market facilitates resource allocation that aligns with organizational priorities.
CommentsOSDI 2026 Paper
