基于SMDP的动态批处理以提升批处理服务的响应性与能效
SMDP-Based Dynamic Batching for Improving Responsiveness and Energy Efficiency of Batch Services
AI总结：
本文将批处理服务队列建模为SMDP，提出近似最优动态批处理策略，以权衡平均响应时间与功耗，并通过抽象成本大幅降低复杂度。
摘要：
对于集成并行计算资源的服务器而言，批处理是大规模提供高效且经济服务的关键技术。并行计算资源在以更大批大小运行时表现出更高的计算效率和能效。然而，在在线服务领域，采用更大的批大小可能导致更长的响应时间。本文旨在提供一种动态批处理方案，以精细平衡延迟与效率。该系统被建模为一个具有依赖于批大小的服务时间的批处理服务队列。随后，动态批处理的设计被表述为一个半马尔可夫决策过程（SMDP）问题，目标是最小化平均响应时间与平均功耗的加权和。文中提出了一种推导近似最优SMDP解的方法，该解表示所选择的动态批处理策略。通过引入抽象成本来反映“尾部”状态的影响，该过程的空间复杂度和时间复杂度可分别降低63.5%和98%。数值结果展示了基于SMDP的批处理策略在各种参数设置下的优越性。此外，所提出的方案在平衡功耗与延迟方面表现出显著的灵活性。
英文摘要：
For servers incorporating parallel computing resources, batching is a pivotal technique for providing efficient and economical services at scale. Parallel computing resources exhibit heightened computational and energy efficiency when operating with larger batch sizes. However, in the realm of online services, the adoption of a larger batch size may lead to longer response times. This paper aims to provide a dynamic batching scheme that delicately balances latency and efficiency. The system is modeled as a batch service queue with size-dependent service times. Then, the design of dynamic batching is formulated as a semi-Markov decision process (SMDP) problem, with the objective of minimizing the weighted sum of average response time and average power consumption. A method is proposed to derive an approximate optimal SMDP solution, representing the chosen dynamic batching policy. By introducing an abstract cost to reflect the impact of "tail" states, the space complexity and the time complexity of the procedure can decrease by 63.5% and 98%, respectively. Numerical results showcase the superiority of SMDP-based batching policies across various parameter setups. Additionally, the proposed scheme exhibits noteworthy flexibility in balancing power consumption and latency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
