离线强化学习基于的应用无关能效功率控制
Offline Reinforcement-Learning-Based Power Control for Application-Agnostic Energy Efficiency
AI总结：
本文提出利用离线强化学习设计自主CPU功率控制器，以提升并行应用的能源效率，同时控制性能降级。
摘要：
能源效率已成为现代计算基础设施设计中的重要组成部分，影响着生产系统的性能、成本、可扩展性和耐用性。在CPU设计中纳入功率执行和传感能力，表明了这一点，使系统软件能够主动监控和调整能源消耗和性能。虽然强化学习（RL）似乎理想用于此类能源效率控制系统的設計，但在线训练带来了从缺乏适当模型来设置合适的模拟环境到扰动（噪声）和可靠性问题等挑战，如果在实时系统上进行训练。本文讨论了使用离线强化学习作为设计自主CPU功率控制器的替代方法，目标是提高并行应用在运行时的能源效率，而不会对性能造成过度影响。离线RL通过利用在训练前从任意策略收集的状态转换数据集来避免在线RL训练带来的问题。我们的方法将离线RL应用于一种灰箱方法来提高能源效率，结合在线应用无关的性能数据（例如心跳）和硬件性能计数器，以确保在有限的性能降级下满足科学目标。在各种计算密集型和内存密集型基准测试上评估我们的方法，并通过Intel的Running Average Power Limit在实时系统上控制功率，我们证明了这种离线训练的代理可以显著减少能耗，而在性能降级成本上是可接受的。
英文摘要：
Energy efficiency has become an integral aspect of modern computing infrastructure design, impacting the performance, cost, scalability, and durability of production systems. The incorporation of power actuation and sensing capabilities in CPU designs is indicative of this, enabling the deployment of system software that can actively monitor and adjust energy consumption and performance at runtime. While reinforcement learning (RL) would seem ideal for the design of such energy efficiency control systems, online training presents challenges ranging from the lack of proper models for setting up an adequate simulated environment, to perturbation (noise) and reliability issues, if training is deployed on a live system. In this paper we discuss the use of offline reinforcement learning as an alternative approach for the design of an autonomous CPU power controller, with the goal of improving the energy efficiency of parallel applications at runtime without unduly impacting their performance. Offline RL sidesteps the issues incurred by online RL training by leveraging a dataset of state transitions collected from arbitrary policies prior to training. Our methodology applies offline RL to a gray-box approach to energy efficiency, combining online application-agnostic performance data (e.g., heartbeats) and hardware performance counters to ensure that the scientific objectives are met with limited performance degradation. Evaluating our method on a variety of compute-bound and memory-bound benchmarks and controlling power on a live system through Intel's Running Average Power Limit, we demonstrate that such an offline-trained agent can substantially reduce energy consumption at a tolerable performance degradation cost.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
