EdgeTimer：移动边缘计算中基于深度强化学习的自适应多时间尺度调度
EdgeTimer: Adaptive Multi-Timescale Scheduling in Mobile Edge Computing with Deep Reinforcement Learning
AI总结：
EdgeTimer首次利用深度强化学习自动生成自适应时间尺度，通过三层分层框架和安全多智能体DRL优化移动边缘计算多层调度，在Kubernetes生产轨迹上实现最高9.1倍利润提升且不牺牲延迟性能。
摘要：
在移动边缘计算（MEC）中，资源调度对任务请求的性能和服务提供商的成本至关重要，涉及多层异构调度决策。现有调度器通常采用静态时间尺度定期更新各层调度决策，缺乏针对不同层的自适应时间尺度调整，导致实际性能可能较差。我们注意到，自适应时间尺度能够显著改善运营成本与延迟性能之间的权衡。基于这一认识，我们提出EdgeTimer，这是首个利用深度强化学习（DRL）自动生成自适应时间尺度以更新多层调度决策的工作。首先，EdgeTimer采用三层分层DRL框架，将多层决策任务解耦为一系列独立子任务，以提高学习效率。其次，为应对每个子任务，EdgeTimer采用安全的多智能体DRL算法进行分散调度，同时确保系统可靠性。我们将EdgeTimer应用于广泛的Kubernetes调度规则，并使用具有不同工作负载模式的生产轨迹进行评估。大量基于轨迹的实验表明，无论工作负载模式和内置调度规则如何，EdgeTimer都能学习到自适应时间尺度。与现有方法相比，它在不牺牲延迟性能的情况下可获得高达9.1倍的利润提升。
英文摘要：
In mobile edge computing (MEC), resource scheduling is crucial to task requests' performance and service providers' cost, involving multi-layer heterogeneous scheduling decisions. Existing schedulers typically adopt static timescales to regularly update scheduling decisions of each layer, without adaptive adjustment of timescales for different layers, resulting in potentially poor performance in practice. We notice that the adaptive timescales would significantly improve the trade-off between the operation cost and delay performance. Based on this insight, we propose EdgeTimer, the first work to automatically generate adaptive timescales to update multi-layer scheduling decisions using deep reinforcement learning (DRL). First, EdgeTimer uses a three-layer hierarchical DRL framework to decouple the multi-layer decision-making task into a hierarchy of independent sub-tasks for improving learning efficiency. Second, to cope with each sub-task, EdgeTimer adopts a safe multi-agent DRL algorithm for decentralized scheduling while ensuring system reliability. We apply EdgeTimer to a wide range of Kubernetes scheduling rules, and evaluate it using production traces with different workload patterns. Extensive trace-driven experiments demonstrate that EdgeTimer can learn adaptive timescales, irrespective of workload patterns and built-in scheduling rules. It obtains up to 9.1x more profit than existing approaches without sacrificing the delay performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
