arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.16463cs.LGcs.DC
收藏

利用老虎机反馈学习在线任务调度

Learning to Schedule Online Tasks with Bandit Feedback

ShanghaiTech University(上海科技大学)

Yongxin Xu, Shangshang Wang, Hengquan Guo, Xin Liu, Ziyu Shao

AI总结:

本文针对奖励、成本未知且任务到达分布不确定的在线任务调度问题,提出基于双重乐观学习和 Robbins-Monro 方法的 DOL-RM 算法,首次实现 $O(T^{3/4})$ 次线性遗憾,并在实验中取得最佳累积奖励成本比。

摘要:

在线任务调度在云计算和众包等任务密集型应用中发挥着不可或缺的作用。在一定的任务到达分布下,最优调度可以提升系统性能,通常以奖励与成本之比来衡量。一方面,奖励和成本都依赖于任务上下文(例如评估指标),并且在实践中仍然是黑盒。这使得奖励和成本难以建模,因此在决策之前是未知的。另一方面,任务到达行为对不可预测的系统波动等因素仍然敏感,因此对到达分布的先验估计或传统假设(例如泊松分布)可能失效。这意味着另一个实际但经常被忽视的挑战,即不确定的任务到达分布。为了在具有各种不确定性的平稳环境下实现有效调度,我们提出了一种基于双重乐观学习的 Robbins-Monro(DOL-RM)算法。具体来说,DOL-RM 集成了一个学习模块和一个决策模块,前者对奖励与成本之比进行乐观估计,后者利用 Robbins-Monro 方法在做出调度决策的同时隐式学习任务到达分布。理论上,DOL-RM 实现了收敛差距和无遗憾学习,并具有 $O(T^{3/4})$ 的次线性遗憾,这是在不确定任务到达分布以及未知奖励和成本条件下在线任务调度的首个结果。我们在合成实验和真实应用中的数值结果表明,与其他最先进的基线相比,DOL-RM 在实现最佳累积奖励与成本之比方面具有有效性。

英文摘要:

Online task scheduling serves an integral role for task-intensive applications in cloud computing and crowdsourcing. Optimal scheduling can enhance system performance, typically measured by the reward-to-cost ratio, under some task arrival distribution. On one hand, both reward and cost are dependent on task context (e.g., evaluation metric) and remain black-box in practice. These render reward and cost hard to model thus unknown before decision making. On the other hand, task arrival behaviors remain sensitive to factors like unpredictable system fluctuation whereby a prior estimation or the conventional assumption of arrival distribution (e.g., Poisson) may fail. This implies another practical yet often neglected challenge, i.e., uncertain task arrival distribution. Towards effective scheduling under a stationary environment with various uncertainties, we propose a double-optimistic learning based Robbins-Monro (DOL-RM) algorithm. Specifically, DOL-RM integrates a learning module that incorporates optimistic estimation for reward-to-cost ratio and a decision module that utilizes the Robbins-Monro method to implicitly learn task arrival distribution while making scheduling decisions. Theoretically, DOL-RM achieves convergence gap and no regret learning with a sub-linear regret of $O(T^{3/4})$, which is the first result for online task scheduling under uncertain task arrival distribution and unknown reward and cost. Our numerical results in a synthetic experiment and a real-world application demonstrate the effectiveness of DOL-RM in achieving the best cumulative reward-to-cost ratio compared with other state-of-the-art baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑