面向按需TV系列的Wi-Fi AP辅助内容预取:一种强化学习方法
Towards Wi-Fi AP-Assisted Content Prefetching for On-Demand TV Series: A Reinforcement Learning Approach
摘要:
配备有巨大存储空间的智能Wi-Fi AP(接入点)的出现开启了关于如何在边缘网络处利用这些资源以提高用户的体验质量(QoE)(例如,短的启动延迟和平滑的回放)。内容预取是这一领域的一个重要研究方向,它可以在用户请求之前预测并准确地获取内容,以便在高峰时段转移流量。然而,在实际应用中,不同的用户之间的视频观看模式,并不断变化的网络连接状态导致时变的服务器负载,这最终使内容预取问题具有挑战性。为了理解这一挑战,本文首先使用真实痕迹对用户的AP连接和电视剧观看模式进行了大规模测量研究。然后,基于所获得的见解,我们制定的内容预取问题作为一个马尔可夫决策过程(MDP)。目标是在不正确预测所引起的增加的预取和存储成本与成功预测所引起的减少的内容下载延迟之间取得平衡。提出了一种基于学习的方法来解决这个问题,并采用另外三种算法作为基线。特别是,首先,我们调查的性能下限使用随机算法,和上限使用理想的离线方法。然后,我们提出了一个启发式算法作为另一个基线。最后,我们设计了一个强化学习算法,更实际的工作在网上的方式。通过广泛的跟踪实验,我们证明了我们的设计的性能增益。值得注意的是,我们基于学习的算法实现了更好的精度和命中率(例如,80%),约70%(分别)。50%)的成本节省相比,随机(resp.启发式)算法。
英文摘要:
The emergence of smart Wi-Fi APs (Access Point), which are equipped with huge storage space, opens a new research area on how to utilize these resources at the edge network to improve users' quality of experience (QoE) (e.g., a short startup delay and smooth playback). One important research interest in this area is content prefetching, which predicts and accurately fetches contents ahead of users' requests to shift the traffic away during peak periods. However, in practice, the different video watching patterns among users, and the varying network connection status lead to the time-varying server load, which eventually makes the content prefetching problem challenging. To understand this challenge, this paper first performs a large-scale measurement study on users' AP connection and TV series watching patterns using real-traces. Then, based on the obtained insights, we formulate the content prefetching problem as a Markov Decision Process (MDP). The objective is to strike a balance between the increased prefetching&storage cost incurred by incorrect prediction and the reduced content download delay because of successful prediction. A learning-based approach is proposed to solve this problem and another three algorithms are adopted as baselines. In particular, first, we investigate the performance lower bound by using a random algorithm, and the upper bound by using an ideal offline approach. Then, we present a heuristic algorithm as another baseline. Finally, we design a reinforcement learning algorithm that is more practical to work in the online manner. Through extensive trace-based experiments, we demonstrate the performance gain of our design. Remarkably, our learning-based algorithm achieves a better precision and hit ratio (e.g., 80%) with about 70% (resp. 50%) cost saving compared to the random (resp. heuristic) algorithm.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。