arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 0906.5325cs.LGcs.MM
收藏

动态多媒体系统的在线强化学习

Online Reinforcement Learning for Dynamic Multimedia Systems

Nicholas Mastronarde, Mihaela van der Schaar

摘要:

在我们以前的工作中,我们提出了一个系统的跨层框架的动态多媒体系统,它允许每一层作出自主和前瞻性的决定,最大限度地提高系统的长期性能,同时满足应用程序的实时延迟约束。所提出的解决方案解决了跨层优化离线,假设多媒体系统的概率动态是已知的先验。然而,在实践中,这些动态是先验未知的,因此必须在线学习。在本文中,我们解决这个问题,让多媒体系统层学习,通过反复相互作用,自主优化系统的长期性能在运行时。我们提出了两种强化学习算法,用于在不同的设计约束下优化系统:第一种算法以集中的方式解决跨层优化,第二种算法以分散的方式解决跨层优化。我们分析这两种算法所需的计算,内存和层间通信开销。在注意到所提出的强化学习算法学习速度太慢之后,我们引入了一种互补的加速学习算法,该算法利用了有关系统动态的部分知识,以显着提高系统的性能。在我们的实验中,我们证明了分散式学习可以像集中式学习一样执行,同时使层能够自主行动。此外,我们表明,现有的应用程序无关的强化学习算法,和现有的近视学习算法部署在多媒体系统中,执行显着不如我们提出的应用程序感知和前瞻性的学习方法。

英文摘要:

In our previous work, we proposed a systematic cross-layer framework for dynamic multimedia systems, which allows each layer to make autonomous and foresighted decisions that maximize the system's long-term performance, while meeting the application's real-time delay constraints. The proposed solution solved the cross-layer optimization offline, under the assumption that the multimedia system's probabilistic dynamics were known a priori. In practice, however, these dynamics are unknown a priori and therefore must be learned online. In this paper, we address this problem by allowing the multimedia system layers to learn, through repeated interactions with each other, to autonomously optimize the system's long-term performance at run-time. We propose two reinforcement learning algorithms for optimizing the system under different design constraints: the first algorithm solves the cross-layer optimization in a centralized manner, and the second solves it in a decentralized manner. We analyze both algorithms in terms of their required computation, memory, and inter-layer communication overheads. After noting that the proposed reinforcement learning algorithms learn too slowly, we introduce a complementary accelerated learning algorithm that exploits partial knowledge about the system's dynamics in order to dramatically improve the system's performance. In our experiments, we demonstrate that decentralized learning can perform as well as centralized learning, while enabling the layers to act autonomously. Additionally, we show that existing application-independent reinforcement learning algorithms, and existing myopic learning algorithms deployed in multimedia systems, perform significantly worse than our proposed application-aware and foresighted learning methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑