arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.10352eess.SYcs.LGcs.SY
收藏

无线系统中基于空间探索的最优确定性资源分配的无模型学习

Model-Free Learning of Optimal Deterministic Resource Allocations in Wireless Systems via Action-Space Exploration

Yale University(耶鲁大学)

Hassaan Hashmi, Dionysios S. Kalogerias

摘要:

无线系统资源分配是指永久的和具有挑战性的非凸约束优化任务,这在涉及具有异构目标和不精确的或甚至未知的模型和/或信道统计的多个用户的现代通信和网络设置中特别及时。在本文中,我们提出了一个技术接地和可扩展的原始-对偶确定性政策梯度方法,有效地学习最优参数化资源分配策略。我们的方法不仅有效地利用了流行的通用策略表示(如深度神经网络)的梯度可用性,而且是真正的无模型,因为它依赖于通过动作空间中的低维扰动构建的相关随机网络服务的一致零阶梯度近似,从而完全绕过了对批评者的任何依赖。理论和数值模拟都证实了所提出的方法的有效性和适用性,以及其在实现接近最佳性能和可扩展性方面优于现有技术的优越性。

英文摘要:

Wireless systems resource allocation refers to perpetual and challenging nonconvex constrained optimization tasks, which are especially timely in modern communications and networking setups involving multiple users with heterogeneous objectives and imprecise or even unknown models and/or channel statistics. In this paper, we propose a technically grounded and scalable primal-dual deterministic policy gradient method for efficiently learning optimal parameterized resource allocation policies. Our method not only efficiently exploits gradient availability of popular universal policy representations, such as deep neural networks, but is also truly model-free, as it relies on consistent zeroth-order gradient approximations of the associated random network services constructed via low-dimensional perturbations in action space, thus fully bypassing any dependence on critics. Both theory and numerical simulations confirm the efficacy and applicability of the proposed approach, as well as its superiority over the current state of the art in terms of both achieving near-optimal performance and scalability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑