用于远程电气优化的离策略学习
Off-policy Learning for Remote Electrical Tilt Optimization
摘要：
我们解决的问题，远程电力调度（RET）优化使用的政策上下文多臂强盗（CMAB）技术。RET优化的目标是控制天线的垂直倾斜角的方向，以优化表示蜂窝网络中用户感知的服务质量（QoS）的关键性能指标（KPI）。学习改进的倾斜更新策略是困难的。一方面，在真实网络中以在线方式提出新策略需要探索以前从未使用过的倾斜更新，并且操作风险太大。另一方面，通过模拟来制定这一政策也存在模拟与现实之间的差距。在本文中，我们绕过这些问题，学习改进的政策，在离线的方式使用现有的数据收集在真实的网络。我们制定的问题，制定这样的政策使用的政策CMAB框架。我们提出了CMAB学习算法，从数据中提取最佳的倾斜更新策略。我们在真实的4G长期演进（LTE）蜂窝网络数据上对这些策略进行训练和评估。我们的策略显示出对用于收集数据的基于规则的日志记录策略的一致改进。
英文摘要：
We address the problem of Remote Electrical Tilt (RET) optimization using off-policy Contextual Multi-Armed-Bandit (CMAB) techniques. The goal in RET optimization is to control the orientation of the vertical tilt angle of the antenna to optimize Key Performance Indicators (KPIs) representing the Quality of Service (QoS) perceived by the users in cellular networks. Learning an improved tilt update policy is hard. On the one hand, coming up with a new policy in an online manner in a real network requires exploring tilt updates that have never been used before, and is operationally too risky. On the other hand, devising this policy via simulations suffers from the simulation-to-reality gap. In this paper, we circumvent these issues by learning an improved policy in an offline manner using existing data collected on real networks. We formulate the problem of devising such a policy using the off-policy CMAB framework. We propose CMAB learning algorithms to extract optimal tilt update policies from the data. We train and evaluate these policies on real-world 4G Long Term Evolution (LTE) cellular network data. Our policies show consistent improvements over the rule-based logging policy used to collect the data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
