一种基于多臂Bandit的移动网络运营商选择方法
A Multi-Armed Bandit-based Approach to Mobile Network Provider Selection
摘要：
我们主张让用户能够从任何移动网络运营商临时租用带宽。我们提出，原型，并评估频谱市场的移动网络接入，其中多个网络运营商提供块的带宽在指定的价格为短期租赁给用户，与自主智能体用户设备上作出购买决策的权衡价格，性能和预算限制。我们表明，供应商选择的问题可以制定为一个所谓的强盗问题。对于供应商同步改变价格的情况，我们通过上下文多武装土匪和强化学习方法（如Q学习）来解决问题，这些方法可以直接应用于土匪最大化问题，也可以间接近似Gittins指数，这些指数已知会产生最佳供应商选择策略。对于对应于实际用例的模拟场景，我们的代理在各种需求、价格和移动性条件下显示出比随机提供商选择更好的$20-41\%$ QoE。我们使用LTE网络和eSIM技术实现了一个原型频谱市场，并将其部署在测试平台上，使用区块链来实现记录带宽购买交易的分类账。实验结果表明，该算法能够有效地学习用户行为和网络性能，并在各种竞争代理场景下记录了$25-74\%$的QoE改善情况。
英文摘要：
We argue for giving users the ability to lease bandwidth temporarily from any mobile network operator. We propose, prototype, and evaluate a spectrum market for mobile network access, where multiple network operators offer blocks of bandwidth at specified prices for short-term leases to users, with autonomous agents on user devices making purchase decisions by trading off price, performance, and budget constraints. We show that the problem of provider selection can be formulated as a so-called Bandit problem. For the case where providers change prices synchronously, we approach the problem through contextual multi-armed bandits and Reinforcement Learning methods like Q-learning either applied directly to the bandit maximization problem or indirectly to approximate the Gittins indices that are known to yield the optimal provider selection policy. For a simulated scenario corresponding to a practical use case, our agent shows a $20-41\%$ QoE improvement over random provider selection under various demand, price and mobility conditions. We implemented a prototype spectrum market using LTE networks and eSIM techology and deployed it on a testbed, using a blockchain to implement the ledger where bandwidth purchase transactions are recorded. Experiments showed that we can learn both user behavior and network performance efficiently, and recorded $25-74\%$ improvements in QoE under various competing agent scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
