能量收集通信中功率分配的在线学习方案
Online Learning Schemes for Power Allocation in Energy Harvesting Communications
摘要：
研究了能量收集通信系统中时变信道中未知信道分布的功率分配问题。在这个问题中，发射机必须基于其电池中存储的能量的量来选择发射功率，目标是最大化随时间获得的平均速率。我们将这个问题建模为马尔可夫决策过程（MDP），其中发射机作为代理，电池状态作为状态，发射功率作为动作，获得的速率作为奖励。MDP上的平均奖励最大化问题可以通过线性规划（LP）来解决，该线性规划使用状态-动作对的转移概率及其奖励值来选择功率分配策略。由于与状态-动作对相关联的奖励是未知的，我们提出了两个在线学习算法：UCLP和Epoch-UCLP，学习这些奖励，并在此过程中调整其策略。UCLP算法在每一步都求解LP，以使用奖励的置信上限来决定其当前策略，而Epoch-UCLP算法将时间划分为epoch，仅在epoch的开始处求解LP，并在该epoch中遵循所获得的策略。我们证明了这两种算法的奖励损失或遗憾的上界常数。与UCLP相比，Epoch-UCLP招致更高的遗憾，但大大减少了计算需求。我们还表明，所提出的算法工作的在线学习成本最小化的问题，如分组调度与功率延迟权衡与微小的变化。
英文摘要：
We consider the problem of power allocation over a time-varying channel with unknown distribution in energy harvesting communication systems. In this problem, the transmitter has to choose the transmit power based on the amount of stored energy in its battery with the goal of maximizing the average rate obtained over time. We model this problem as a Markov decision process (MDP) with the transmitter as the agent, the battery status as the state, the transmit power as the action and the rate obtained as the reward. The average reward maximization problem over the MDP can be solved by a linear program (LP) that uses the transition probabilities for the state-action pairs and their reward values to choose a power allocation policy. Since the rewards associated the state-action pairs are unknown, we propose two online learning algorithms: UCLP and Epoch-UCLP that learn these rewards and adapt their policies along the way. The UCLP algorithm solves the LP at each step to decide its current policy using the upper confidence bounds on the rewards, while the Epoch-UCLP algorithm divides the time into epochs, solves the LP only at the beginning of the epochs and follows the obtained policy in that epoch. We prove that the reward losses or regrets incurred by both these algorithms are upper bounded by constants. Epoch-UCLP incurs a higher regret compared to UCLP, but reduces the computational requirements substantially. We also show that the presented algorithms work for online learning in cost minimization problems like the packet scheduling with power-delay tradeoff with minor changes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
