基于抽象MDP的无线网络编码重传：理论与算法
Coded Retransmission in Wireless Networks Via Abstract MDPs: Theory and Algorithms
摘要：
考虑一种在故障广播信道上具有单个发射机和多个接收机的传输方案。对于每个接收器，发送器具有唯一的无限数据包流，其目标是以尽可能高的吞吐量传递数据包。虽然这样的多单播模型一般是未解决的，几个基于网络编码的方案被建议。在这样的方案中，发射机可以发送未编码的分组，或者发送作为几个分组的函数的编码分组。发送的数据包可以被指定的接收器接收（有一定的概率），也可以被其他接收器听到和存储。考虑两种功能模式;第一种假设存储时间是无限的，而在第二种模式中，它受到给定的到期时间（TTE）参数的限制。我们将传输过程建模为无限时域马尔可夫决策过程（MDP）。由于大的状态空间使精确的解决方案计算不切实际，我们引入政策限制和诱导MDPs显着减少状态空间，并证明了与适当的奖励函数，他们有相等的最佳价值函数（因此相等的最佳吞吐量）。然后，我们推导出一个强化学习算法，学习的最优策略诱导MDP。这种诱导MDP的最优策略一旦应用于策略限制的策略，就比未编码的方案显着改善。接下来，我们通过分析所得奖励函数的结构特性来增强算法。我们证明了我们的方法可以很好地扩展用户的数量，并自动适应分组丢失率，事先未知。此外，将性能与Wang最近的约束进行比较，该约束假设更强的编码（例如，会话内和编码分组的缓冲），但被示出是可比较的。
英文摘要：
Consider a transmission scheme with a single transmitter and multiple receivers over a faulty broadcast channel. For each receiver, the transmitter has a unique infinite stream of packets, and its goal is to deliver them at the highest throughput possible. While such multiple-unicast models are unsolved in general, several network coding based schemes were suggested. In such schemes, the transmitter can either send an uncoded packet, or a coded packet which is a function of a few packets. The packets sent can be received by the designated receiver (with some probability) or heard and stored by other receivers. Two functional modes are considered; the first presumes that the storage time is unlimited, while in the second it is limited by a given Time to Expire (TTE) parameter. We model the transmission process as an infinite-horizon Markov Decision Process (MDP). Since the large state space renders exact solutions computationally impractical, we introduce policy restricted and induced MDPs with significantly reduced state space, and prove that with proper reward function they have equal optimal value function (hence equal optimal throughput). We then derive a reinforcement learning algorithm, which learns the optimal policy for the induced MDP. This optimal strategy of the induced MDP, once applied to the policy restricted one, significantly improves over uncoded schemes. Next, we enhance the algorithm by means of analysis of the structural properties of the resulting reward functional. We demonstrate that our method scales well in the number of users, and automatically adapts to the packet loss rates, unknown in advance. In addition, the performance is compared to the recent bound by Wang, which assumes much stronger coding (e.g., intra-session and buffering of coded packets), yet is shown to be comparable.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
