BitTorrent系统中的强化学习
Reinforcement Learning in BitTorrent Systems
摘要：
最近的研究工作表明，流行的BitTorrent协议不提供公平的资源互惠，并可能允许搭便车。在本文中，我们提出了一个类似BitTorrent的协议，取代定期BitTorrent协议的对等选择机制与一种新的强化学习（RL）为基础的机制。由于P2P系统固有的操作，这涉及到重复的互动，在很长一段时间内，对等点可以有效地识别搭便车者以及理想的合作者，通过学习他们的相关对等点的行为。因此，它可以帮助对等点提高他们的下载速率，并阻止搭便车，同时提高系统的公平性。我们将BitTorrent网络中的节点交互建模为重复交互博弈，明确考虑节点的策略行为。应用基于RL的机制的对等体使用相关对等体统计互惠行为的部分观察历史来确定其最佳响应并估计对其预期效用的相应影响。该策略决定了对等体与其他对等体的资源交换，这将最大限度地提高对等体的长期性能，从而做出有远见的决策。我们已经实现了建议的基于学习的机制，并将其纳入现有的BitTorrent客户端。我们在一个受控的Planetlab测试台上进行了大量的实验。我们的研究结果证实，我们提出的协议（1）促进公平的激励方面，每个节点的贡献，例如，高容量的对等体提高他们的下载完成时间高达33\%，（2）提高系统的稳定性和鲁棒性，例如，减少对等体选择波动57\%，和（3）不鼓励搭便车，例如，对等体减少64\%，他们上传到\FR，与常规\BT ~协议相比。
英文摘要：
Recent research efforts have shown that the popular BitTorrent protocol does not provide fair resource reciprocation and may allow free-riding. In this paper, we propose a BitTorrent-like protocol that replaces the peer selection mechanisms in the regular BitTorrent protocol with a novel reinforcement learning (RL) based mechanism. Due to the inherent opration of P2P systems, which involves repeated interactions among peers over a long period of time, the peers can efficiently identify free-riders as well as desirable collaborators by learning the behavior of their associated peers. Thus, it can help peers improve their download rates and discourage free-riding, while improving fairness in the system. We model the peers' interactions in the BitTorrent-like network as a repeated interaction game, where we explicitly consider the strategic behavior of the peers. A peer, which applies the RL-based mechanism, uses a partial history of the observations on associated peers' statistical reciprocal behaviors to determine its best responses and estimate the corresponding impact on its expected utility. The policy determines the peer's resource reciprocations with other peers, which would maximize the peer's long-term performance, thereby making foresighted decisions. We have implemented the proposed reinforcement-learning based mechanism and incorporated it into an existing BitTorrent client. We have performed extensive experiments on a controlled Planetlab test bed. Our results confirm that our proposed protocol (1) promotes fairness in terms of incentives to each peer's contribution e.g. high capacity peers improve their download completion time by up to 33\%, (2) improves the system stability and robustness e.g. reducing the peer selection luctuations by 57\%, and (3) discourages free-riding e.g. peers reduce by 64\% their upload to \FR, in comparison to the regular \BT~protocol.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
