arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2511.06159cs.DC
收藏

混合强化学习的弹性数据传输优化

Elastic Data Transfer Optimization with Hybrid Reinforcement Learning

Missouri University of Science and Technology(密苏里科技大学)

Rasman Mubtasim Swargo, Md Arifuzzaman

AI总结:

混合强化学习方法实现高效数据传输,通过优化并发性和并行性提升传输效率。

摘要:

现代科学数据采集生成了数百亿字节的数据,必须传输到地理位置遥远的计算集群。传统工具要么依赖预配置会话,这些会话对于没有领域专业知识的用户来说难以调整,要么只适应性地优化并发性,而忽略其他重要的参数。我们提出了\name,一种自适应的数据传输方法,综合考虑了多个参数。我们的解决方案结合了基于启发式的并行性、无限流水线和基于深度强化学习的并发优化器。为了使代理训练实用,我们引入了一个轻量级网络模拟器,将训练时间减少到不到四分钟,并且比在线训练快2750倍。实验评估显示,\name在各种数据集上都优于现有方法,达到比最先进解决方案高9.5倍的吞吐量。

英文摘要:

Modern scientific data acquisition generates petabytes of data that must be transferred to geographically distant computing clusters. Conventional tools either rely on preconfigured sessions, which are difficult to tune for users without domain expertise, or they adaptively optimize only concurrency while ignoring other important parameters. We present \name, an adaptive data transfer method that jointly considers multiple parameters. Our solution incorporates heuristic-based parallelism, infinite pipelining, and a deep reinforcement learning based concurrency optimizer. To make agent training practical, we introduce a lightweight network simulator that reduces training time to less than four minutes and provides a $2750\times$ speedup compared to online training. Experimental evaluation shows that \name consistently outperforms existing methods across diverse datasets, achieving up to 9.5x higher throughput compared to state-of-the-art solutions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑