arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.08879cs.LGcs.AIcs.MAmath.OC
收藏

使用自适应分布式强化学习的多目标优化

Multi-Objective Optimization Using Adaptive Distributed Reinforcement Learning

Huawei Technology Munich Research Center(华为技术慕尼黑研究中心);Hasso Plattner Institute, University of Potsdam(波茨坦大学哈索·普拉特纳研究所)

Jing Tan, Ramin Khalili, Holger Karl

AI总结:

该研究提出一种多目标多智能体强化学习算法,在动态分布式ITS环境中实现自适应小样本学习,并在边缘云计算场景下优于现有基准,且推理仅需6毫秒。

摘要:

智能交通系统(ITS)环境以动态和分布式著称,其中参与者(车辆用户、运营商等)具有多重、变化且可能相互冲突的目标。尽管强化学习(RL)算法通常被应用于优化ITS应用,如资源管理和卸载,但大多数RL算法专注于单目标。在许多情况下,将多目标问题转换为单目标问题是不可能的、难以处理的或不充分的,这使得此类RL算法不适用。我们提出了一种具有高学习效率和低计算需求的多目标、多智能体强化学习(MARL)算法,该算法在动态、分布式和嘈杂的环境中,在稀疏和延迟奖励的情况下,自动触发自适应的小样本学习。我们在具有边缘云计算的ITS环境中测试了我们的算法。实证结果表明,该算法能够快速适应新环境,并在所有个体和系统指标上均优于最先进的基准。我们的算法还通过其模块化和异步在线训练方法解决了各种实际问题。除了云模拟外,我们还在单板计算机上测试了我们的算法,并表明它可以在6毫秒内进行推理。

英文摘要:

The Intelligent Transportation System (ITS) environment is known to be dynamic and distributed, where participants (vehicle users, operators, etc.) have multiple, changing and possibly conflicting objectives. Although Reinforcement Learning (RL) algorithms are commonly applied to optimize ITS applications such as resource management and offloading, most RL algorithms focus on single objectives. In many situations, converting a multi-objective problem into a single-objective one is impossible, intractable or insufficient, making such RL algorithms inapplicable. We propose a multi-objective, multi-agent reinforcement learning (MARL) algorithm with high learning efficiency and low computational requirements, which automatically triggers adaptive few-shot learning in a dynamic, distributed and noisy environment with sparse and delayed reward. We test our algorithm in an ITS environment with edge cloud computing. Empirical results show that the algorithm is quick to adapt to new environments and performs better in all individual and system metrics compared to the state-of-the-art benchmark. Our algorithm also addresses various practical concerns with its modularized and asynchronous online training method. In addition to the cloud simulation, we test our algorithm on a single-board computer and show that it can make inference in 6 milliseconds.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑