arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2211.04813cs.LGmath.OC
收藏

Deep W-Networks:用深度强化学习求解多目标优化问题

Deep W-Networks: Solving Multi-Objective Optimisation Problems With Deep Reinforcement Learning

Trinity College Dublin(都柏林圣三一学院)

Jernej Hribar, Luke Hackett, Ivana Dusparic

AI总结:

本文提出Deep W-Networks(DWN)方法,通过用DQN替换Q表并引入W-Networks替代表格权重,将W-learning扩展至大型状态空间,有效解决多目标策略竞争并在基准测试中优于基线且找到Pareto前沿。

摘要:

在本文中,我们基于Deep Q-Networks(DQN)方法引入的进展,将多目标表格型强化学习(RL)算法W-learning扩展到大型状态空间。W-learning算法能够自然地解决多目标环境中多个单一策略之间的竞争。然而,表格版本在具有大型状态空间的环境中扩展性不佳。为解决此问题,我们用DQN替换底层的Q表,并提出增加W-Networks作为表格权重(W)表示的替代。我们在两个广泛接受的多目标RL基准中评估了由此产生的Deep W-Networks(DWN)方法:deep sea treasure和multi-objective mountain car。我们表明,DWN在以DQN解决方案作为基线的情况下表现更优,同时解决了多个策略之间的竞争。此外,我们证明所提出的算法能够在两个测试环境中找到Pareto前沿。

英文摘要:

In this paper, we build on advances introduced by the Deep Q-Networks (DQN) approach to extend the multi-objective tabular Reinforcement Learning (RL) algorithm W-learning to large state spaces. W-learning algorithm can naturally solve the competition between multiple single policies in multi-objective environments. However, the tabular version does not scale well to environments with large state spaces. To address this issue, we replace underlying Q-tables with DQN, and propose an addition of W-Networks, as a replacement for tabular weights (W) representations. We evaluate the resulting Deep W-Networks (DWN) approach in two widely-accepted multi-objective RL benchmarks: deep sea treasure and multi-objective mountain car. We show that DWN solves the competition between multiple policies while outperforming the baseline in the form of a DQN solution. Additionally, we demonstrate that the proposed algorithm can find the Pareto front in both tested environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑