arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.00761cs.LGcs.AI
收藏

多任务强化学习中的共享-独特特征与任务感知优先采样

Shared-unique Features and Task-aware Prioritized Sampling on Multi-task Reinforcement Learning

National Taiwan University(台湾大学);National Yang Ming Chiao Tung University(国立阳明交通大学);MobileDrive

Po-Shao Lin, Jia-Fong Yeh, Yi-Ting Chen, Winston H. Hsu

AI总结:

针对多任务强化学习中的性能不平衡问题,提出STARS方法,通过共享-独特特征提取器与任务感知优先采样策略,在Meta-World基准上有效提升整体性能并缓解任务间性能差异。

摘要:

我们观察到,当前最先进(SOTA)方法在执行多任务强化学习(MTRL)任务时存在性能不平衡问题。尽管这些方法平均性能令人瞩目,但在少数任务上表现极差。为此,我们提出一种名为STARS的新有效方法,包含两项新策略:共享-独特特征提取器和任务感知优先采样。首先,共享-独特特征提取器同时学习共享和任务特定特征,以实现不同任务间知识的更好协同。其次,任务感知采样策略与优先经验回放结合,用于在性能较差的任务上高效学习。通过在主流Meta-World基准上的实验,验证了STARS的有效性和稳定性。结果表明,STARS在统计上优于当前SOTA方法,并缓解了性能不平衡问题。此外,我们可视化所学特征以支持我们的主张并增强STARS的可解释性。

英文摘要:

We observe that current state-of-the-art (SOTA) methods suffer from the performance imbalance issue when performing multi-task reinforcement learning (MTRL) tasks. While these methods may achieve impressive performance on average, they perform extremely poorly on a few tasks. To address this, we propose a new and effective method called STARS, which consists of two novel strategies: a shared-unique feature extractor and task-aware prioritized sampling. First, the shared-unique feature extractor learns both shared and task-specific features to enable better synergy of knowledge between different tasks. Second, the task-aware sampling strategy is combined with the prioritized experience replay for efficient learning on tasks with poor performance. The effectiveness and stability of our STARS are verified through experiments on the mainstream Meta-World benchmark. From the results, our STARS statistically outperforms current SOTA methods and alleviates the performance imbalance issue. Besides, we visualize the learned features to support our claims and enhance the interpretability of STARS.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑