arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.17411cs.AI
收藏

通过语义聚类增强深度强化学习的可解释性

Enhancing Interpretability in Deep Reinforcement Learning through Semantic Clustering

College of Information Science University of Arizona(信息科学学院亚利桑那大学)

Liang Zhang, Justin Lieffers, Adarsh Pyarelal

AI总结:

本文提出一种结合特征降维与在线聚类的语义聚类模块,集成到深度强化学习训练流程中,以提高可解释性,并通过新分析方法揭示策略层次结构与特征空间语义组织。

摘要:

本文探讨了深度强化学习(DRL)的语义聚类特性,以提高其可解释性并加深对其内部语义组织的理解。在此背景下,语义聚类指的是神经网络根据输入在特征空间中的语义相似性对输入进行聚类的能力。我们提出了一种DRL架构,该架构包含一个新颖的语义聚类模块,该模块将特征降维与在线聚类相结合。该模块无缝集成到DRL训练流程中,解决了t-SNE的不稳定性,并消除了先前语义分析方法中固有的大量手动标注需求。我们通过实验验证了所提出模块的有效性,并展示了其揭示DRL中语义聚类特性的能力。此外,我们基于这些特性引入了新的分析方法,以深入了解策略的层次结构和特征空间内的语义组织。我们的代码可在https://github.com/ualiangzhang/semantic_rl获取。

英文摘要:

In this paper, we explore semantic clustering properties of deep reinforcement learning (DRL) to improve its interpretability and deepen our understanding of its internal semantic organization. In this context, semantic clustering refers to the ability of neural networks to cluster inputs based on their semantic similarity in the feature space. We propose a DRL architecture that incorporates a novel semantic clustering module that combines feature dimensionality reduction with online clustering. This module integrates seamlessly into the DRL training pipeline, addressing the instability of t-SNE and eliminating the need for extensive manual annotation inherent to prior semantic analysis methods. We experimentally validate the effectiveness of the proposed module and demonstrate its ability to reveal semantic clustering properties within DRL. Furthermore, we introduce new analytical methods based on these properties to provide insights into the hierarchical structure of policies and semantic organization within the feature space. Our code is available at https://github.com/ualiangzhang/semantic_rl.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑