通过部分监督强化学习在事后可观测 POMDPs 中学习可解释策略
Learning Interpretable Policies in Hindsight-Observable POMDPs through Partially Supervised Reinforcement Learning
AI总结：
提出部分监督强化学习（PSRL）框架，通过融合监督语义状态和无监督潜在表示作为策略网络输入，在部分可观测环境中提升模型可解释性，同时在奖励和收敛速度上保持或显著超越传统方法。
摘要：
深度强化学习在视频游戏、机器人控制、自动驾驶和药物发现等多个领域取得了显著成就。在部分可观测领域，常用方法主要依赖从图像等高维观测中进行端到端学习，而未对真实状态进行显式推理。我们提出了一种替代方向，引入了部分监督强化学习（PSRL）框架。PSRL 的核心是融合监督学习和无监督学习。该方法利用状态估计器从训练时通常完全可观测的高维观测中提取监督语义状态信息。这产生了将状态预测与控制相结合的更可解释的策略。同时，它捕获了无监督的潜在表示。这两者——语义状态和潜在状态——随后被融合并用作策略网络的输入。这种并置为从业者提供了一个灵活且动态的谱系：从强调监督状态信息到整合更丰富的潜在见解。广泛的实验结果表明，通过合并这两种双重表示，PSRL 提供了有力的平衡，在增强模型可解释性的同时，保持并通常在奖励和收敛速度方面显著优于传统方法设定的性能基准。
英文摘要：
Deep reinforcement learning has demonstrated remarkable achievements across diverse domains such as video games, robotic control, autonomous driving, and drug discovery. Common methodologies in partially-observable domains largely lean on end-to-end learning from high-dimensional observations, such as images, without explicitly reasoning about true state. We suggest an alternative direction, introducing the Partially Supervised Reinforcement Learning (PSRL) framework. At the heart of PSRL is the fusion of both supervised and unsupervised learning. The approach leverages a state estimator to distill supervised semantic state information from high-dimensional observations which are often fully observable at training time. This yields more interpretable policies that compose state predictions with control. In parallel, it captures an unsupervised latent representation. These two-the semantic state and the latent state-are then fused and utilized as inputs to a policy network. This juxtaposition offers practitioners a flexible and dynamic spectrum: from emphasizing supervised state information to integrating richer, latent insights. Extensive experimental results indicate that by merging these dual representations, PSRL offers a potent balance, enhancing model interpretability while preserving, and often significantly outperforming, the performance benchmarks set by traditional methods in terms of reward and convergence speed.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
