发表机构
Benin School of Computer Science and Engineering, The Hebrew University of Jerusalem; Hebrew University Business School, Data Science Department, The Hebrew University of Jerusalem（耶路撒冷希伯来大学贝林计算机科学与工程学院; 耶路撒冷希伯来大学希伯来大学商学院数据科学系）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究针对深度强化学习决策难解释问题，提出模型无关的 SPOT 框架，通过构建采样策略观测树解释策略，在交通信号控制领域验证了其有效性。
AI 中文摘要
深度强化学习（DRL）智能体在复杂环境中表现出色，但其决策过程仍难以解释。我们提出 SPOT（Sampling Policy Observation Tree，采样策略观测树），这是一种新型的模型无关、基于采样的 DRL 策略解释框架。在获取策略和环境模拟器的情况下，SPOT 通过采样动作并递归模拟后续状态，构建出可解释的有限时间范围树。该树提供了策略动作偏好及其可能下游演化的经验表示。我们提供形式化保证，确立 SPOT 渐近恢复策略唯一最可能动作的能力，并刻画高熵策略下的分歧行为。我们在 SUMO-RL 交通信号控制领域验证了 SPOT，案例研究表明，其基于树的表示可用于检查策略偏好、比较替代未来轨迹，并揭示单时间步特征归因方法无法观测的下游行为。
英文摘要
Deep reinforcement learning (DRL) agents achieve strong performance in complex environments, yet their decision-making processes remain difficult to interpret. We introduce SPOT (Sampling Policy Observation Tree), a novel model-agnostic, sampling-based framework for interpreting DRL policies. Given access to the policy and an environment simulator, SPOT constructs an interpretable finite-horizon tree by sampling actions and recursively simulating the resulting successor states. The tree provides an empirical representation of the policy's action preferences and their possible downstream evolution. We provide formal guarantees establishing SPOT's asymptotic recovery of the policy's unique most probable action and characterizing its disagreement behavior under high-entropy policies. We demonstrate SPOT in the SUMO-RL traffic-signal control domain. The case study illustrates how its tree-based representation can be used to inspect policy preferences, compare alternative future trajectories, and reveal downstream behaviors that are not visible through single-timestep feature-attribution methods.
