机制很重要：知识图谱何时有助于强化学习
The Mechanism Matters: When Knowledge Graphs Help Reinforcement Learning
AI总结：
研究探讨知识图谱对强化学习的影响，通过对照研究改变任务、机制和质量，发现结构化KG在特定任务上提升样本效率，其价值与知识量有关，安全性取决于机制，为从业者使用KG指导RL提供具体指导。
AI中文摘要：
知识图谱（KGs）被广泛用于将先验知识注入强化学习（RL），但现有文献多为单领域、积极结果的方法论文。我们进行了一项对照研究，独立改变RL任务、注入机制（状态特征、动作掩码或基于势能的奖励塑造）和KG质量。使用合成的、完全可控的KG在MiniGrid环境上，我们报告了三个发现。首先，在组合稀疏奖励任务上，结构化KG指导提高了样本效率和求解可靠性，洗牌控制会使收益降至基线。其次，KG价值与图中包含的任务相关知识量成比例。第三，安全性取决于机制：软的、保持最优性的注入受益于正确知识并忽略错误知识，而硬掩码很脆弱。我们的结果为从业者提供了具体指导。
英文摘要：
Knowledge graphs (KGs) are widely used to inject prior knowledge into reinforcement learning (RL), yet the literature is dominated by single-domain, positive-result method papers, so we lack a systematic account of when KG structure helps an agent, when it is neutral, and when it hurts. We conduct a controlled study that independently varies the RL task, the injection mechanism (state features, action masking, or potential-based reward shaping), and KG quality. Using a synthetic, fully controllable KG over MiniGrid environments, we report three findings. First, on compositional sparse-reward tasks structured KG guidance improves sample efficiency and solve reliability (70% to 97% of seeds), and a shuffle control that permutes the KG's edges while preserving their count collapses the benefit toward baseline (masking p=0.0001; shaping p=0.006), so the gain is structural rather than generic regularization. Second, KG value scales with the amount of task-relevant knowledge the graph contains. Third, and most consequential, safety depends on the mechanism: soft, optimality-preserving injection benefits from correct knowledge and harmlessly ignores incorrect knowledge, whereas hard masking is brittle, forbidding essential actions when the KG is incomplete or corrupted and making a wrong KG worse than none. A UMLS-derived clinical case study on sepsis management under offline RL is a careful null, underscoring that benefits require task structure the chosen mechanism can exploit. Our results give practitioners concrete guidance on how, and how much, to trust a KG when using it to guide RL.
