修正：关于带节点丢弃的因子化马尔可夫决策过程的置信策略评估
Corrected: On Confident Policy Evaluation for Factored Markov Decision Processes with Node Dropouts
摘要：
在这项工作中，我们调查的重要性抽样方法评估政策的结构时变因素的马尔可夫决策过程（MDP），即政策的价值估计的高概率置信区间。特别是，我们开始与一个多智能体MDP控制的一个已知的政策，但未知的过渡动态。然后从系统中删除一个代理-即系统经历节点丢失-形成剩余代理的新MDP，具有新的状态空间、动作空间和新的转换动态。我们假设，删除代理的影响对应于其因素的边缘化的过渡动态。奖励功能也可能被边缘化，或者在新系统中完全重新定义。鲁棒策略重要性抽样，然后使用新系统的候选策略进行评估，并估计值的概率置信界。这个计算是在没有观察到新系统的情况下完成的，这意味着可以在dropout发生之前找到安全的策略。这种方法的效用在模拟和Monte Carlo模拟的新系统相比。
英文摘要：
In this work we investigate an importance sampling approach for evaluating policies for a structurally time-varying factored Markov decision process (MDP), i.e. the policy's value is estimated with a high-probability confidence interval. In particular, we begin with a multi-agent MDP controlled by a known policy but with unknown transition dynamics. One agent is then removed from the system - i.e. the system experiences node dropout - forming a new MDP of the remaining agents, with a new state space, action space, and new transition dynamics. We assume that the effect of removing an agent corresponds to the marginalization of its factor in the transition dynamics. The reward function may likewise be marginalized, or it may be entirely redefined for the new system. Robust policy importance sampling is then used to evaluate candidate policies for the new system, and estimated values are presented with probabilistic confidence bounds. This computation is completed with no observations of the new system, meaning that a safe policy may be found before dropout occurs. The utility of this approach is demonstrated in simulation and compared to Monte Carlo simulation of the new system.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
