A new mechanism for efficiently solving the Markov decision processes (MDPs) is proposed in this paper. We introduce the notion of reachability landscape where we use the Mean First Passage Time (MFPT) as a means to characterize the reachability of every state in the state space. We show that such reachability characterization very well assesses the importance of states and thus provides a natural basis for effectively prioritizing states and approximating policies. Built on such a novel observation, we design two new algorithms -- Mean First Passage Time based Value Iteration (MFPT-VI) and Mean First Passage Time based Policy Iteration (MFPT-PI) -- that have been modified from the state-of-the-art solution methods. To validate our design, we have performed numerical evaluations in robotic decision-making scenarios, by comparing the proposed new methods with corresponding classic baseline mechanisms. The evaluation results showed that MFPT-VI and MFPT-PI have outperformed the state-of-the-art solutions in terms of both practical runtime and number of iterations. Aside from the advantage of fast convergence, this new solution method is intuitively easy to understand and practically simple to implement.
Sample-Efficient Learning of Probabilistic Causes for Reachability in Markov Decision Processes with Probabilistic Guarantees
Toyota Research Institute of North America（丰田美国研究院）；Toyota Motor North America Frontier Research Center（丰田北美前沿研究中心）；Toyota Motor Corporation（丰田公司）；Toyota Motor North America（丰田北美）
Bidding Games on Markov Decision Processes with Quantitative Reachability Objectives
University of Haifa（海法大学）；Masaryk University（马萨里克大学）；Institute of Science and Technology Austria (ISTA)（奥地利科学技术研究所）；IMDEA Software Institute（IMDEA软件研究所）
Reachability analysis in stochastic directed graphs by reinforcement learning
Istituto di Analisi dei Sistemi ed Informatica “A. Ruberti”, Consiglio Nazionale delle Ricerche（意大利国家研究委员会“A.鲁贝尔蒂”系统分析与信息研究所）；University of Catania（卡塔尼亚大学）；Politecnico di Torino（都灵理工大学）
