In this paper, we consider a class of continuous-time, continuous-space stochastic optimal control problems. Building upon recent advances in Markov chain approximation methods and sampling-based algorithms for deterministic path planning, we propose a novel algorithm called the incremental Markov Decision Process (iMDP) to compute incrementally control policies that approximate arbitrarily well an optimal policy in terms of the expected cost. The main idea behind the algorithm is to generate a sequence of finite discretizations of the original problem through random sampling of the state space. At each iteration, the discretized problem is a Markov Decision Process that serves as an incrementally refined model of the original problem. We show that with probability one, (i) the sequence of the optimal value functions for each of the discretized problems converges uniformly to the optimal value function of the original stochastic optimal control problem, and (ii) the original optimal value function can be computed efficiently in an incremental manner using asynchronous value iterations. Thus, the proposed algorithm provides an anytime approach to the computation of optimal control policies of the continuous problem. The effectiveness of the proposed approach is demonstrated on motion planning and control problems in cluttered environments in the presence of process noise.
IA-TIGRIS: An Incremental and Adaptive Sampling-Based Planner for Online Informative Path Planning
Robotics Institute, School of Computer Science, Carnegie Mellon University（卡内基梅隆大学计算机科学学院机器人研究所）；Department of Mechanical Engineering, Brigham Young University（杨百翰大学机械工程系）
Information-Theoretic Stochastic Optimal Control via Incremental Sampling-based Algorithms
Georgia Institute of Technology（佐治亚理工学院）；The Daniel Guggenheim School of Aerospace Engineering（航空航天工程丹尼尔·古根海姆学校）；Institute of Robotics and Intelligent Machines（机器人与智能机器研究所）；Control Systems Laboratory（控制系统实验室）；Autonomous Control and Decision Systems Laboratory（自主控制与决策系统实验室）
An Incremental Sampling and Segmentation-Based Approach for Motion Planning Infeasibility
Robotics Research Center, IIIT Hyderabad（IIIT海得拉巴机器人研究中心）；Department of Informatics, Bioengineering, Robotics, and Systems Engineering, University of Genoa（热那亚大学信息学院、生物工程、机器人与系统工程系）
