In this survey, we describe controlled interacting particle systems (CIPS) to approximate the solution of the optimal filtering and the optimal control problems. Part I of the survey is focussed on the feedback particle filter (FPF) algorithm, its derivation based on optimal transportation theory, and its relationship to the ensemble Kalman filter (EnKF) and the conventional sequential importance sampling-resampling (SIR) particle filters. The central numerical problem of FPF -- to approximate the solution of the Poisson equation -- is described together with the main solution approaches. An analytical and numerical comparison with the SIR particle filter is given to illustrate the advantages of the CIPS approach. Part II of the survey is focussed on adapting these algorithms for the problem of reinforcement learning. The survey includes several remarks that describe extensions as well as open problems in this subject.
Feedback particle filter (FPF) is a Monte-Carlo (MC) algorithm to approximate the solution of a stochastic filtering problem. In contrast to conventional particle filters, the Bayesian update step in FPF is implemented via a mean-field type feedback control law. The objective for this paper is to situate the development of FPF and related controlled interacting particle system algorithms within the framework of optimal transportation theory. Starting from the simplest setting of the Bayes' update formula, a coupling viewpoint is introduced to construct particle filters. It is shown that the co…
Probabilistic Data Association-Feedback Particle Filter for Multiple Target Tracking Applications
Coordinated Science Laboratory and the Department of Mechanical Science and Engineering at the University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校协同科学实验室与机械科学与工程系）
In this paper, we relate the feedback capacity of parallel additive colored Gaussian noise (ACGN) channels to a variant of the Kalman filter. By doing so, we obtain lower bounds on the feedback capacity of such channels, as well as the corresponding feedback (recursive) coding schemes, which are essentially power allocation policies with feedback, to achieve the bounds. The results are seen to reduce to existing lower bounds in the case of a single ACGN feedback channel, whereas when it comes to parallel additive white Gaussian noise (AWGN) channels with feedback, the recursive coding scheme r…
In this paper, we relate a feedback channel with any finite-order autoregressive moving-average (ARMA) Gaussian noises to a variant of the Kalman filter. In light of this, we obtain relatively explicit lower bounds on the feedback capacity for such colored Gaussian noises, and the bounds are seen to be consistent with various existing results in the literature. Meanwhile, this variant of the Kalman filter also leads to explicit recursive coding schemes with clear structures to achieve the lower bounds. In general, our results provide an alternative perspective while pointing to potentially tig…
Non-overshooting stabilization is a form of safe control where the setpoint chosen by the user is at the boundary of the safe set. Exponential non-overshooting stabilization, including suitable extensions to systems with deterministic and stochastic disturbances, has been solved by the second author and his coauthors. In this paper we develop homogeneous feedback laws for fixed-time nonovershooting stabilization for nonlinear systems that are input-output linearizable with a full relative degree, i.e., for systems that are diffeomorphically equivalent to the chain of integrators. These homogen…
In real-world applications, we often require reliable decision making under dynamics uncertainties using noisy high-dimensional sensory data. Recently, we have seen an increasing number of learning-based control algorithms developed to address the challenge of decision making under dynamics uncertainties. These algorithms often make assumptions about the underlying unknown dynamics and, as a result, can provide safety guarantees. This is more challenging for other widely used learning-based decision making algorithms such as reinforcement learning. Furthermore, the majority of existing approac…
A major problem in motor control is understanding how the brain plans and executes proper movements in the face of delayed and noisy stimuli. A prominent framework for addressing such control problems is Optimal Feedback Control (OFC). OFC generates control actions that optimize behaviorally relevant criteria by integrating noisy sensory stimuli and the predictions of an internal model using the Kalman filter or its extensions. However, a satisfactory neural model of Kalman filtering and control is lacking because existing proposals have the following limitations: not considering the delay of …
Closed-loop attitude steering can be used to implement a non-standard attitude maneuver by using a conventional attitude control system to track a non-standard attitude profile. The idea has been employed to perform zero-propellant maneuvers on the International Space Station and minimum time maneuvers on NASA's TRACE space telescope. A challenge for operational implementation of the idea is the finite capacity of a space vehicle's command storage buffer. One approach to mitigate the problem is to downsample-and-hold the attitude commands as a set of waypoints for the attitude control system t…
