This work presents a multiscale framework to solve an inverse reinforcement learning (IRL) problem for continuous-time/state stochastic systems. We take advantage of a diffusion wavelet representation of the associated Markov chain to abstract the state space. This not only allows for effectively handling the large (and geometrically complex) decision space but also provides more interpretable representations of the demonstrated state trajectories and also of the resulting policy of IRL. In the proposed framework, the problem is divided into the global and local IRL, where the global approximation of the optimal value functions are obtained using coarse features and the local details are quantified using fine local features. An illustrative numerical example on robot path control in a complex environment is presented to verify the proposed method.
In this paper, we propose a generic framework for devising an adaptive approximation scheme for value function approximation in reinforcement learning, which introduces multiscale approximation. The two basic ingredients are multiresolution analysis as well as tree approximation. Starting from simple refinable functions, multiresolution analysis enables us to construct a wavelet system from which the basis functions are selected adaptively, resulting in a tree structure. Furthermore, we present the convergence rate of our multiscale approximation which does not depend on the regularity of basi…
A Reinforcement Learning-Based Model for Mapping and Goal-Directed Navigation Using Multiscale Place Fields
Department of Computer Science, University of Cincinnati（计算机科学系，辛辛那提大学）；Department of Electrical and Computer Engineering, University of Cincinnati（电气与计算机工程系，辛辛那提大学）
The Chinese University of Hong Kong（香港中文大学）；Texas A&M University（得克萨斯农工大学）；North-Eastern Federal University（东北联邦大学）；University of California, Irvine（加州大学欧文分校）
In this work, we propose a multi-agent actor-critic reinforcement learning (RL) algorithm to accelerate the multi-level Monte Carlo Markov Chain (MCMC) sampling algorithms. The policies (actors) of the agents are used to generate the proposal in the MCMC steps; and the critic, which is centralized, is in charge of estimating the long term reward. We verify our proposed algorithm by solving an inverse problem with multiple scales. There are several difficulties in the implementation of this problem by using traditional MCMC sampling. Firstly, the computation of the posterior distribution involv…
Oklahoma State University（俄克拉荷马州立大学）；Norwegian University of Science and Technology（挪威科技大学）；SINTEF Digital（SINTEF数字研究所）
A central challenge in the computational modeling and simulation of a multitude of science applications is to achieve robust and accurate closures for their coarse-grained representations due to underlying highly nonlinear multiscale interactions. These closure models are common in many nonlinear spatiotemporal systems to account for losses due to reduced order representations, including many transport phenomena in fluids. Previous data-driven closure modeling efforts have mostly focused on supervised learning approaches using high fidelity simulation data. On the other hand, reinforcement lea…
Prior studies using graph neural networks (GNNs) for image classification have focused on graphs generated from a regular grid of pixels or similar-sized superpixels. In the latter, a single target number of superpixels is defined for an entire dataset irrespective of differences across images and their intrinsic multiscale structure. On the contrary, this study investigates image classification using graphs generated from an image-specific number of multiscale superpixels. We propose WaveMesh, a new wavelet-based superpixeling algorithm, where the number and sizes of superpixels in an image a…
An efficient wavelet-based physics-informed neural network for multiscale problems
Department of Computational and Data Sciences, Indian Institute of Science, Bangalore, 560012, India（计算与数据科学系，印度科学研究院，班加罗尔，560012，印度）；Department of Mathematics, University of Manchester, Oxford Road, Manchester M13 9PL, UK（数学系，曼彻斯特大学，牛津路，曼彻斯特 M13 9PL，英国）
F10.7 Index Prediction: A Multiscale Decomposition Strategy with Wavelet Transform for Performance Optimization
School of Computer Science, Jiangsu University of Science and Technology（江苏科技大学计算机科学学院）；State Key Laboratory of Space Weather, National Space Science Center, Chinese Academy of Sciences（中国科学院空间天气状态重点实验室）；Yunnan Astronomical Observatory, Chinese Academy of Sciences（中国科学院云南天文台）；Radio Cosmology Lab, Centre for Astronomy and Astrophysics Research, Department of Physics, Faculty of Science, Universiti Malaya（马来亚大学科学学院物理系天文宇宙实验室）；National Centre for Particle Physics, Universiti Malaya（马来亚大学粒子物理研究中心）；Department of Electrical Engineering, Faculty of Engineering, Universiti Malaya（马来亚大学工程学院电子工程系）
