The prediction of interfacial structure in two-phase flow systems is difficult and challenging. In this paper, a novel physics-informed reinforcement learning-aided framework (PIRLF) for the interfacial area transport is proposed. A Markov Decision Process that describes the bubble transport is established by assuming that the development of two-phase flow is a stochastic process with Markov property. The framework aims to capture the complexity of two-phase flow using the advantage of reinforcement learning (RL) in discovering complex patterns with the help of the physical model (Interfacial Area Transport Equation) as reference. The details of the framework design are described including the design of the environment and the algorithm used in solving the RL problem. The performance of the PIRLF is tested through experiments using the experimental database for vertical upward bubbly air-water flows. The result shows a good performance of PIRLF with rRMSE of 6.556%. The case studies on the PIRLF performance also show that the type of reward function that is related to the physical model can affect the framework performance. Based on the study, the optimal reward function is established. The approaches to extending the capability of PIRLF are discussed, which can be a reference for the further development of this methodology.
C-STEP: Continuous Space-Time Empowerment for Physics-informed Safe Reinforcement Learning of Mobile Agents
Faculty of Technology, Bielefeld University, Germany（比勒菲尔德大学技术学院，德国）；Department of Electrical Engineering and Information Technology, Paderborn University, Germany（帕德博恩大学电气工程与信息科技系，德国）
MIT Lincoln Laboratory（麻省理工学院林肯实验室）；Stanford University（斯坦福大学）
This paper proposes a definition of system health in the context of multiple agents optimizing a joint reward function. We use this definition as a credit assignment term in a policy gradient algorithm to distinguish the contributions of individual agents to the global reward. The health-informed credit assignment is then extended to a multi-agent variant of the proximal policy optimization algorithm and demonstrated on particle and multiwalker robot environments that have characteristics such as system health, risk-taking, semi-expendable agents, continuous action spaces, and partial observab…
University of Potsdam（波茨坦大学）；Institute of Mathematics（数学研究所）；Institute of Pharmacy（药剂学研究所）
Model-informed precision dosing (MIPD) using therapeutic drug/biomarker monitoring offers the opportunity to significantly improve the efficacy and safety of drug therapies. Current strategies comprise model-informed dosing tables or are based on maximum a-posteriori estimates. These approaches, however, lack a quantification of uncertainty and/or consider only part of the available patient-specific information. We propose three novel approaches for MIPD employing Bayesian data assimilation (DA) and/or reinforcement learning (RL) to control neutropenia, the major dose-limiting side effect in a…
University of Oxford（牛津大学）；Facebook AI Research（Facebook人工智能研究院）；Massachusetts Institute of Technology（麻省理工学院）；University College London（伦敦大学学院）
To be successful in real-world tasks, Reinforcement Learning (RL) needs to exploit the compositional, relational, and hierarchical structure of the world, and learn to transfer it to the task at hand. Recent advances in representation learning for language make it possible to build models that acquire world knowledge from text corpora and integrate this knowledge into downstream decision making problems. We thus argue that the time is right to investigate a tight integration of natural language understanding into RL in particular. We survey the state of the field, including work on instruction…
Deep reinforcement learning is a technique for solving problems in a variety of environments, ranging from Atari video games to stock trading. This method leverages deep neural network models to make decisions based on observations of a given environment with the goal of maximizing a reward function that can incorporate cost and rewards for reaching goals. With the aim of pathfinding, reward conditions can include reaching a specified target area along with costs for movement. In this work, multiple Deep Q-Network (DQN) agents are trained to operate in a partially observable environment with t…
Model-based reinforcement learning (MBRL) is believed to have much higher sample efficiency compared to model-free algorithms by learning a predictive model of the environment. However, the performance of MBRL highly relies on the quality of the learned model, which is usually built in a black-box manner and may have poor predictive accuracy outside of the data distribution. The deficiencies of the learned model may prevent the policy from being fully optimized. Although some uncertainty analysis-based remedies have been proposed to alleviate this issue, model bias still poses a great challeng…
In deep reinforcement learning (RL) tasks, an efficient exploration mechanism should be able to encourage an agent to take actions that lead to less frequent states which may yield higher accumulative future return. However, both knowing about the future and evaluating the frequentness of states are non-trivial tasks, especially for deep RL domains, where a state is represented by high-dimensional image frames. In this paper, we propose a novel informed exploration framework for deep RL, where we build the capability for an RL agent to predict over the future transitions and evaluate the frequ…
Physics-informed Imitative Reinforcement Learning for Real-world Driving
Robotics and Autonomous Systems Thrust, Systems Hub, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）机器人与自主系统 thrust、系统枢纽）；Department of Computer Science and Engineering, School of Engineering, The Hong Kong University of Science and Technology（香港科技大学计算机科学与工程系、工程学院）
The Enhanced Physics-Informed Kolmogorov-Arnold Networks: Applications of Newton's Laws in Financial Deep Reinforcement Learning (RL) Algorithms
Virginia Polytechnic Institute（弗吉尼亚理工大学）；State University, Blacksburg, USA（州立大学，弗吉尼亚州黑斯堡）；Ho Chi Minh University of Banking, Ho Chi Minh City, Vietnam（胡志明银行大学，胡志明市）
Coordinated Science Laboratory（协同科学实验室）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Making decisions in the presence of a strategic opponent requires one to take into account the opponent's ability to actively mask its intended objective. To describe such strategic situations, we introduce the non-cooperative inverse reinforcement learning (N-CIRL) formalism. The N-CIRL formalism consists of two agents with completely misaligned objectives, where only one of the agents knows the true objective function. Formally, we model the N-CIRL formalism as a zero-sum Markov game with one-sided incomplete information. Through interacting with the more informed player, the less informed p…
Logic-informed reinforcement learning for cross-domain optimization of large-scale cyber-physical systems
State Key Laboratory of Robotics and Intelligent Systems（机器人与智能系统国家重点实验室）；Shenyang Institute of Automation（沈阳自动化研究所）；Chinese Academy of Sciences（中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）
Resolving Action Bottleneck: Agentic Reinforcement Learning Informed by Token-Level Energy
University of Illinois Chicago（伊利诺伊大学芝加哥分校）；Potsdam Institute for Climate Impact Research（波茨坦气候影响研究所）；Technical University of Berlin（柏林技术大学）；University of Southern California（南加州大学）；University of Hong Kong（香港大学）；Broad Institute of MIT and Harvard（MIT和哈佛大学Broad研究所）
Federated Multi-Agent Deep Reinforcement Learning Approach via Physics-Informed Reward for Multi-Microgrid Energy Management
China-Belt and Road Joint Laboratory on Measurement and Control Technology（中国一带一路测量与控制技术联合实验室）；China-EU Institute for Clean and Renewable Energy, Huazhong University of Science and Technology（华中科技大学中欧清洁与可再生能源学院）；School of Electrical Engineering, Northeast Electric Power University（东北电力大学电气工程学院）；University of Victoria（维多利亚大学）
Task Aware Dreamer for Task Generalization in Reinforcement Learning
Department of Computer Science and Technology（计算机科学与技术系）；Beijing National Research Center for Information Science and Technology（北京信息科学国家研究中心）；Tsinghua-Bosch Joint Center for Machine Learning, Institute for Artificial Intelligence（清华大学-博世联合机器学习中心，人工智能研究院）；Tsinghua University（清华大学）
MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）；IAIFI（人工智能与基础交互研究所）
Current model-based reinforcement learning methods struggle when operating from complex visual scenes due to their inability to prioritize task-relevant features. To mitigate this problem, we propose learning Task Informed Abstractions (TIA) that explicitly separates reward-correlated visual features from distractors. For learning TIA, we introduce the formalism of Task Informed MDP (TiMDP) that is realized by training two models that learn visual features via cooperative reconstruction, but one model is adversarially dissociated from the reward signal. Empirical evaluation shows that TIA lead…
Empowering recommender systems using automatically generated Knowledge Graphs and Reinforcement Learning
Insight Research Ireland Centre for Data Analytics, Data Science Institute, University of Galway（爱尔兰高威大学数据科学研究所Insight爱尔兰数据分析研究中心）；Fidelity Investments（富达投资）
Physics-informed neural operator for parametric phase-field modelling of interfacial degradation and microstructural evolution
Tongji University, College of Civil Engineering（同济大学，土木工程学院）；Tongji University, State Key Laboratory of Disaster Reduction in Civil Engineering（同济大学，灾害防治工程重点实验室）
Maluuba, A Microsoft Company（Maluuba，微软旗下公司）；McGill University（麦吉尔大学）
Policy gradient methods in reinforcement learning have become increasingly prevalent for state-of-the-art performance in continuous control tasks. Novel methods typically benchmark against a few key algorithms such as deep deterministic policy gradients and trust region policy optimization. As such, it is important to present and use consistent baselines experiments. However, this can be difficult due to general variance in the algorithms, hyper-parameter tuning, and environment stochasticity. We investigate and discuss: the significance of hyper-parameters in policy gradients for continuous c…
