Nowadays, model-free reinforcement learning algorithms have achieved remarkable performance on many decision making and control tasks, but high sample complexity and low sample efficiency still hinder the wide use of model-free reinforcement learning algorithms. In this paper, we argue that if we intend to design an intelligent agent that learns fast and transfers well, the agent must be able to reflect key elements of intelligence, like intuition, Memory, PredictionandCuriosity. We propose an agent framework that integrates off-policy reinforcement learning with world model learning, so as to embody the important features of intelligence in our algorithm design. We adopt the state-of-art model-free reinforcement learning algorithm, Soft Actor-Critic, as the agent intuition, and world model learning through RNN to endow the agent with memory, curiosity, and the ability to predict. We show that these ideas can work collaboratively with each other and our agent (RMC) can give new state-of-art results while maintaining sample efficiency and training stability. Moreover, our agent framework can be easily extended from MDP to POMDP problems without performance loss.
We consider the problem of providing users of deep Reinforcement Learning (RL) based systems with a better understanding of when their output can be trusted. We offer an explainable artificial intelligence (XAI) framework that provides a three-fold explanation: a graphical depiction of the systems generalization and performance in the current game state, how well the agent would play in semantically similar environments, and a narrative explanation of what the graphical information implies. We created a user-interface for our XAI framework and evaluated its efficacy via a human-user experiment…
Artificial Intelligence for Climate Adaptation: Reinforcement Learning for Climate Change-Resilient Transport
Department of Technology, Management and Economics, Technical University of Denmark（技术、管理与经济学系，丹麦技术大学）；J.E. Cairnes School of Business and Economics, University of Galway（J.E. Cairnes 商业与经济学学院，Galway大学）
Tsinghua University, Shenzhen International Graduate School（清华大学深圳国际研究生院）；University of California, San Diego（加州大学圣地亚哥分校）；Georgia Institute of Technology（佐治亚理工学院）
Inspired by organisms evolving through cooperation and competition between different populations on Earth, we study the emergence of artificial collective intelligence through massive-agent reinforcement learning. To this end, We propose a new massive-agent reinforcement learning environment, Lux, where dynamic and massive agents in two teams scramble for limited resources and fight off the darkness. In Lux, we build our agents through the standard reinforcement learning algorithm in curriculum learning phases and leverage centralized control via a pixel-to-pixel policy network. As agents co-e…
Deep Reinforcement Learning Agents are not even close to Human Intelligence
Department of Computer Science, Technical University Darmstadt（德累斯顿技术大学计算机科学系）；National Research Center for Applied Cybersecurity (ATHENE)（应用网络安全国家研究中心（ATHENE））；Hessian Center for Artificial Intelligence (hessian.AI)（黑森人工智能中心（hessian.AI））；Centre for Cognitive Science, Darmstadt（达姆施塔特认知科学中心）；Institute of Psychology, Technical University Darmstadt（德累斯顿技术大学心理学系）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心（DFKI））；Sorbonne Université, Paris, France（巴黎索邦大学）
Combination of machine learning (for generating machine intelligence), computer vision (for better environment perception), and robotic systems (for controlled environment interaction) motivates this work toward proposing a vision-based learning framework for intelligent robot control as the ultimate goal (vision-based learning robot). This work specifically introduces deep reinforcement learning as the the learning framework, a General-purpose framework for AI (AGI) meaning application-independent and platform-independent. In terms of robot control, this framework is proposing specifically a …
World Models for Cognitive Agents: Transforming Edge Intelligence in Future Networks
College of Computing and Data Science, Nanyang Technological University（计算与数据科学学院，南洋理工大学）；College of Computer Science and Technology, Jilin University（计算机科学与技术学院，吉林大学）；Department of Electrical and Computer Engineering, Auburn University（电气与计算机工程系，阿伯茨罕大学）；Department of Electrical and Computer Engineering, Sungkyunkwan University（电气与计算机工程系，成均馆大学）
Reinforcement Learning with Comparative Evidence for Social Intelligence
MIT（麻省理工学院）；NUS（新加坡国立大学）；Prince Sattam bin Abdulaziz University（沙特国王大学）；University of Cambridge（剑桥大学）；NTU（南洋理工大学）；UBC（不列颠哥伦比亚大学）；DSO National Laboratories（国防科技局国家实验室）；A*STAR（新加坡科技研究局）
The University of Texas at Dallas（德克萨斯大学达拉斯分校）；Georgia Institute of Technology and Emory University（佐治亚理工学院和埃默里大学）
As artificial intelligence research advances, the platforms used to evaluate AI agents need to adapt and grow to continue to challenge them. We present the Polycraft World AI Lab (PAL), a task simulator with an API based on the Minecraft mod Polycraft World. Our platform is built to allow AI agents with different architectures to easily interact with the Minecraft world, train and be evaluated in multiple tasks. PAL enables the creation of tasks in a flexible manner as well as having the capability to manipulate any aspect of the task during an evaluation. All actions taken by AI agents and ex…
A large set of the explainable Artificial Intelligence (XAI) literature is emerging on feature relevance techniques to explain a deep neural network (DNN) output or explaining models that ingest image source data. However, assessing how XAI techniques can help understand models beyond classification tasks, e.g. for reinforcement learning (RL), has not been extensively studied. We review recent works in the direction to attain Explainable Reinforcement Learning (XRL), a relatively new subfield of Explainable Artificial Intelligence, intended to be used in general public applications, with diver…
From Digital Twins to World Models:Opportunities, Challenges, and Applications for Mobile Edge General Intelligence
State-Province Joint Engineering and Research Center of Advanced Networking and Intelligent Information Services, College of Computer, Northwest University（高级网络与智能信息服务省州联合工程研究中心，计算机学院，西北大学）；College of Computing and Data Science, Nanyang Technological University（计算与数据科学学院，南洋理工大学）；Automation of School, Guangdong University of Technology（自动化学院，广东工业大学）
Continual Learning as Computationally Constrained Reinforcement Learning
Department of Computer Science, Stanford University（计算机科学系，斯坦福大学）；Department of Electrical Engineering, Stanford University（电气工程系，斯坦福大学）；Department of Management Science and Engineering, Stanford University（管理科学与工程系，斯坦福大学）
University of New South Wales（新南威尔士大学）；Deakin University（迪肯大学）；Federation University（联邦大学）；Universidad Central de Chile（智利中央大学）
Explainable artificial intelligence is a research field that tries to provide more transparency for autonomous intelligent systems. Explainability has been used, particularly in reinforcement learning and robotic scenarios, to better understand the robot decision-making process. Previous work, however, has been widely focused on providing technical explanations that can be better understood by AI practitioners than non-expert end-users. In this work, we make use of human-like explanations built from the probability of success to complete the goal that an autonomous robot shows after performing…
Oxford University（牛津大学）；University of Cincinnati（辛辛那提大学）；Australian National University（澳大利亚国立大学）
Algorithmic Information Theory has inspired intractable constructions of general intelligence (AGI), and undiscovered tractable approximations are likely feasible. Reinforcement Learning (RL), the dominant paradigm by which an agent might learn to solve arbitrary solvable problems, gives an agent a dangerous incentive: to gain arbitrary "power" in order to intervene in the provision of their own reward. We review the arguments that generally intelligent algorithmic-information-theoretic reinforcement learners such as Hutter's (2005) AIXI would seek arbitrary power, including over us. Then, usi…
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；School of Intelligent Systems Science and Engineering, Jinan University（暨南大学智能系统科学与工程学院）
The dynamic job-shop scheduling problem (DJSP) is a class of scheduling tasks that specifically consider the inherent uncertainties such as changing order requirements and possible machine breakdown in realistic smart manufacturing settings. Since traditional methods cannot dynamically generate effective scheduling strategies in face of the disturbance of environments, we formulate the DJSP as a Markov decision process (MDP) to be tackled by reinforcement learning (RL). For this purpose, we propose a flexible hybrid framework that takes disjunctive graphs as states and a set of general dispatc…
Task Allocation for Autonomous Machines using Computational Intelligence and Deep Reinforcement Learning
Faculty of Information Technology, Monash University（信息技术学院，墨尔本大学）；Griffith University（格里菲斯大学）；Deakin University（德肯大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；The University of Queensland（昆士兰大学）；Swinburne University of Technology（斯威本技术大学）
On Enhancing Network Throughput using Reinforcement Learning in Sliced Testbeds
Institute of Exact and Technological Sciences -- Federal University of Viçosa (UFV)（维索萨联邦大学精确与技术科学学院（UFV））；Faculty of Computing (FACOM) -- Federal University of Uberlândia (UFU)（乌贝兰迪亚联邦大学计算机学院（UFU））
We present a deep reinforcement learning-based artificial intelligence agent that could provide optimized development plans given a basic description of the reservoir and rock/fluid properties with minimal computational cost. This artificial intelligence agent, comprising of a convolutional neural network, provides a mapping from a given state of the reservoir model, constraints, and economic condition to the optimal decision (drill/do not drill and well location) to be taken in the next stage of the defined sequential field development planning process. The state of the reservoir model is def…
Institut de Recherche en Génie Civil et Mécanique, Ecole Centrale de Nantes（南特中央理工大学土木工程与力学研究所）
Earthquakes are lethal and costly. This study aims at avoiding these catastrophic events by the application of injection policies retrieved through reinforcement learning. With the rapid growth of artificial intelligence, prediction-control problems are all the more tackled by function approximation models that learn how to control a specific task, even for systems with unmodeled/unknown dynamics and important uncertainties. Here, we show for the first time the possibility of controlling earthquake-like instabilities using state-of-the-art deep reinforcement learning techniques. The controller…
We are at the cusp of a transition from "learning from data" to "learning what data to learn from" as a central focus of artificial intelligence (AI) research. While the first-order learning problem is not completely solved, large models under unified architectures, such as transformers, have shifted the learning bottleneck from how to effectively train our models to how to effectively acquire and use task-relevant data. This problem, which we frame as exploration, is a universal aspect of learning in open-ended domains, such as the real world. Although the study of exploration in AI is largel…
In recent years, a wide range of investment models have been created using artificial intelligence. Automatic trading by artificial intelligence can expand the range of trading methods, such as by conferring the ability to operate 24 hours a day and the ability to trade with high frequency. Automatic trading can also be expected to trade with more information than is available to humans if it can sufficiently consider past data. In this paper, we propose an investment agent based on a deep reinforcement learning model, which is an artificial intelligence model. The model considers the transact…
Enhancing IoT Intelligence: A Transformer-based Reinforcement Learning Methodology
Aqaba University of Technology（阿卡巴理工大学）；Concordia University（康考迪亚大学）；Khalifa University（哈利法大学）；The University of Jordan（约旦大学）；Applied Science Private University（应用科学私立大学）；King Fahd University of Petroleum & Minerals（法赫德国王石油与矿业大学）
Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）；Tsinghua University（清华大学）；East China Normal University（华东师范大学）；Nantong University（南通大学）；National Mobile Communications Research Laboratory（移动通信国家重点实验室）；Southeast University（东南大学）；National Key Laboratory on Communications（通信抗干扰技术国家级重点实验室）；University of Electronic Science and Technology of China（电子科技大学）
Mobile edge computing (MEC) is considered a novel paradigm for computation-intensive and delay-sensitive tasks in fifth generation (5G) networks and beyond. However, its uncertainty, referred to as dynamic and randomness, from the mobile device, wireless channel, and edge network sides, results in high-dimensional, nonconvex, nonlinear, and NP-hard optimization problems. Thanks to the evolved reinforcement learning (RL), upon iteratively interacting with the dynamic and random environment, its trained agent can intelligently obtain the optimal policy in MEC. Furthermore, its evolved versions, …
Graph-enabled Reinforcement Learning for Time Series Forecasting with Adaptive Intelligence
School of Mathematics, Physics & Computing, University of Southern Queensland（数学、物理与计算学院，南方昆士兰大学）；Department of Computing and Decision Sciences, Lingnan University（计算与决策科学系，岭南大学）；School of Computer and Artificial Intelligence, Wuhan University of Technology（计算机与人工智能学院，武汉科技大学）；School of Business at the University of Southern Queensland（昆士兰大学商学院）；School of Computer Science, Queensland University of Technology（计算机科学学院，昆士兰技术大学）
Simulation of Nanorobots with Artificial Intelligence and Reinforcement Learning for Advanced Cancer Cell Detection and Tracking
Dentofacial Deformities Research Center（牙颌面畸形研究中心）；Research Institute of Dental Sciences（牙科科学研究所）；Shahid Beheshti University of Medical Sciences（沙希德·贝赫什提医科大学）
Warehouse Management Systems have been evolving and improving thanks to new Data Intelligence techniques. However, many current optimizations have been applied to specific cases or are in great need of manual interaction. Here is where Reinforcement Learning techniques come into play, providing automatization and adaptability to current optimization policies. In this paper, we present Storehouse, a customizable environment that generalizes the definition of warehouse simulations for Reinforcement Learning. We also validate this environment against state-of-the-art reinforcement learning algori…
The U.S. Securities and Exchange Commission（美国证券交易委员会）
After generalizing the Archimedean property of real numbers in such a way as to make it adaptable to non-numeric structures, we demonstrate that the real numbers cannot be used to accurately measure non-Archimedean structures. We argue that, since an agent with Artificial General Intelligence (AGI) should have no problem engaging in tasks that inherently involve non-Archimedean rewards, and since traditional reinforcement learning rewards are real numbers, therefore traditional reinforcement learning probably will not lead to AGI. We indicate two possible ways traditional reinforcement learnin…
