Institute of Systems Science, AMSS, Chinese Academy of Sciences（中国科学院数学与系统科学研究院系统科学研究所）；Nanjing University（南京大学）；Central South University（中南大学）；City University of Hong Kong（香港城市大学）；Washington University in St. Louis（圣路易斯华盛顿大学）
The key approaches for machine learning, especially learning in unknown probabilistic environments are new representations and computation mechanisms. In this paper, a novel quantum reinforcement learning (QRL) method is proposed by combining quantum theory and reinforcement learning (RL). Inspired by the state superposition principle and quantum parallelism, a framework of value updating algorithm is introduced. The state (action) in traditional RL is identified as the eigen state (eigen action) in QRL. The state (action) set can be represented with a quantum superposition state and the eigen state (eigen action) can be obtained by randomly observing the simulated quantum state according to the collapse postulate of quantum measurement. The probability of the eigen action is determined by the probability amplitude, which is parallelly updated according to rewards. Some related characteristics of QRL such as convergence, optimality and balancing between exploration and exploitation are also analyzed, which shows that this approach makes a good tradeoff between exploration and exploitation using the probability amplitude and can speed up learning through the quantum parallelism. To evaluate the performance and practicability of QRL, several simulated experiments are given and the results demonstrate the effectiveness and superiority of QRL algorithm for some complex problems. The present work is also an effective exploration on the application of quantum computation to artificial intelligence.
The development of quantum machine learning (QML) has received a lot of interest recently thanks to developments in both quantum computing (QC) and machine learning (ML). One of the ML paradigms that can be utilized to address challenging sequential decision-making issues is reinforcement learning (RL). It has been demonstrated that classical RL can successfully complete many difficult tasks. A leading method of building quantum RL agents relies on the variational quantum circuits (VQC). However, training QRL algorithms with VQCs requires significant amount of computational resources. This iss…
In recent years, quantum computing (QC) has been getting a lot of attention from industry and academia. Especially, among various QC research topics, variational quantum circuit (VQC) enables quantum deep reinforcement learning (QRL). Many studies of QRL have shown that the QRL is superior to the classical reinforcement learning (RL) methods under the constraints of the number of training parameters. This paper extends and demonstrates the QRL to quantum multi-agent RL (QMARL). However, the extension of QRL to QMARL is not straightforward due to the challenge of the noise intermediate-scale qu…
Towards Heisenberg limit without critical slowing down via quantum reinforcement learning
Shanghai Jiao Tong University（上海交通大学）；Hefei National Laboratory（合肥国家实验室）；Shanghai Research Center for Quantum Sciences（上海量子科学研究中心）；Lenovo Research（联想研究院）
Variational Quantum Circuit-Based Reinforcement Learning for Dynamic Portfolio Optimization
Chair of Information Systems Humboldt University Berlin（信息系统教授 Humboldt 大学柏林）；Department of Mathematics National University of Singapore（数学系 新加坡国立大学）；Bucharest University of Economic Studies（布加勒斯特经济大学）
It's-A-Me, Quantum Mario: Scalable Quantum Reinforcement Learning with Multi-Chip Ensembles
Interdisciplinary Program in Artificial Intelligence（人工智能交叉学科计划）；Computational Science Initiative（计算科学倡议）；Wells Fargo（沃尔斯福戈）；Department of Psychology（心理学系）；Department of Brain and Cognitive Sciences（脑与认知科学系）
Optimistic Reinforcement Learning with Quantile Objectives
Grado Department of Industrial & Systems Engineering, Virginia Tech, Blacksburg, VA 24061, USA（弗吉尼亚理工大学工业与系统工程系）；Department of Industrial, Manufacturing, and Systems Engineering, University of Texas at Arlington, Arlington, TX 76019, USA（德克萨斯理工大学工业、制造与系统工程系）
Recent advances in quantum computing (QC) and machine learning (ML) have drawn significant attention to the development of quantum machine learning (QML). Reinforcement learning (RL) is one of the ML paradigms which can be used to solve complex sequential decision making problems. Classical RL has been shown to be capable to solve various challenging tasks. However, RL algorithms in the quantum world are still in their infancy. One of the challenges yet to solve is how to train quantum RL in the partially observable environments. In this paper, we approach this challenge through building QRL a…
Quantum Reinforcement Learning-Guided Diffusion Model for Image Synthesis via Hybrid Quantum-Classical Generative Model Architectures
Independent Researcher Cambridge, USA（独立研究者 美国剑桥）；Department of Electrophysics National Yang Ming Chiao Tung University Hsinchu, Taiwan（电子物理系 国立阳明交通大学 台北县）
Quantum framework for Reinforcement Learning: Integrating Markov decision process, quantum arithmetic, and trajectory search
Graduate School of Science and Technology, Keio University（Keio大学科学技术研究生院）；Graduate School of Information Science and Electrical Engineering, Kyushu University（九州大学信息科学与电气工程研究生院）；RIKEN Center for Computational Science（理化学研究所计算科学中心）
Challenges in Applying Variational Quantum Algorithms to Dynamic Satellite Network Routing
Department of Telecommunication Engineering, Bonch-Bruevich St. Petersburg State University of Telecommunications（圣彼得堡电信大学电信工程系）；Mathematics, Statistics and Computer Science, University of Wisconsin–Stout（威斯康星州立大学斯普林格分校数学、统计与计算机科学系）
University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）
We introduce the reinforcement quantum annealing (RQA) scheme in which an intelligent agent interacts with a quantum annealer that plays the stochastic environment role of learning automata and tries to iteratively find better Ising Hamiltonians for the given problem of interest. As a proof-of-concept, we propose a novel approach for reducing the NP-complete problem of Boolean satisfiability (SAT) to minimizing Ising Hamiltonians and show how to apply the RQA for increasing the probability of finding the global optimum. Our experimental results on two different benchmark SAT problems (namely f…
University of Innsbruck（因斯布鲁克大学）；Microsoft Research（微软研究院）；Leiden University（莱顿大学）；University of Konstanz（康斯坦茨大学）；Austrian Academy of Sciences（奥地利科学院）
Quantum error correction is widely thought to be the key to fault-tolerant quantum computation. However, determining the most suited encoding for unknown error channels or specific laboratory setups is highly challenging. Here, we present a reinforcement learning framework for optimizing and fault-tolerantly adapting quantum error correction codes. We consider a reinforcement learning agent tasked with modifying a family of surface code quantum memories until a desired logical error rate is reached. Using efficient simulations with about 70 data qubits with arbitrary connectivity, we demonstra…
Quantum computing has a superior advantage in tackling specific problems, such as integer factorization and Simon's problem. For more general tasks in machine learning, by applying variational quantum circuits, more and more quantum algorithms have been proposed recently, especially in supervised learning and unsupervised learning. However, little work has been done in reinforcement learning, arguably more important and challenging. Previous work in quantum reinforcement learning mainly focuses on discrete control tasks where the action space is discrete. In this work, we develop a quantum rei…
Nanjing University（南京大学）；University of New South Wales（新南威尔士大学）
In this paper, a novel training paradigm inspired by quantum computation is proposed for deep reinforcement learning (DRL) with experience replay. In contrast to traditional experience replay mechanism in DRL, the proposed deep reinforcement learning with quantum-inspired experience replay (DRL-QER) adaptively chooses experiences from the replay buffer according to the complexity and the replayed times of each experience (also called transition), to achieve a balance between exploration and exploitation. In DRL-QER, transitions are first formulated in quantum representations, and then the prep…
Quantum machine learning (QML) has received a lot of attention according to its light training parameter numbers and speeds; and the advances of QML lead to active research on quantum multi-agent reinforcement learning (QMARL). Existing classical multi-agent reinforcement learning (MARL) features non-stationarity and uncertain properties. Therefore, this paper presents a simulation software framework for novel QMARL to control autonomous multi-drones, i.e., quantum multi-drone reinforcement learning. Our proposed framework accomplishes reasonable reward convergence and service quality performa…
As one of the latest fields of interest in both academia and industry, quantum computing has garnered significant attention. Among various topics in quantum computing, variational quantum circuits (VQC) have been noticed for their ability to carry out quantum deep reinforcement learning (QRL). This paper verifies the potential of QRL, which will be further realized by implementing quantum multi-agent reinforcement learning (QMARL) from QRL, especially for Internet-connected autonomous multi-robot control and coordination in smart factory applications. However, the extension is not straightforw…
We develop a general method for incentive-based programming of hybrid quantum-classical computing systems using reinforcement learning, and apply this to solve combinatorial optimization problems on both simulated and real gate-based quantum computers. Relative to a set of randomly generated problem instances, agents trained through reinforcement learning techniques are capable of producing short quantum programs which generate high quality solutions on both types of quantum resources. We observe generalization to problems outside of the training set, as well as generalization from the simulat…
Quantum reinforcement learning (QRL) models augment classical reinforcement learning schemes with quantum-enhanced kernels. Different proposals on how to construct such models empirically show a promising performance. In particular, these models might offer a reduced parameter count and shorter times to reach a solution than classical models. It is however presently unclear how these quantum-enhanced kernels as subroutines within a reinforcement learning pipeline need to be constructed to indeed result in an improved performance in comparison to classical models. In this work we exactly addres…
National Taiwan University（台湾大学）；Georgia Institute of Technology（佐治亚理工学院）；IBM Research（IBM研究院）
The state-of-the-art machine learning approaches are based on classical von Neumann computing architectures and have been widely used in many industrial and academic domains. With the recent development of quantum computing, researchers and tech-giants have attempted new quantum circuits for machine learning tasks. However, the existing quantum computing platforms are hard to simulate classical deep learning models or problems because of the intractability of deep quantum circuits. Thus, it is necessary to design feasible quantum algorithms for quantum machine learning for noisy intermediate s…
