Quantum information processing often requires the preparation of arbitrary quantum states, such as all the states on the Bloch sphere for two-level systems. While numerical optimization can prepare individual target states, they lack the ability to find general solutions that work for a large class of states in more complicated quantum systems. Here, we demonstrate global quantum control by preparing a continuous set of states with deep reinforcement learning. The protocols are represented using neural networks, which automatically groups the protocols into similar types, which could be useful for finding classes of protocols and extracting physical insights. As application, we generate arbitrary superposition states for the electron spin in complex multi-level nitrogen-vacancy centers, revealing classes of protocols characterized by specific preparation timescales. Our method could help improve control of near-term quantum computers, quantum sensing devices and quantum simulations.
Accurate and efficient preparation of quantum state is a core issue in building a quantum computer. In this paper, we investigate how to prepare a certain single- or two-qubit target state from arbitrary initial states in semiconductor double quantum dots with the aid of deep reinforcement learning. Our method is based on the training of the network over numerous preparing tasks. The results show that once the network is well trained, it works for any initial states in the continuous Hilbert space. Thus repeated training for new preparation tasks is avoided. Our scheme outperforms the traditio…
Quantum control has been of increasing interest in recent years, e.g. for tasks like state initialization and stabilization. Feedback-based strategies are particularly powerful, but also hard to find, due to the exponentially increased search space. Deep reinforcement learning holds great promise in this regard. It may provide new answers to difficult questions, such as whether nonlinear measurements can compensate for linear, constrained control. Here we show that reinforcement learning can successfully discover such feedback strategies, without prior knowledge. We illustrate this for state p…
Deep reinforcement learning for near-deterministic preparation of cubic- and quartic-phase gates in photonic quantum computing
Department of Physics, University of Virginia（弗吉尼亚大学物理系）；Charles L. Brown Department of Electrical and Computer Engineering, University of Virginia（弗吉尼亚大学查尔斯·L·布朗电气与计算机工程系）
Reinforcement learning has been widely used in many problems, including quantum control of qubits. However, such problems can, at the same time, be solved by traditional, non-machine-learning methods, such as stochastic gradient descent and Krotov algorithms, and it remains unclear which one is most suitable when the control has specific constraints. In this work, we perform a comparative study on the efficacy of three reinforcement learning algorithms: tabular Q-learning, deep Q-learning, and policy gradient, as well as two non-machine-learning methods: stochastic gradient descent and Krotov …
We propose a method based on deep reinforcement learning that efficiently prepares a quantum many-body pure state in thermal or prethermal equilibrium. The main physical intuition underlying the method is that the information on the equilibrium states can be efficiently encoded/extracted by focusing on only a few local observables, relying on the typicality of equilibrium states. Instead of resorting to the expensive preparation protocol that adopts global features such as the quantum state fidelity, we show that the equilibrium states can be efficiently prepared only by learning the expectati…
I demonstrate the potential of reinforcement learning (RL) to prepare quantum states of strongly periodically driven non-linear single-particle models. The ability of Q-Learning to control systems far away from equilibrium is exhibited by steering the quantum Kapitza oscillator to the Floquet-engineered stable inverted position in the presence of a strong periodic drive within several shaking cycles. The study reveals the potential of the intra-period (micromotion) dynamics, often neglected in Floquet engineering, to take advantage over pure stroboscopic control at moderate drive frequencies. …
An energy gap develops near quantum critical point of quantum phase transition in a finite many-body (MB) system, facilitating the ground state transformation by adiabatic parameter change. In real application scenarios, however, the efficacy for such a protocol is compromised by the need to balance finite system life time with adiabaticity, as exemplified in a recent experiment that prepares three-mode balanced Dicke state near deterministically [PNAS {\bf 115}, 6381 (2018)]. Instead of tracking the instantaneous ground state as unanimously required for most adiabatic crossing, this work repo…
University of California, Berkeley（加州大学伯克利分校）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）；St. Kliment Ohridski University of Sofia（圣基里尔·奥赫里德斯基索菲亚大学）
The quantum alternating operator ansatz (QAOA) is a prominent example of variational quantum algorithms. We propose a generalized QAOA called CD-QAOA, which is inspired by the counterdiabatic driving procedure, designed for quantum many-body systems and optimized using a reinforcement learning (RL) approach. The resulting hybrid control algorithm proves versatile in preparing the ground state of quantum-chaotic many-body spin chains by minimizing the energy. We show that using terms occurring in the adiabatic gauge potential as generators of additional control unitaries, it is possible to achi…
Nanjing University（南京大学）；University of New South Wales（新南威尔士大学）
In this paper, a novel training paradigm inspired by quantum computation is proposed for deep reinforcement learning (DRL) with experience replay. In contrast to traditional experience replay mechanism in DRL, the proposed deep reinforcement learning with quantum-inspired experience replay (DRL-QER) adaptively chooses experiences from the replay buffer according to the complexity and the replayed times of each experience (also called transition), to achieve a balance between exploration and exploitation. In DRL-QER, transitions are first formulated in quantum representations, and then the prep…
Automating Variational Quantum Sensing through Reinforcement-Learned Circuit Structures
City University of Hong Kong（香港城市大学）；City University of Hong Kong Shenzhen Research Institute（香港城市大学深圳研究院）；Quantum Science Center of Guangdong-Hong Kong-Macao Greater Bay Area（粤港澳大湾区量子科学中心）
Towards Heisenberg limit without critical slowing down via quantum reinforcement learning
Shanghai Jiao Tong University（上海交通大学）；Hefei National Laboratory（合肥国家实验室）；Shanghai Research Center for Quantum Sciences（上海量子科学研究中心）；Lenovo Research（联想研究院）
Machine learning (ML) has become an attractive tool in information processing, however few ML algorithms have been successfully applied in the quantum domain. We show here how classical reinforcement learning (RL) could be used as a tool for quantum state engineering (QSE). We employ a measurement based control for QSE where the action sequences are determined by the choice of the measurement basis and the reward through the fidelity of obtaining the target state. Our analysis clearly displays a learning feature in QSE, for example in preparing arbitrary two-qubit entangled states. It delivers…
University of Science and Technology of China（中国科学技术大学）；University of New South Wales（新南威尔士大学）
Due to its property of not requiring prior knowledge of the environment, reinforcement learning has significant potential for quantum control problems. In this work, we investigate the effectiveness of continuous control policies based on deep deterministic policy gradient. To solve the sparse reward signal in quantum learning control problems, we propose an auxiliary task-based deep reinforcement learning (AT-DRL) for quantum control. In particular, we first design a guided reward function based on the fidelity of quantum states that enables incremental fidelity improvement. Then, we introduc…
Quantum control of solid state qubits via Reinforcement Learning
Ulm University（乌尔姆大学）；Deutsches Zentrum für Luft- und Raumfahrt e.V. (DLR)（德国航空航天中心）；Center for Integrated Quantum Science and Technology (IQST) Ulm-Stuttgart, Germany（乌尔姆-斯图加特集成量子科学与技术中心）
Reinforcement learning for ion shuttling on trapped-ion quantum computers
Institute for Information Processing (tnt), L3S, Leibniz University Hannover, Germany；Institute for Gravitational Physics, Leibniz University Hannover, Germany；QUDORA Technologies GmbH；Institute for Theoretical Physics, Leibniz University Hannover, Germany
Direct fidelity estimation through joint fiducial grouping
ICFO - Institut de Ciències Fotòniques, The Barcelona Institute of Science and Technology（光子科学研究所，巴塞罗那科技研究院）；Centre for Quantum Technologies, National University of Singapore（量子技术中心，新加坡国立大学）
National Institute for Theoretical Physics（国家理论物理研究所）
Quantum machine learning witnesses an increasing amount of quantum algorithms for data-driven decision making, a problem with potential applications ranging from automated image recognition to medical diagnosis. Many of those algorithms are implementations of quantum classifiers, or models for the classification of data inputs with a quantum computer. Following the success of collective decision making with ensembles in classical machine learning, this paper introduces the concept of quantum ensembles of quantum classifiers. Creating the ensemble corresponds to a state preparation routine, aft…
Optimal control is highly desirable in many current quantum systems, especially to realize tasks in quantum information processing. We introduce a method based on differentiable programming to leverage explicit knowledge of the differential equations governing the dynamics of the system. In particular, a control agent is represented as a neural network that maps the state of the system at a given time to a control pulse. The parameters of this agent are optimized via gradient information obtained by direct differentiation through both the neural network \emph{and} the differential equation of …
Reinforcement Learning to Disentangle Multiqubit Quantum States from Partial Observations
Department of Mathematics and Informatics（数学与信息学系）；St. Kliment Ohridski University of Sofia（索菲亚圣克莱门特·欧赫里迪斯大学）；Joint Quantum Institute（联合量子研究所）；Department of Physics（物理系）；University of Maryland（马里兰大学）；National Quantum Laboratory (QLab)（国家量子实验室（QLab））；Institute of Physics（物理研究所）；École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院（EPFL））；Center for Quantum Science and Engineering（量子科学与工程中心）；Quantum Systems Unit（量子系统单元）；OIST Graduate University（OIST研究生大学）；Duke Quantum Center（杜克量子中心）；Max Planck Institute for the Physics of Complex Systems（复杂系统物理研究所）
Quantum Logic Gate Synthesis as a Markov Decision Process
Rigetti Computing（里盖蒂计算公司）；Quantum Artificial Intelligence Laboratory (QuAIL), NASA Ames Research Center（美国国家航空航天局艾姆斯研究中心量子人工智能实验室）；USRA Research Institute for Advanced Computer Science (RIACS)（美国空间研究协会高级计算机科学研究所）；University of Maryland（马里兰大学）；Ames Laboratory（艾姆斯实验室）；Iowa State University（爱荷华州立大学）；Department of Physics and Astronomy, Iowa State University（爱荷华州立大学物理与天文学系）
Learning Unified Control of Intrinsic Nonlinear Spin Dynamics in Atomic Qudits for Magnetometry
College of Physics, Nanjing University of Aeronautics（南京航空航天大学物理学院）；China Mobile (Suzhou) Software Technology Co., Ltd.（中国移动（苏州）软件技术有限公司）；Shanghai Institute of Optics and Fine Mechanics（上海光学精密机械研究所）；Chinese Academy of Sciences（中国科学院）；Key Laboratory of Aerospace Information Sensing（航空航天信息感知重点实验室）
