The Markov decision process is the mathematical formalization underlying the modern field of reinforcement learning when transition and reward functions are unknown. We derive a pseudo-Boolean cost function that is equivalent to a K-spin Hamiltonian representation of the discrete, finite, discounted Markov decision process with infinite horizon. This K-spin Hamiltonian furnishes a starting point from which to solve for an optimal policy using heuristic quantum algorithms such as adiabatic quantum annealing and the quantum approximate optimization algorithm on near-term quantum hardware. In proving that the variational minimization of our Hamiltonian is equivalent to the Bellman optimality condition we establish an interesting analogy with classical field theory. Along with proof-of-concept calculations to corroborate our formulation by simulated and quantum annealing against classical Q-Learning, we analyze the scaling of physical resources required to solve our Hamiltonian on quantum hardware.
Harvard University（哈佛大学）；DeepMind（深度思维）；Tsinghua University（清华大学）；University of Houston（休斯顿大学）
Reinforcement learning has recently been used to approach well-known NP-hard combinatorial problems in graph theory. Among these problems, Hamiltonian cycle problems are exceptionally difficult to analyze, even when restricted to individual instances of structurally complex graphs. In this paper, we use Monte Carlo Tree Search (MCTS), the search algorithm behind many state-of-the-art reinforcement learning algorithms such as AlphaZero, to create autonomous agents that learn to play the game of Snake, a game centered on properties of Hamiltonian cycles on grid graphs. The game of Snake can be f…
Azar Aytash Co.（阿扎尔·阿伊塔什公司）；University of Tabriz（大不里士大学）
Graphical Models have various applications in science and engineering which include physics, bioinformatics, telecommunication and etc. Usage of graphical models needs complex computations in order to evaluation of marginal functions,so there are some powerful methods including mean field approximation, belief propagation algorithm and etc. Quantum graphical models have been recently developed in context of quantum information and computation, and quantum statistical physics, which is possible by generalization of classical probability theory to quantum theory. The main goal of this paper is p…
Quantum Markov networks are a generalization of quantum Markov chains to arbitrary graphs. They provide a powerful classification of correlations in quantum many-body systems---complementing the area law at finite temperature---and are therefore useful to understand the powers and limitations of certain classes of simulation algorithms. Here, we extend the characterization of quantum Markov networks and in particular prove the equivalence of positive quantum Markov networks and Gibbs states of Hamiltonians that are the sum of local commuting terms on graphs containing no triangles. For more ge…
We show that finding the lowest eigenvalue of a 3-local symmetric stochastic matrix is QMA-complete. We also show that finding the highest energy of a stoquastic Hamiltonian is QMA-complete and that adiabatic quantum computation using certain excited states of a stoquastic Hamiltonian is universal. We also show that adiabatic evolution in the ground state of a stochastic frustration free Hamiltonian is universal. Our results give a new QMA-complete problem arising in the classical setting of Markov chains, and new adiabatically universal Hamiltonians that arise in many physical systems.
We obtain the analytical expression for the Kraus decomposition of the quantum map of an environment modeled by an arbitrary quadratic fermionic Hamiltonian acting on one or two qubits, and derive simple functions to check the non-positivity of the intermediate map. These functions correspond to two different sufficient criteria for non-Markovianity. In the particular case of an environment represented by the Ising Hamiltonian, we discuss the two sources of non-Markovianity in the model, one due to the finite size of the lattice, and another due to the kind of interactions.
Decoherence in Markovian systems can result indirectly from the action of a system Hamiltonian which is usually fixed and unavoidable. Here, we show that in general in Markovian systems, because of the system Hamiltonian, quantum information decoheres. We give conditions for the system Hamiltonian that must be satisfied if coherence is to be preserved. Finally, we show how to construct robust subspaces for quantum information processing.
We generalize the classical theory of Brownian motion so as to reckon with non-Markovian effects on both Klein-Kramers and Smoluchowski equations. For a free particle and a harmonic oscillator, it is shown that such non-Markovian effects account for the differentiability of the Brownian trajectories as well as the breakdown of the energy equipartition of statistical mechanics at short times in some physical situations. This non-Markovian approach is also extended to look at anomalous diffusion. Next, we bring in the dynamical-quantization method for investigating open quantum systems, which do…
Hamiltonian quantum computing, such as the adiabatic and holonomic models, can be protected against decoherence using an encoding into stabilizer subspace codes for error detection and the addition of energy penalty terms. This method has been widely studied since it was first introduced by Jordan, Farhi, and Shor (JFS) in the context of adiabatic quantum computing. Here we extend the original result to general Markovian environments, not necessarily in Lindblad form. We show that the main conclusion of the original JFS study holds under these general circumstances: assuming a physically reaso…
We introduce the Markovian matrix product density operator, which is a special subclass of the matrix product density operator. We show that the von Neumann entropy of such ansatz can be computed efficiently on a classical computer. This is possible because one can efficiently certify that the global state forms an approximate quantum Markov chain by verifying a set of inequalities. Each of these inequalities can be verified in time that scales polynomially with the bond dimension and the local Hilbert space dimension. The total number of inequalities scale linearly with the system size. We us…
Using the Trotter-Kato theorem we prove the convergence of the unitary dynamics generated by an increasingly singular Hamiltonian in the case of a single field coupling. The limit dynamics is a quantum stochastic evolution of Hudson-Parthasarathy type, and we establish in the process a graph limit convergence of the pre-limit Hamiltonian operators to the Chebotarev-Gregoratti-von Waldenfels Hamiltonian generating the quantum Ito evolution.
In the previous paper, it has been proved that elastic scattering processes of two quantum particles are always accompanied with nonlocal processes. Furthermore, it is found that setting an additional Hamiltonian after the originally scattering one can help to describe the two type of processes in a united frame. Here we discuss the contribution of this additional Hamiltonian to irreversible process in isolated quantum systems. The use of the Hamiltonian can induce the non-Markovian Langevin equation, showing a complex memory effect, and thus revealing the irreversible essence of isolated syst…
Markov chains for probability distributions related to matrix product states and 1D Hamiltonians are introduced. With appropriate 'inverse temperature' schedules, these chains can be combined into a random approximation scheme for ground states of such Hamiltonians. Numerical experiments suggest that a linear, i.e. fast, schedule is possible in non-trivial cases. A natural extension of these chains to 2D settings is next presented and tested. The obtained results compare well with Euclidean evolution. The proposed Markov chains are easy to implement and are inherently sign problem free (even f…
Recently, the idea of classical Metropolis sampling through Markov chains has been generalized for quantum Hamiltonians. However, the underlying Markov chain of this algorithm is still classical in nature. Due to Szegedy's method, the Markov chains of classical Hamiltonians can achieve a quadratic quantum speedup in the eigenvalue gap of the corresponding transition matrix. A natural question to ask is whether Szegedy's quantum speedup is merely a consequence of employing classical Hamiltonians, where the eigenstates simply coincide with the computational basis, making cloning of the classical…
It is argued that the existing schemes of fault-tolerant quantum computation designed for discrete-time models and based on quantum error correction fail for continuous-time Hamiltonian models even with Markovian noise.
A semiclassical approximation for an evolving density operator, driven by a "closed" hamiltonian operator and "open" markovian Lindblad operators, is obtained. The theory is based on the chord function, i.e. the Fourier transform of the Wigner function. It reduces to an exact solution of the Lindblad master equation if the hamiltonian operator is a quadratic function and the Lindblad operators are linear functions of positions and momenta. Initially, the semiclassical formulae for the case of hermitian Lindblad operators are reinterpreted in terms of a (real) double phase space, generated by a…
By using the quantum Yang-Baxterization approach to the three different Hamiltonians, we investigate the behavior of the quantum Fisher information (QFI) under the actions of these Hamiltonians on the different two-qubit input states and by estimating the meaningful parameter $φ$. We address the overall estimation properties by evaluating the QFI for the whole system undergone different unitary evolution. The results show that the behavior of the QFI depends on the choice of the initial states. Choosing the optimal input states can improve the precision of quantum parameter estimation. On the …
A quantum network is an open system consisting of several component Markovian input-output subsystems interconnected by boson field channels carrying quantum stochastic signals. Generalizing the work of Chebotarev and Gregoratti, we formulate the model description by prescribing a candidate Hamiltonian for the network including details the component systems, the field channels, their interconnections, interactions and any time delays arising from the geometry of the network. (We show that the candidate is a symmetric operator and proceed modulo the proof of self-adjointness.) The model is non-…
We consider finite-dimensional Markovian open quantum systems, and characterize the extent to which time-independent Hamiltonian control may allow to stabilize a target quantum state or subspace and optimize the resulting convergence speed. For a generic Lindblad master equation, we introduce a dissipation-induced decomposition of the associated Hilbert space, and show how it serves both as a tool to analyze global stability properties for given control resources and as the starting point to synthesize controls that ensure rapid convergence. The resulting design principles are illustrated in r…
We analyse the problem of estimating a scalar parameter $g$ that controls the Hamiltonian of a quantum system subject to Markovian noise. Specifically, we place bounds on the growth rate of the quantum Fisher information with respect to $g$, in terms of the Lindblad operators and the $g$-derivative of the Hamiltonian $H$. Our new bounds are not only more generally applicable than those in the literature -- for example, they apply to systems with time-dependent Hamiltonians and/or Lindblad operators, and to infinite-dimensional systems such as oscillators -- but are also tighter in the settings…
Conditions sufficient for a quantum dynamical semigroup (QDS) to be unital are proved for a class of problems in quantum optics with Hamiltonians which are self-adjoint polynomials of any finite order in creation and annihilation operators. The order of the Hamiltonian may be higher than the order of completely positive part of the formal generator of a QDS. The unital property of a minimal quantum dynamical semigroup implies the uniqueness of the solution of the corresponding Markov master equation in the class of quantum dynamical semigroups and, in the corresponding representation, it ensur…
We prove that any one-dimensional (1D) quantum state with small quantum conditional mutual information in all certain tripartite splits of the system, which we call a quantum approximate Markov chain, can be well-approximated by a Gibbs state of a short-range quantum Hamiltonian. Conversely, we also derive an upper bound on the (quantum) conditional mutual information of Gibbs states of 1D short-range quantum Hamiltonians. We show that the conditional mutual information between two regions A and C conditioned on the middle region B decays exponentially with the square root of the length of B. …
The Wigner function is known to evolve classically under the exclusive action of a quadratic hamiltonian. If the system does interact with the environment through Lindblad operators that are linear functions of position and momentum, we show that the general evolution is the convolution of the classically evolving Wigner function with a phase space gaussian that broadens in time. We analyze the three generic cases of elliptic, hyperbolic and parabolic Hamiltonians. The Wigner function always becomes positive in a definite time, which is shortest in the hyperbolic case. We also derive an exact …
We propose an effective Hamiltonian approach to investigate decoherence of a quantum system in a non-Markovian reservoir, naturally imposing the complete positivity on the reduced dynamics of the system. The formalism is based on the notion of an effective reservoir, i.e., certain collective degrees of freedom in the reservoir that are responsible for the decoherence. As examples for completely positive decoherence, we present three typical decoherence processes for a qubit such as dephasing, depolarizing, and amplitude-damping. The effects of the non-Markovian decoherence are compared to the …
Path integral quantum Monte Carlo (PIMC) is a method for estimating thermal equilibrium properties of stoquastic quantum spin systems by sampling from a classical Gibbs distribution using Markov chain Monte Carlo. The PIMC method has been widely used to study the physics of materials and for simulated quantum annealing, but these successful applications are rarely accompanied by formal proofs that the Markov chains underlying PIMC rapidly converge to the desired equilibrium distribution. In this work we analyze the mixing time of PIMC for 1D stoquastic Hamiltonians, including disordered transv…
Unveiling coherent dynamics in non-Markovian open quantum systems: exact expression and recursive perturbation expansion
Università degli Studi di Milano（米兰大学）；INFN, Sezione di Milano（意大利国家核物理研究所米兰分部）；Institute of Physics, University of Freiburg（弗赖堡大学物理研究所）；EUCOR Centre for Quantum Science and Quantum Computing, University of Freiburg（弗赖堡大学EUCOR量子科学与量子计算中心）；Naturwissenschaftlich-Technische Fakultät, Universität Siegen（锡根大学自然科学与技术学院）；Física Teòrica: Informació i Fenòmens Quàntics, Department de Física, Universitat Autònoma de Barcelona（巴塞罗那自治大学物理系理论物理：信息与量子现象部门）
Diagonalization of the effective Hamiltonian describing an open quantum system is the usual method of tracking its exceptional points. Although, such a method is successful for tracking EPs in Markovian systems, it may be problematic in non-Markovian systems where a closed expression of the effective Hamiltonian describing the open system may not exist. In this work we provide an alternative method of tracking EPs in open quantum systems, using an experimentally measurable quantity, namely the effective decay rate of a qubit. The quantum system under consideration consists of two non-identical…
In quantum systems theory one of the fundamental problems boils down to: Given an initial state, which final states can be reached by the dynamic system in question? Formulated in the framework of bilinear control systems, the evolution shall be governed by an inevitable Hamiltonian drift term, finitely many control Hamiltonians allowing for (at least) piecewise constant control amplitudes, plus a (possibly bang-bang switchable) noise term in Kossakowski-Lindblad form. Now assuming switchable coupling of finite-dimensional systems to a thermal bath of arbitrary temperature, the core problem of…
We consider the class of quantum stochastic evolutions ($SLH$-models) leading to a quantum dynamical semigroup over a fixed quantum mechanical system (taken to be finite-dimensional). We show that if the semigroup is dissipative, that is, the coupling operators are non-zero, then a dynamical decoupling scheme based on unitary rotations on the system space cannot suppress decoherence even in the limit where the period between pulses vanishes. We emphasize the role of the Fock space dilation used here to construct a quantum stochastic model, as there are often dilations of the same semigroup usi…
We derive a semiclassical approximation for the evolution generated by the Lindblad equation as a generalization of complex WKB theory. Linear coupling to the environment is assumed, but the Hamiltonian can be a general function of positions and momenta. The theory is carried out in the chord representation and describes the evolved quantum characteristic function, which gives direct access to the Wigner function and the position representation of the density operator by Fourier transforms. The propagation is shown to be of Liouville type in a complex double phase space, the imaginary part of …
We propose a scheme for inverse engineering control in open quantum systems. Starting from an undetermined time evolution operator, a time-dependent Hamiltonian is derived in order to guide the system to attain an arbitrary target state at a predefined time. We analyze the fidelity of our control protocol under noise with respect to the stochastic fluctuation of the linear parameters of the Hamiltonian during the time evolution. For a special family of Hamiltonians for two-level systems, we show that the control evolution of the system under noise can be categorized into two standard decoherin…
We consider a lattice of weakly interacting quantum Markov processes. Without interaction, the dynamics at each site is relaxing exponentially to a unique stationary state. With interaction, we show that there remains a unique stationary state in the thermodynamic limit, i.e. absence of phase coexistence, and the relaxation towards it is exponentially fast for local observables. We do not assume that the quantum Markov process is reversible (detailed balance) w.r.t. a local Hamiltonian.
We consider an open quantum system in $M_{d}(\mathbb{C})$ governed by quasiperiodic Hamiltonian with rationally independent frequencies and under assumption of Lyapunov-Perron reducibility of associated Schroedinger equation. We construct the Markovian Master Equation and resulting CP-divisible evolution in weak coupling limit regime, generalizing our previous results from periodic case. The analysis is conducted with application of projection operator techniques and concluded with some results regarding stability of solutions and existence of quasiperiodic global steady state.
In this work we study how the non-Markovian character of the dynamics can affect the thermodynamic performance of a quantum thermal engine, by analysing the maximum power output of Carnot and Otto cycles departing from the quasi-static and infinite-time-thermalization regime respectively, introducing techniques for their control optimization in general dynamical models. In our model, non-Markovianity is introduced by allowing some degrees of freedom of the reservoirs to be taken into account explicitly and share correlations with the engine by Hamiltonian coupling. It is found that the non-Mar…
We show how to obtain perfect samples from a quantum Gibbs state on a quantum computer. To do so, we adapt one of the `Coupling from the Past'-algorithms proposed by Propp and Wilson. The algorithm has a probabilistic run-time and produces perfect samples without any previous knowledge of the mixing time of a quantum Markov chain. To implement it, we assume we are able to perform the phase estimation algorithm for the underlying Hamiltonian and implement a quantum Markov chain such that the transition probabilities between eigenstates only depend on their energy. We provide some examples of qu…
We study the behavior of non-Markovianity with respect to the localization of the initial environmental state. The "amount" of non-Markovianity is measured using divisibility and distinguishability as indicators, employing several schemes to construct the measures. The system used is a qubit coupled to an environment modeled by an Ising spin chain kicked by ultra-short pulses of a magnetic field. In the integrable regime, non-Markovianity and localization do not have a simple relation, but as the chaotic regime is approached, simple relations emerge, which we explore in detail. We also study t…
One-spin and two-spin in a thermodynamic limit spin-1/2 chain as open quantum systems are considered. The dynamics of system is generated by XX Heisenberg interaction and three-spin interaction (TSI) among all the nearest three spins in the chain. Two variety Hamiltonians of the three-spin interaction are considered. Using the fermionization technique and calculating the trace distance, non-Markovianity as a function of the TSI is evaluated, and the results for the two different open quantum systems are compared. In addition, the time behavior of the probability amplitudes are studied. The res…
Using a recently proposed measure for divisibility of a dynamical map, we study the non-Markovian character of a quantum evolution of a driven spin-$S$ system weakly coupled to a bosonic bath. The complete tomographic knowledge about the dynamics of the open system is obtained by the time-convolutionless master equation in the secular approximation. The derived equation can be applied to a wide range of spin-boson models with the Hermitian or non-Hermitian coupling operator in the system-environment interaction Hamiltonian. Besides the influence of the environmental spectral densities, the tun…
The purpose of this paper is to set out the problems of modeling quantum communication and signal processing where the communication between systems via a non-Markovian channel. This is a general feature of quantum transmission lines. Our ultimate objective is to extend the networks rules that have been developed for Markovian models. To this end we recall the Hamiltonian description of such non-Markov models of transmission lines and their quantization. These have occurred in the context of non-quilibrium thermodynamics, but our interest is in the transmission lines as carriers of information…
Starting from the forward and backward infinitesimal generators of bilateral, time-homogeneous Markov processes, the self-adjoint Hamiltonians of the generalized Schroedinger equations are first introduced by means of suitable Doob transformations. Then, by broadening with the aid of the Dirichlet forms the results of the Nelson stochastic mechanics, we prove that it is possible to associate bilateral, and time-homogeneous Markov processes to the wave functions stationary solutions of our generalized Schroedinger equations. Particular attention is then paid to the special case of the Levy-Schr…
