The technological development of increasingly larger quantum processors on different quantum platforms raises the problem of how to fairly compare their performance, known as quantum benchmarking of quantum processors. This is a challenge that computer scientists have already faced when comparing classical processors, leading to the development of various mathematical tools to address it, but also to the identification of the limits of this problem. In this work, we briefly review the most important aspects of both classical processor benchmarks and the metrics comprising them, providing precise definitions and analyzing the quality attributes that they should exhibit. Subsequently, we analyze the intrinsic properties that characterize the paradigm of quantum computing and hinder the naive transfer of strategies from classical benchmarking. However, we can still leverage some of the lessons learned such as the quality attributes of a \textit{good} benchmark. Additionally, we review some of the most important metrics and benchmarks for quantum processors proposed in the literature, assessing what quality attributes they fulfill. Finally, we propose general guidelines for quantum benchmarking. These guidelines aim to pave the way for establishing a roadmap towards standardizing the performance evaluation of quantum devices, ultimately leading to the creation of an organization akin to the Standard Performance Evaluation Corporation (SPEC).
Benchmarking is how the performance of a computing system is determined. Surprisingly, even for classical computers this is not a straightforward process. One must choose the appropriate benchmark and metrics to extract meaningful results. Different benchmarks test the system in different ways and each individual metric may or may not be of interest. Choosing the appropriate approach is tricky. The situation is even more open ended for quantum computers, where there is a wider range of hardware, fewer established guidelines, and additional complicating factors. Notably, quantum noise significa…
We present a framework that utilizes quantum algorithms, an architecture aware quantum noise model and an ideal simulator to benchmark quantum computers. The benchmark metrics highlight the difference between the quantum computer evolution and the simulated noisy and ideal quantum evolutions. We utilize our framework for benchmarking three IBMQ systems. The use of multiple algorithms, including continuous-time ones, as benchmarks stresses the computers in different ways highlighting their behaviour for a diverse set of circuits. The complexity of each quantum circuit affects the efficiency of …
The performance of quantum gates is often assessed using some form of randomized benchmarking. However, the existing methods become infeasible for more than approximately five qubits. Here we show how to use a simple and customizable class of circuits -- randomized mirror circuits -- to perform scalable, robust, and flexible randomized benchmarking of Clifford gates. We show that this technique approximately estimates the infidelity of an average many-qubit logic layer, and we use simulations of up to 225 qubits with physically realistic error rates in the range 0.1-1% to demonstrate its scala…
Quantum computers promise to solve certain problems more efficiently than their digital counterparts. A major challenge towards practically useful quantum computing is characterizing and reducing the various errors that accumulate during an algorithm running on large-scale processors. Current characterization techniques are unable to adequately account for the exponentially large set of potential errors, including cross-talk and other correlated noise sources. Here we develop cycle benchmarking, a rigorous and practically scalable protocol for characterizing local and global errors across mult…
Quantum chemistry provides key applications for near-term quantum computing, but these are greatly complicated by the presence of noise. In this work we present an efficient ansatz for the computation of two-electron atoms and molecules within a hybrid quantum-classical algorithm. The ansatz exploits the fundamental structure of the two-electron system, and treating the nonlocal and local degrees of freedom on the quantum and classical computers, respectively. Here the nonlocal degrees of freedom scale linearly with respect to basis-set size, giving a linear ansatz with only $\mathcal{O}(1)$ c…
With the rapid development of quantum hardware technologies, benchmarking the performance of quantum computers has become attractive. In this paper, we propose a new aspect of benchmarking quantum computers by evaluating the limitation of hardware utilization using a multi-programming mechanism -- a technique that simultaneously executes multiple circuits in a quantum machine. This is the first attempt to compare the evaluation of multi-programming on trapped-ion and superconducting devices. Based on the experimental results, performing multi-programming on a trapped-ion device demonstrates be…
Quantum computers have the potential to outperform classical computers in a range of computational tasks, such as prime factorisation and unstructured searching. However, real-world quantum computers are subject to noise. Quantifying noise is of vital importance, since it is often the dominant factor preventing the successful realisation of advanced quantum computations. Here we propose and demonstrate an interleaved randomised benchmarking protocol for measurement-based quantum computers that can be used to estimate the fidelity of any single-qubit measurement-based gate. We tested the protoc…
We use the Variational Quantum Eigensolver (VQE) as implemented in the Qiskit software package to compute the ground state energy of small molecules derived from water, H$_2$O, and hydrogen cyanide, HCN. The work aims to benchmark algorithms for calculating the electronic structure and energy surfaces of molecules of relevance to prebiotic chemistry, beginning with water and hydrogen cyanide, and to run them on the available simulated and physical quantum hardware. The numerical calculations of the algorithms for small quantum processors allow us to design more efficient protocols to be run in…
The increasing scale of near-term quantum hardware motivates the need for efficient noise characterization methods, since qubit and gate level techniques cannot capture crosstalk and correlated noise in many qubit systems. While scalable approaches, such as cycle benchmarking, are known for special classes of quantum circuits, the characterization of noise in general circuits with non-Clifford gates has been an unreachable task. We develop an algorithm that can sample-efficiently estimate the total amount of noise induced by a layer of arbitrary non-Clifford gates, including all crosstalks, an…
This paper presents the benchmark score definitions of QPack, an application-oriented cross-platform benchmarking suite for quantum computers and simulators, which makes use of scalable Quantum Approximate Optimization Algorithm and Variational Quantum Eigensolver applications. Using a varied set of benchmark applications, an insight of how well a quantum computer or its simulator performs on a general NISQ-era application can be quantitatively made. This paper presents what quantum execution data can be collected and transformed into benchmark scores for application-oriented quantum benchmark…
With the advent of public access to small gate-based quantum processors, it becomes necessary to develop a benchmarking methodology such that independent researchers can validate the operation of these processors. We explore the usefulness of a number of simple quantum circuits as benchmarks for gate-based quantum computing devices and show that circuits performing identity operations are very simple, scalable and sensitive to gate errors and are therefore very well suited for this task. We illustrate the procedure by presenting benchmark results for the IBM Quantum Experience, a cloud-based p…
We use a massively parallel simulator of a universal quantum computer to benchmark some of the most powerful supercomputers in the world. We find nearly ideal scaling behavior on the Sunway TaihuLight, the K computer, the IBM BlueGene/Q JUQUEEN, and the Intel Xeon based clusters JURECA and JUWELS. On the Sunway TaihuLight and the K computer, universal quantum computers with up to 48 qubits can be simulated by means of an adaptive two-byte encoding to reduce the memory requirements by a factor of eight. Additionally, we discuss an alternative approach to alleviate the memory bottleneck by decom…
The field of quantum computing has grown from concept to demonstration devices over the past 20 years. Universal quantum computing offers efficiency in approaching problems of scientific and commercial interest, such as factoring large numbers, searching databases, simulating intractable models from quantum physics, and optimizing complex cost functions. Here, we present an 11-qubit fully-connected, programmable quantum computer in a trapped ion system composed of 13 $^{171}$Yb$^{+}$ ions. We demonstrate average single-qubit gate fidelities of 99.5$\%$, average two-qubit-gate fidelities of 97.…
Quantum communication relies on the existence of entanglement between two nodes of a network. However, due to its fragile nature, it is nearly impossible to establish entanglement at large distances through the direct transmission of qubits. Quantum repeaters have been proposed to solve this problem, which split-up the network to create small-scale entangled links and then connect them up to create the large-scale link. As researchers race to establish entanglement over larger and larger distances, it becomes essential to gauge the performance and robustness of the different protocols that hav…
Amplitude Estimation (AE) is a critical subroutine in many quantum algorithms, allowing for a quadratic speedup in various applications like those involving estimating statistics of various functions as in financial Monte Carlo simulations. Much work has gone into devising methods to efficiently estimate the amplitude of a quantum state without expensive operations like the Quantum Fourier Transform (QFT), which is especially prohibitive given the constraints of current NISQ devices. Newer methods have reduced the number of operations required on a quantum computer and are the most promising n…
Quantum computers are on the verge of becoming a commercially available reality. They represent a paradigm shift in computing, with a steep learning gradient. The creation of games is a way to ease the transition for beginners. We present a game similar to the Poker variant Texas hold 'em with the intention to serve as an engaging pedagogical tool to learn the basics rules of quantum computing. The concepts of quantum states, quantum operations and measurement can be learned in a playful manner. The difference to the classical variant is that the community cards are replaced by a quantum regis…
Quantum computers open the possibility of performing real-time calculations for quantum field theory scattering processes. We propose to use an index averaging the absolute value of the difference between the accurately calculated Trotter evolution of site occupations and their actual measurements on NISQ machines. The average is over all the qubits for a certain number of Trotter steps. We use this metric to quantify the progress made in successive state-of-the-art machines and error-mitigation techniques. We illustrate the concept with the transverse Ising model in one spatial dimension with…
We lay the foundation for a benchmarking methodology for assessing current and future quantum computers. We pose and begin addressing fundamental questions about how to fairly compare computational devices at vastly different stages of technological maturity. We critically evaluate and offer our own contributions to current quantum benchmarking efforts, in particular those involving adiabatic quantum computation and the Adiabatic Quantum Optimizers produced by D-Wave Systems, Inc. We find that the performance of D-Wave's Adiabatic Quantum Optimizers scales roughly on par with classical approac…
The Variational Quantum Eigensolver (VQE) is a promising algorithm for Noisy Intermediate Scale Quantum (NISQ) computation. Verification and validation of NISQ algorithms' performance on NISQ devices is an important task. We consider the exactly-diagonalizable Lipkin-Meshkov-Glick (LMG) model as a candidate for benchmarking NISQ computers. We use the Bethe ansatz to construct eigenstates of the trigonometric LMG model using quantum circuits inspired by the LMG's underlying algebraic structure. We construct circuits with depth $\mathcal{O}(N)$ and $\mathcal{O}(\log_2N)$ that can prepare any tri…
Randomized benchmarking (RB) is an important protocol for robustly characterizing the error rates of quantum gates. The technique is typically applied to the Clifford gates since they form a group that satisfies a convenient technical condition of forming a unitary 2-design, in addition to having a tight connection to fault-tolerant quantum computing and an efficient classical simulation. In order to achieve universal quantum computing one must add at least one additional gate such as the T gate (also known as the $π$/8 gate). Here we propose and analyze a simple variation of the standard inte…
We describe how randomized benchmarking can be used to reconstruct the unital part of any trace-preserving quantum map, which in turn is sufficient for the full characterization of any unitary evolution, or more generally, any unital trace-preserving evolution. This approach inherits randomized benchmarking's robustness to preparation, measurement, and gate imperfections, therefore avoiding systematic errors caused by these imperfections. We also extend these techniques to efficiently estimate the average fidelity of a quantum map to unitary maps outside of the Clifford group. The unitaries we…
Center for Quantum Information and Control, University of New Mexico（新墨西哥大学量子信息与控制中心）；Department of Physics and Astronomy, University of New Mexico（新墨西哥大学物理与天文学系）
We propose network benchmarking: a procedure to efficiently benchmark the quality of a quantum network link connecting quantum processors in a quantum network. This procedure is based on the standard randomized benchmarking protocol and provides an estimate for the fidelity of a quantum network link. We provide statistical analysis of the protocol as well as a simulated implementation inspired by NV-center systems using Netsquid, a special purpose simulator for noisy quantum networks.
Quantum chemistry is one of the most promising applications of quantum computers in the near future. For noisy intermediate-scale quantum devices, the quantum-classical hybrid framework based on the variational quantum eigensolver (VQE) has become the method of choice. In the literature, there are many different variants of VQE, but it is not known which one is optimal for a given molecule. For this purpose, we perform a thorough benchmarking on more than ten different kinds of VQE ansatzes (in systems up to 30 qubits), based on their performance on the energy accuracy, runtime until convergen…
