Existing protocols for benchmarking current quantum co-processors fail to meet the usual standards for assessing the performance of High-Performance-Computing platforms. After a synthetic review of these protocols -- whether at the gate, circuit or application level -- we introduce a new benchmark, dubbed Atos Q-score (TM), that is application-centric, hardware-agnostic and scalable to quantum advantage processor sizes and beyond. The Q-score measures the maximum number of qubits that can be used effectively to solve the MaxCut combinatorial optimization problem with the Quantum Approximate Optimization Algorithm. We give a robust definition of the notion of effective performance by introducing an improved approximation ratio based on the scaling of random and optimal algorithms. We illustrate the behavior of Q-score using perfect and noisy simulations of quantum processors. Finally, we provide an open-source implementation of Q-score that makes it easy to compute the Q-score of any quantum hardware.
Quantum processors with sizes in the 10-100 qubit range are now increasingly common. However, with increased size comes increased complexity for benchmarking. The effectiveness of a given device may vary greatly between different tasks, and will not always be easy to predict from single and two qubit gate fidelities. For this reason, it is important to assess processor quality for a range of important tasks. In this work we propose and implement tests based on random quantum circuits. These are used to evaluate multiple different superconducting qubit devices, with sizes from 5 to 19 qubits, f…
Quantum computers have the potential to outperform classical computers in a range of computational tasks, such as prime factorisation and unstructured searching. However, real-world quantum computers are subject to noise. Quantifying noise is of vital importance, since it is often the dominant factor preventing the successful realisation of advanced quantum computations. Here we propose and demonstrate an interleaved randomised benchmarking protocol for measurement-based quantum computers that can be used to estimate the fidelity of any single-qubit measurement-based gate. We tested the protoc…
In the evaluation of quantum annealers, metrics based on ground state success rates have two major drawbacks. First, evaluation requires computation time for both quantum and classical processors that grows exponentially with problem size. This makes evaluation itself computationally prohibitive. Second, results are heavily dependent on the effects of analog noise on the quantum processors, which is an engineering issue that complicates the study of the underlying quantum annealing algorithm. We introduce a novel "time-to-target" metric which avoids these two issues by challenging software sol…
We evaluate the performance of small error-correcting codes, which we tailor to hardware platforms of very different connectivity and coherence: on a superconducting processor based on transmon qubits and a spintronic quantum register consisting of a nitrogen-vacancy center in diamond. Taking the hardware-specific errors and connectivity into account, we investigate the dependence of the resulting logical error rate on the platform features such as the native gates, native connectivity, gate times, and coherence times. Using a standard error model parameterized for the given hardware, we simul…
We present experimental results on running 4-qubit unstructured search on IBM quantum processors. Our best attempt attained probability of success around 24.5%. We try several algorithms and use the most recent developments in quantum search to reduce the number of entangling gates that are currently considered the main source of errors in quantum computations. Comparing theoretical expectations of an algorithm performance with the actual data, we explore the hardware limits, showing sharp, phase-transition-like degradation of performance on quantum processors. We conclude that it is extremely…
The first generation of small noisy quantum processors have recently become available to non-specialists who are not required to understand specifics of the physical platforms and, in particular, the types and sources of noise. As such, it is useful to benchmark the performance of such computers against specific tasks that may be of interest to users, ideally keeping both the circuit depth and width as free parameters. Here we benchmark the IBM Quantum Experience using the Deterministic Quantum Computing with 1 qubit (DQC1) algorithm originally proposed by Knill and Laflamme in the context of …
Ternary quantum processors offer significant computational advantages over conventional qubit technologies, leveraging the encoding and processing of quantum information in qutrits (three-level systems). To evaluate and compare the performance of such emerging quantum hardware it is essential to have robust benchmarking methods suitable for a higher-dimensional Hilbert space. We demonstrate extensions of industry standard Randomized Benchmarking (RB) protocols, developed and used extensively for qubits, suitable for ternary quantum logic. Using a superconducting five-qutrit processor, we find …
In this letter, we present an experimental benchmark of operational control methods in quantum information processors extended up to 12 qubits. We implement universal control of this large Hilbert space using two complementary approaches and discuss their accuracy and scalability. Despite decoherence, we were able to reach a 12-coherence state (or 12-qubits pseudo-pure cat state), and decode it into an 11 qubit plus one qutrit labeled observable pseudo-pure state using liquid state nuclear magnetic resonance quantum information processors.
Benchmarking neutral atom-based quantum processors at scale
Jülich Supercomputing Centre, Forschungszentrum Jülich（于利希超算中心，于利希研究中心）；RWTH Aachen University（亚琛工业大学）；Department of Computer Science, University of Cologne（科隆大学计算机系）
Experimental demonstration of scalable cross-entropy benchmarking to detect measurement-induced phase transitions on a superconducting quantum processor
We propose network benchmarking: a procedure to efficiently benchmark the quality of a quantum network link connecting quantum processors in a quantum network. This procedure is based on the standard randomized benchmarking protocol and provides an estimate for the fidelity of a quantum network link. We provide statistical analysis of the protocol as well as a simulated implementation inspired by NV-center systems using Netsquid, a special purpose simulator for noisy quantum networks.
We present a continuous monitoring system for intermediate-scale quantum processors that allows extracting estimates of noisy native gate and read-out measurements based on the set of executed quantum circuits and resulting measurement outcomes. In contrast to standard approaches for calibration and benchmarking quantum processors, the executed circuits, which are input to the monitoring system, are assumed to be out of any control. We provide the results of applying our system to the synthetically generated data obtained from a quantum emulator, as well as to the experimental data collected f…
Quantum computers promise to solve certain problems more efficiently than their digital counterparts. A major challenge towards practically useful quantum computing is characterizing and reducing the various errors that accumulate during an algorithm running on large-scale processors. Current characterization techniques are unable to adequately account for the exponentially large set of potential errors, including cross-talk and other correlated noise sources. Here we develop cycle benchmarking, a rigorous and practically scalable protocol for characterizing local and global errors across mult…
Design and Benchmarking of a Quantum Photonic Chip
Rotonium S.r.l.（罗托尼乌斯有限公司）；Centre for Quantum Technologies, National University of Singapore（新加坡国立大学量子技术中心）；Inveriant Pte. Ltd.（因维安特私人有限公司）；Department of Electronics, Information and Bioengineering, Politecnico di Milano（米兰理工大学电子、信息与生物工程系）；Photonic Networks e Technologies National Laboratory, CNIT（意大利国家理工学院光子网络与技术国家实验室）
With the advent of public access to small gate-based quantum processors, it becomes necessary to develop a benchmarking methodology such that independent researchers can validate the operation of these processors. We explore the usefulness of a number of simple quantum circuits as benchmarks for gate-based quantum computing devices and show that circuits performing identity operations are very simple, scalable and sensitive to gate errors and are therefore very well suited for this task. We illustrate the procedure by presenting benchmark results for the IBM Quantum Experience, a cloud-based p…
Estimating the features of noise is the first step in a chain of protocols that will someday lead to fault tolerant quantum computers. The randomized benchmarking (RB) protocol is designed with this exact mindset, estimating the average strength of noise in a quantum processor with relative ease in practice. However, RB, along with most other benchmarking and characterization methods, is limited in scope because it assumes that the noise is temporally uncorrelated (Markovian), which is increasingly evident not to be the case. Here, we combine the RB protocol with a recent framework describing …
We describe a simple randomized benchmarking protocol for quantum information processors and obtain a sequence of models for the observable fidelity decay as a function of a perturbative expansion of the errors. We are able to prove that the protocol provides an efficient and reliable estimate of an average error-rate for a set operations (gates) under a general noise model that allows for both time and gate-dependent errors. We determine the conditions under which this estimate remains valid and illustrate the protocol through numerical examples.
As quantum information processors grow in quantum bit (qubit) count and functionality, the control and measurement system becomes a limiting factor to large scale extensibility. To tackle this challenge and keep pace with rapidly evolving classical control requirements, full control stack access is essential to system level optimization. We design a modular FPGA (field-programmable gate array) based system called QubiC to control and measure a superconducting quantum processing unit. The system includes room temperature electronics hardware, FPGA gateware, and engineering software. A prototype…
Quantum processors promise a paradigm shift in high-performance computing which needs to be assessed by accurate benchmarking measures. In this work, we introduce a new benchmark for variational quantum algorithm (VQA), recently proposed as a heuristic algorithm for small-scale quantum processors. In VQA, a classical optimization algorithm guides the quantum dynamics of the processor to yield the best solution for a given problem. A complete assessment of scalability and competitiveness of VQA should take into account both the quality and the time of dynamics optimization. The method of optima…
PyGSTi is a Python software package for assessing and characterizing the performance of quantum computing processors. It can be used as a standalone application, or as a library, to perform a wide variety of quantum characterization, verification, and validation (QCVV) protocols on as-built quantum processors. We outline pyGSTi's structure, and what it can do, using multiple examples. We cover its main characterization protocols with end-to-end implementations. These include gate set tomography, randomized benchmarking on one or many qubits, and several specialized techniques. We also discuss …
The successful implementation of algorithms on quantum processors relies on the accurate control of quantum bits (qubits) to perform logic gate operations. In this era of noisy intermediate-scale quantum (NISQ) computing, systematic miscalibrations, drift, and crosstalk in the control of qubits can lead to a coherent form of error which has no classical analog. Coherent errors severely limit the performance of quantum algorithms in an unpredictable manner, and mitigating their impact is necessary for realizing reliable quantum computations. Moreover, the average error rates measured by randomi…
Error models for quantum computing processors describe their deviation from ideal behavior and predict the consequences in applications. But those processors' experimental behavior -- the observed outcome statistics of quantum circuits -- are rarely consistent with error models, even in characterization experiments like randomized benchmarking (RB) or gate set tomography (GST), where the error model was specifically extracted from the data in question. We show how to resolve these inconsistencies, and quantify the rate of unmodeled errors, by augmenting error models with a parameterized wildca…
One of the main challenges in building a quantum processor is to characterize the environmental noise. Noise characterization can be achieved by exploiting different techniques, such as randomization where several sequences of random quantum gates are applied to the qubit under test to derive statistical characteristics about the affecting noises. A scalable and robust algorithm able to benchmark the full set of Clifford gates using randomization techniques is called randomized benchmarking. In this study, we simulated randomized benchmarking protocols in a semiconducting all-electrical three-…
Gates in error-prone quantum information processors are often modeled using sets of one- and two-qubit process matrices, the standard model of quantum errors. However, the results of quantum circuits on real processors often depend on additional external "context" variables. Such contexts may include the state of a spectator qubit, the time of data collection, or the temperature of control electronics. In this article we demonstrate a suite of simple, widely applicable, and statistically rigorous methods for detecting context dependence in quantum circuit experiments. They can be used on any d…
On neutral atom platforms, preparing specific quantum states is usually achieved by pulse shaping, i.e., by optimizing the time-dependence of the Hamiltonian related to the system. This process can be extremely costly, as it requires sampling of the final state in the quantum processor many times. Hence, determining a good pulse, as well as a good embedding, to solve specific combinatorial graph problems is one of the most important bottlenecks of the analog approach. In this work, we propose a novel protocol for solving hard combinatorial graph problems that combines variational analog quantu…
State preparation and measurement errors are commonly regarded as indistinguishable. The problem of distinguishing state preparation (SPAM) errors from measurement errors is important to the field of characterizing quantum processors. In this work, we propose a method to separately characterize SPAM errors using a novel type of algorithmic cooling protocol called measurement-based algorithmic cooling (MBAC). MBAC assumes the ability to perform (potentially imperfect) projective measurements on individual qubits, which is available on many modern quantum processors. We demonstrate that MBAC can…
We propose a method to reliably and efficiently extract the fidelity of many-qubit quantum circuits composed of continuously parametrized two-qubit gates called matchgates. This method, which we call matchgate benchmarking, relies on advanced techniques from randomized benchmarking as well as insights from the representation theory of matchgate circuits. We argue the formal correctness and scalability of the protocol, and moreover deploy it to estimate the performance of matchgate circuits generated by two-qubit XY spin interactions on a quantum processor.
Benchmarking methods that can be adapted to multi-qubit systems are essential for assessing the overall or "holistic" performance of nascent quantum processors. The current industry standard is Clifford randomized benchmarking (RB), which measures a single error rate that quantifies overall performance. But scaling Clifford RB to many qubits is surprisingly hard. It has only been performed on 1, 2, and 3 qubits as of this writing. This reflects a fundamental inefficiency in Clifford RB: the $n$-qubit Clifford gates at its core have to be compiled into large circuits over the 1- and 2-qubit gat…
Experimental robustness benchmarking of quantum neural networks on a superconducting quantum processor
Laboratory of Quantum Information, School of Physics, University of Science（量子信息实验室，物理学院，科学大学）；CAS Center For Excellence in Quantum Information（中国科学院量子信息卓越中心）；Quantum Physics, University of Science（量子物理，科学大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, Anhui, 230088, China（人工智能研究所，合肥国家综合性科学中心，合肥，安徽，230088，中国）；Department of ICT（信息与通信技术系）；Center for AI Research, University of Agder (UiA), Jon Lilletuns vei 9, 4879 Grimstad, Norway（人工智能研究中心，阿格德大学（UiA），Jon Lilletuns vei 9，4879 Grimstad，挪威）；Anhui University, Hefei, Anhui, 230039, China（安徽大学，合肥，安徽，230039，中国）；Key Laboratory of System Software (Chinese Academy of Sciences)（系统软件重点实验室（中国科学院））；State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing 100190, China（计算机科学国家重点实验室，软件研究所，中国科学院，北京100190，中国）
As progress is made towards the first generation of error-corrected quantum computers, robust characterization and validation protocols are required to assess the noise environments of physical quantum processors. While standard coherence metrics and characterization protocols such as T1 and T2, process tomography, and randomized benchmarking are now ubiquitous, these techniques provide only partial information about the dynamic multi-qubit loss channels responsible for processor errors, which can be described more fully by a Lindblad operator in the master equation formalism. Here, we introdu…
Silicon spin qubits satisfy the necessary criteria for quantum information processing. However, a demonstration of high fidelity state preparation and readout combined with high fidelity single- and two-qubit gates, all of which must be present for quantum error correction, has been lacking. We use a two qubit Si/SiGe quantum processor to demonstrate state preparation and readout with fidelity over 97%, combined with both single- and two-qubit control fidelities exceeding 99%. The operation of the quantum processor is quantitatively characterized using gate set tomography and randomized benchm…
