TensorFlow Quantum: A Software Framework for Quantum Machine Learning Michael Broughton,1, 5, ∗ Guillaume Verdon,1, 2, 4, 6, y Trevor McCourt,1, 7 Antonio J. Martinez,1, 2, 4, 8 Jae Hyeon Yoo,2, 3 Sergei V. Isakov,1 Philip Massey,3 Ramin Halavati,3 Murphy Yuezhen Niu,1 Alexander Zlokapa,9, 1 Evan Peters,4, 6, 10 Owen Lockwood,11 Andrea Skolik,12, 13, 14, 15 Sofiene Jerbi,16 Vedran Dunjko,13 Martin Leib,12 Michael Streif,12, 14, 15, 17 David Von Dollen,18 Hongxiang Chen,19, 20 Shuxiang Cao,19, 21 Roeland Wiersema,22, 23 Hsin-Yuan Huang,1, 24, 25 Jarrod R. McClean,1 Ryan Babbush,1 Sergio Boixo,1 Dave Bacon,1 Alan K. Ho,1 Hartmut Neven,1 and Masoud Mohseni1, z 1Google Quantum AI, Mountain View, CA 2Sandbox@Alphabet, Mountain View, CA 3Google, Mountain View, CA 4Institute for Quantum Computing, University of Waterloo, Waterloo, Ontario, N2L 3G1, Canada 5School of Computer Science, University of Waterloo, Waterloo, Ontario, N2L 3G1, Canada 6Department of Applied Mathematics, University of Waterloo, Waterloo, Ontario, N2L 3G1, Canada 7Department of Mechanical & Mechatronics Engineering, University of Waterloo, Waterloo, Ontario, N2L 3G1, Canada 8Department of Physics & Astronomy, University of Waterloo, Waterloo, Ontario, N2L 3G1, Canada 9Division of Physics, Mathematics and Astronomy, Caltech, Pasadena, CA 91125 10Fermi National Accelerator Laboratory, P.O. Box 500, Batavia, IL, 605010 11Department of Computer Science, Rensselaer Polytechnic Institute, Troy, NY 12180, USA 12Data:Lab, Volkswagen Group, Ungererstr. 69, 80805 München, Germany 13Leiden University, Niels Bohrweg 1, 2333 CA Leiden, Netherlands 14Quantum Artificial Intelligence Laboratory, NASA Ames Research Center (QuAIL) 15USRA Research Institute for Advanced Computer Science (RIACS) 16Institute for Theoretical Physics, University of Innsbruck, Technikerstr. 21a, A-6020 Innsbruck, Austria 17University Erlangen-Nürnberg (FAU), Institute of Theoretical Physics, Staudtstr. 7, 91058 Erlangen, Germany 18Volkswagen Group Advanced Technologies, San Francisco, CA 94108 19Rahko Ltd., Finsbury Park, London, N4 3JP, United Kingdom 20Department of Computer Science, University College London, WC1E 6BT, United Kingdom 21Clarendon Laboratory, Department of Physics, University of Oxford. Oxford, OX1 3PU, United Kingdom 22Vector Institute, MaRS Centre, Toronto, Ontario, M5G 1M1, Canada 23Department of Physics and Astronomy, University of Waterloo, Ontario, N2L 3G1, Canada 24Institute for Quantum Information and Matter, Caltech, Pasadena, CA, USA 25Department of Computing and Mathematical Sciences, Caltech, Pasadena, CA, USA (Dated: August 30, 2021) We introduce TensorFlow Quantum (TFQ), an open source library for the rapid prototyping of hybrid quantum-classical models for classical or quantum data. This framework offers high-level ab- stractions for the design and training of both discriminative and generative quantum models under TensorFlow and supports high-performance quantum circuit simulators. We provide an overview of the software architecture and building blocks through several examples and review the theory of hybrid quantum-classical neural networks. We illustrate TFQ functionalities via several basic appli- cations including supervised learning for quantum classification, quantum control, simulating noisy quantum circuits, and quantum approximate optimization. Moreover, we demonstrate how one can apply TFQ to tackle advanced quantum learning tasks including meta-learning, layerwise learning, Hamiltonian learning, sampling thermal states, variational quantum eigensolvers, classification of quantum phase transitions, generative adversarial networks, and reinforcement learning. We hope arXiv:2003.02989v2 [quant-ph] 26 Aug 2021 this framework provides the necessary tools for the quantum computing and machine learning re- search communities to explore models of both natural and artificial quantum systems, and ultimately discover new quantum algorithms which could potentially yield a quantum advantage. CONTENTS A. Quantum Machine Learning 3 B. Hybrid Quantum-Classical Models 3 I. Introduction 3 C. Quantum Data 4 D. TensorFlow Quantum 5 II. Software Architecture & Building Blocks 5 ∗ [email protected] A. Cirq 5 y [email protected] B. TensorFlow 6 z [email protected] C. Technical Hurdles in Combining Cirq with 2 TensorFlow 6 2. Implementation 31 D. TFQ architecture 7 1. Design Principles and Overview 7 V. Advanced Quantum Applications 31 2. The Abstract TFQ Pipeline for a specific A. Meta-learning for Variational Quantum hybrid discriminator model 8 Optimization 32 3. Hello Many-Worlds: Binary Classifier for Quantum Data 9 B. Vanishing Gradients and Adaptive Layerwise E. TFQ Building Blocks 10 Training Strategies 33 1. Quantum Computations as Tensors 11 1. Random Quantum Circuits and Barren 2. Composing Quantum Models 11 Plateaus 33 3. Sampling and Expectation Values 11 2. Layerwise quantum circuit learning 34 4. Differentiating Quantum Circuits 12 C. Hamiltonian Learning with Quantum Graph 5. Simplified Layers 12 Recurrent Neural Networks 35 6. Basic Quantum Datasets 13 1. Motivation: Learning Quantum Dynamics F. High Performance Quantum Circuit with a Quantum Representation 35 Simulation with qsim 13 2. Implementation 36 1. Comment on the Simulation of Quantum Circuits 13 D. Generative Modelling of Quantum Mixed 2. Gate Fusion with qsim 13 States with Hybrid Quantum-Probabilistic 3. Hardware-Acceleration 14 Models 38 4. Benchmarks 14 1. Background 38 5. Large-scale simulation for quantum 2. Variational Quantum Thermalizer 39 machine learning 15 3. Quantum Generative Learning from 6. Noise in qsim 15 Quantum Data 41 E. Subspace-Search Variational Quantum III. Theory of Hybrid Quantum-Classical Machine Eigensolver for Excited States: Integration Learning 15 with OpenFermion 41 A. Quantum Neural Networks 15 B. Sampling and Expectations 16 1. Background 41 C. Gradients of Quantum Neural Networks 17 2. Implementation 42 1. Finite difference methods 17 F. Classification of Quantum Many-Body Phase 2. Parameter shift methods 17 Transitions 44 3. Stochastic Parameter Shift Gradient 1. Background 44 Estimation 18 2. Implementation 45 4. Adjoint Gradient Backpropagation in G. Quantum Generative Adversarial Networks 46 Simulations 18 D. Hybrid Quantum-Classical Computational 1. Background 46 Graphs 20 2. Noise Suppression with Adversarial 1. Hybrid Quantum-Classical Neural Learning 46 Networks 20 3. Approximate Quantum Random Access E. Autodifferentiation through hybrid Memory 47 quantum-classical backpropagation 21 H. Reinforcement Learning with Parameterized Quantum Circuits 48 IV. Basic Quantum Applications 22 A. Hybrid Quantum-Classical Convolutional 1. Background 48 Neural Network Classifier 22 2. Policy-Gradient Reinforcement Learning 1. Background 22 with PQCs 49 2. Implementations 23 3. Implementation 49 B. Hybrid Machine Learning for Quantum 4. Value-Based Reinforcement Learning with Control 25 PQCs 51 1. Time-Constant Hamiltonian Control 26 5. Quantum Environments 51 2. Time-dependent Hamiltonian Control 27 C. Simulating Noisy Circuits 28 VI. Closing Remarks 52 1. Background 28 2. Noise in Cirq and TFQ 28 3. Training Models with Noise 29 VII. Acknowledgements 52 D. Quantum Approximate Optimization 30 1. Background 30 References 52 3 I. INTRODUCTION be embedded into quantum states [19], a process whose scalability is under debate [20]. Additionally, many com- A. Quantum Machine Learning mon approaches for applying these algorithms to classical data rely on specific structure in the data that can also be exploited by classical algorithms, sometimes precluding Machine learning (ML) is the construction of algo- the possibility of a quantum speedup [21]. Tests based rithms and statistical models which can extract infor- on the structure of a classical dataset have recently been mation hidden within a dataset. By learning a model developed that can sometimes determine if a quantum from a dataset, one then has the ability to make predic- speedup is possible on that data [22]. Continuing de- tions on unseen data from the same underlying probabil- bates around speedups and assumptions make it prudent ity distribution. For several decades, research in machine to look beyond classical data for applications of quantum learning was focused on models that can provide theoret- computation to machine learning. ical guarantees for their performance [1–4]. However, in recent years, methods based on heuristics have become With the availability of Noisy Intermediate-Scale dominant, partly due to an abundance of data and com- Quantum (NISQ) processors [23], the second generation putational resources [5]. of QML has emerged [8, 12, 18, 24–44]. In contrast to the first generation, this new trend in QML is based on Deep learning is one such heuristic method which has heuristic methods which can be studied empirically due seen great success [6, 7]. Deep learning methods are to the increased computational capability of quantum based on learning a representation of the dataset in the hardware. This is reminiscent of how machine learning form of networks of parameterized layers. These parame- evolved towards deep learning with the advent of new ters are then tuned by minimizing a function of the model computational capabilities [45]. These new algorithms outputs, called the loss function. This function quantifies use parameterized quantum transformations called pa- the fit of the model to the dataset. rameterized quantum circuits (PQCs) or Quantum Neu- In parallel to the recent advances in deep learning, ral Networks