Proceedings of Student-Faculty Research Day Conference, CSIS, Pace University, May 8th, 2020 Performing Quantum Computer Vision Tasks on IBM Quantum Computers and Simulators

Raj Ponnusamy Seidenberg School of CSIS, Pace University, Pleasantville, New York 10570 Email: [email protected]

Abstract—Computer Vision is an interdisciplinary I.INTRODUCTION scientific field that deals with how computers can gain a high-level understanding of digital images or videos and Computer vision tasks such as Data retrieval, automate tasks that the human visual system can do. Recent advancements in Quantum computation. It has encoding, and storage are processes considered been found out that quantum computation has great easy and commonplace in classical computing. advantages in computer vision tasks such acquiring, However, they remain challenging in the quantum processing, analyzing and understanding digital images, realm. One of these challenges is storing and and extraction of high-dimensional data from the real processing image data efficiently. Due to the world for decision-making applications because of its decoherence issue and for the better utilization of parallel processing characteristics where we are seeing the potential of providing exponential speed up when the quantum communication channel. A relevant compared to classical algorithm execution. This speed-up representation making use of quantum properties is can play a big role in vision machine learning where essential for a useful treatment of images, putting training a model is usually very slow and when mathematical devices like the quantum wavelet processing with images and videos. In recent years, there transform to avail. On the other hand, the model has been a lot of researches on quantum image needs more data for better accuracy which will processing and machine learning, including the representation and classification of an image in a take high training time, For example, in a classical quantum computer. For quantum computer vision, activity recognition task, there need to be around quantum image representation plays a key role, which 1000 videos for training a classifier. Also by using substantively determines the kinds of processing tasks modern deep learning algorithms, the training time and how well they can be performed. In this paper, first, can become even longer. Because of that, we discuss Flexible Representation of Quantum Images researchers usually use Graphics Processing Units method and implement them to store and retrieve (GPUs) to make computations feasible. And hence MNIST handwritten digit into an IBM quantum device. Then we use the Quantum Support Vector Machine a next-generation method for reducing the storage model that runs on IBM quantum devices to classify and computation time for those algorithms is MNIST image data-set. This demonstrates the potential utilizing quantum computers. The purpose of this of using quantum computers for vision tasks such as paper is to study these tasks on quantizing object classification for highly efficient image and video classical computer vision tasks and addressing processing for in the big data era. The result obtained in some open questions. this paper could be used in more quantum computer vision or image processing and machine learning applications. Many machine learning problems utilize linear algebra as there are efficient ways to Index Terms—Quantum Computer Vision, compute matrix operations by representing the Quantum Image representation, IBM , Quantum data in matrices. makes some Machine Learning. linear algebra computations faster, so implicitly improves classical machine learning tasks. An example of this is fast matrix inversion which has been used in generating the hyperplane for QSVM. After going through all the literature we can (NEQR) [3], Quantum Image Representation positively infer that the field of storage and Through the Two-Dimensional Quantum States retrieving images from a quantum computer and Normalized Amplitude (2D-QSNA)[5] has remains largely a nascent engineering application. taken advantage on quantum properties and based but there are limits to these methods for practical on a number of necessary to store an image uses. and efficiently. FRQI and NEQR are best among these algorithms as they use low numbers of We will implement the FRQI quantum qubits to represent an image and make use of few image representation algorithm to store MNIST numbers of gates to implement operations on the data-set on IBM Quantum devices and retrieving stored images. however, there are limitations to results and apply the QSVM algorithms to classify these algorithms as it cannot be used on a large, this data set. The results of our testing are touched highly detailed image and cannot extend to on as well as the implications of the progress and rectangular images. But 2D QSNA addresses some shortcomings of our approach. We finish with a of this limitation but that was having other discussion of what we can look forward to in the challenges to isolate the quantum states in order to future of this field and for ourselves as we work. In this paper, I will be implementing the continue our research on this topic. FRQI method and hence we will look at how that will work. In FRQI representation, the image is stored in a state given by the following: II.BACKGROUND 22n−1 1 X I(θ) >= (cos θ |0 > + sin θ | 1 >) ⊗ i > A. Flexible Representation of Quantum Images 2n i i i=0 (1) Computer Vision tasks are expensive in in which theta encodes the colors of the image and terms of computational complexity and |ii encodes the positions of the pixels in the fundamental task is to represent the image. On image. Getting from an initial stat |0 >⊗(2n+1) to classical computer representing image pixel by the FRQI state requires the use of a unitary pixel required lot of computation resource and to operation, which we will call P. First, Hadamard process the image we use Fast Fourier Transform gates are applied to each of the qubits in the initial method to speed up the processing of the image in state. Using controlled rotation gates on the the frequency domain based on convolution current state transforms it into the FRQI state. theorem but on quantum computers, we have the Referring to the controlled rotations as R and the ability to store N bits of classical information in Hadamard gates as H, we can say that P=RH. This only log2 N quantum bits (qubits) and we have process uses 2n Hadamard gates and 22n efficient image processing techniques like controlled rotations. These controlled rotations can 2n Quantum Fourier Transform, Quantum Wavelet be implemented by Cy (2θ) and NOT operations. 2n Transform of Fourier Transform which is better It has also been shown that C Ry(2θ) can be than Fast Fourier Transform in classical computers broken down into 22n − 1 simple operations 2θ1  −2θ1  2n and also we can take advantage of quantum Ry 22n−1 and Ry 22n−1 , and 2 − 1 NOT properties like entanglement when image Operations. Therefore, the total number of simple represented in a quantum state. There are various operations to get from the initial state to the FRQI methods researched and explored to represent the state is 2n + 22n (22n−1 − 1 + 22n − 1 − 2) = image in quantum state . First Venegas-Andraca 24n − 3 (22n) + which is quadratic in 22n. The and Bose [1] introduced lattice method to number of gates used to get to the FRQI state can represent each pixel in quantum which is purely still be quite large when considering the depth of hypothetical and impractical design but other large images how many pixels large images methods flexible representation of quantum images require. In order to reduce the number of simple (FRQI)[2], Novel enhanced quantum representation gates required, a process called Quantum Image Compression can be used based on image where Nx is the normalization factor . By using representation called real ket devised by Latorre Hamiltonian simulation, e−iK∆t is obtained where ˆ K et. al. K = tr K . The algorithm solves the least-squares SVM problem defined as the following N B. Quantum Support Vector Machines 1 T γ X 2 min J(w, b, e) = w w + ek (4) w,b,e 2 2 k=1 T  Support Vector Machines(SVM) are s.t. yk w φ (xk) + b = 1 − ek When the dual supervised learning models with associated problem is solved, one gets the following result: learning algorithms that analyze data used for −→ !  b  0 1 T  b   0  classification and regression analysis.it is used to F = −→ = find a hyperplane in an N-dimensional space that ~α 1 T K + γ−1I ~α ~y distinctly classifies the data points. To separate the (5) two classes of data points, there are many possible −iFˆ δt hyper-planes that could be chosen and the Then by using Hamiltonian technique, e algorithm is to find a plane that has the maximum can be constructed and the system is solved by margin using a cost function. The equation of the HHL algorithm[7]. Then the state for the variables is defined in the computational basis as the hyper-plane is the following: ~w · ~xi − b. The problem is solved by the following optimization following −→ M ! framework: minψ,b~ k~wk s.t. yi(~w · xi − b) ≥ 1. The 1 X |b, ~αi = b|0i + α |ki (6) dual formulation is the following: C k PM 1 PM k=1 max~α L(~α) = j=1 yjαj − 2 j,k=1 αjxj · xkαk where C is the normalizing state. In the M classification part, the aim is to classify the query X s.t. αj = 0, yjαj ≥ 0 (2) state defined as the following j=1 M ! 1 X −→ |x˜i = |0i|0i + |~x||ki| · xki (7) For nonlinear classification, kernel function Nx¯ K = k (x , x ) is introduced and by Mercer’s i=1 jk j k Training data is constructed by using oracle in the theorem, PM α x · x α can be transformed j,k=1 j j k k following way: PM into the form j,k=1 αjKjkαk. At the end, the M + ! 1 X primal parameters are recovered as ~w = |b, ~αi → √ b|0i|0i + α |−→x k|k |−→x i PM −→ N k k k j=1 αj~xj, and b = yj − ~w · xj . The decision for u˜ i=1 binary classification is the following: (8) Pn −→ y(~x) = sign ( i=1 αjk(xj , ~x)+ b. Quantum SVM (QSVM) has been published in 2014 [6] for III.EXPERIMENTAND RESULTS solving least-squares SVM which is nothing but quantum version of classical SVM. In QSVM, at I have chosen IBM QISKit on IBM first the training data is represented as a quantum Quantum Experience cloud to run QSVM and state using Quantum Random Access Memory FRQI algorithm for MNIST image data set due to where n qubits to address any quantum it’s the simple and effective feature and superposition of N = 2n memory cells. The state −→ 1 PN −→ extensively used quantum simulators and quantum |xj i = −→ k=1(xj )k|ki represents the training |xj | devices available online. I have implemented the data. Training data oracle gives the following state FRQI algorithm and used the QSVM algorithm as it maps indices to data: that comes with the Qiskit Aqua package for

M M multi-classification and runs this experiment both 1 X 1 X √ |ii → |χi = |−→x kii|−→x i (3) on IBM Quantum Experience simulators and p i i quantum devices. M i=1 Nχ i=1 A. Environment and Tools Setup can affect the execution of the code. It must also be noted that, because of the randomness of the I have used the following tools and packages quantum process, each instance of the experiment to run this experiment. can have different results. The IBM Quantum Experience allows users to specify the number of 1) Registration and API key from IBM’s runs (shots) for an experiment. In this particular Quantum Experience to access the device. experiment, 1024 shots were specified. Once 2) IBM’s Quantum Experience Qiskit Notebooks completed, the results were displayed as a with the following package installed. histogram shown in figure 9.

To represent the color image in the quantum software version circuit, First, we need to create the superposition Qiskit 0.19.1 of all states which were initially in the zero states, Terra 0.14.1 and then perform the controlled rotation operations Aer 0.5.1 which encode the information of color with each Ignis 0.3.0 state. The final state I(θ) represents the qubits in a Aqua 0.7.0 superposition where each bit-string (n-1) cvxopt 1.2.4 represents the position of a pixel, tensored with Tensorflow 2.1.0 one qubit which is used for encoding the IBM Q Provider 0.5.0 information of color. The first part is pretty Python 3.7.6 familiar, the cascaded Hadamard operation. The OS Linux next part is performing controlled rotations on each state in such a way that every operation acts B. Methodology like a counter and rotates only the last qubit of the current state with the angle required for that qubit. we mainly focused on the ”Qiskit Aqua” to implement quantum algorithms. First, we need to To build the QSVM model, we split the get access to the quantum device which we can data-set into two parts called the training and get it free from IBM by logged into IBM Q testing part. The training set has the features and experience cloud and generated an API token. This labels. This set is used for the model to learn. The token was subsequently entered into the Qiskit other testing part has the feature points without configuration file for access to the IBM quantum any labels. By using this data-set we try to computer via the API. Next, we will start writing validate the accuracy by the model. The training and running code in Qiskit notebook on cloud and part is drawn 20 different samples from each class based on the QISKit workflow. These three steps of distribution randomly of the data which of the development of a quantum program are the contains 200 and the testing part drawn 10 same as a classic program with the following different samples from each class of distribution functions: build, compile, and run. The first step, randomly of the data which contains 100. Finally, building consists of creating a we run the experiment to 1257 samples to predict composed of quantum registers and adding gates its label. As shown in Table 1, the following to it to use and manipulate the registers. These dimension of the training set and the test set used gates will perform directly on qubits. The different for our experiments. back-end servers available can then be chosen during the compile step if, for example, the code Feature Training-set Test-set Prediction-set should be executed on a , on the 15 20*10 10*10 1257 local machine, or an IBM quantum chip. After TABLE I running the code in the final step, and receiving a EXPERIMENT SETUP result, the user can select a variety of options that To run the experiment, first, we need to number of qubits as 15 which is same as the total pre-process the data where we need to remove the number of features to train the model and depth as mean and scaling to the unit variance for 2 which is a default that means the number of standardizing. Centering and scaling appear times the circuit will repeat and entanglement = independently on every feature by computing the ‘full’ means all qubits will be entangled with each applicable statistics on the samples in the training other, ‘linear’ is nearest-neighbor and ’sca’ is a set. The mean and popular deviation is then stored shifted-circular-alternating entanglement. We used to be used on later data using the radically change linear entanglement to observe the results. method. Standardization of a dataset is a common requirement for many machine learning estimators. They would possibly behave badly if the individual Fig. 2: QSVM circuit points do not greater or much less seem to be like standard usually distributed data. Finally, we need to reduce the dimension of original data-set due to C. Results limitation of IBM quantum device as they allowed to run the experiment with 15 feature set on The storage and retrieval of the image were device and 32 on the simulator and hence we need implemented very much like was laid out in the to apply PCA to reduce the dimension and split papers which we discussed on the FRQI section the training and testing data based on the capacity and Fig3 and Fig4 will be the result of the original that is available to us to run the experiment. The image and image encoded and decoded into format of label data-set where the label ”A” quantum device 1024 shots and Fig5, Fig6, Fig7, represents the digit ”0” and the label ”B” and Fig8 shows Bloch vector and qsphere of represents the digit ”1”. This data-set was prepared Digit4 and 5 in the quantum state. you can see to fit in a 16 qubit machine. Due to the qubit decoded images mostly captured all the features of limitation, we could only take a few data points. the original images but it is not like original image and reason for that since measured quantum state as probabilities. As shown in Fig4 and Fig5, we can store and retrieve simple images with satisfactory precision. this helped us understanding how quantum image representation is working. However, we were unable to fully retrieve an image using the full 256 RGB values generally used to store images. We hope to be able to refine our encoding mechanisms to account for this as well as make our implementation more efficient in general. Implementing different algorithms as well as more operations on quantum images could further enhance our understanding of this field in Fig. 1: PCA the future and we look forward to expanding on this work. Here is the QSVM accuracy of this we will be running the experiment in the experiment. state vector simulator and qasm simulator first for 1024 shots and then quantum device ibmq16 Observation ibmq qasm statevector ibmq16melbourne Accuracy 64% 63% 71% melbourne which 15 qubit device that we can get Running time 16Hrs 40Min’s 14Hrs at max but simulator support up to 32 qubits. I TABLE II will be using a linear entangler map and a EXPERIMENT RESULTS second-order feature map since this is a multi-class classification. On this paper, we are using the Fig. 5: Bloch Multivector for Digit 5

Fig. 3: Original image of Digit 4 and result return from FRQI encoded image from Quantum state

Fig. 4: Original image of Digit 5 and result return Fig. 6: Qsphere representation of a quantum state from FRQI encoded image from Quantum state for Digit 5

we can use. As of now simulator support at max It is evident the great difference between 32 qubits and real back-end device support 15 results from the two different simulators, we qubits and hence we reduced 64 feature data set to remark that we used state vector, QASM 15 feature data-set and this can led to the loss of simulator, and 16 qubits ibmq16 Melbourne device information and number of shots used also playing to get an idea about the accuracy of the current a key in getting better accuracy. configuration of the algorithm. From this experiment, we can say that the state vector simulator is much faster than the QASM simulator IV. CONCLUSIONAND FUTURE WORK as well as the quantum device. State vector simulator completed jobs in 40 minutes, while the In this study, we explored the realm of QASM simulator took more than 16 hours and a quantum computer vision tasks such as object quantum device for almost 14 hours. Results from classification and representation of images as part the simulator and real back-ends are quite different of image processing and run the FRQI method of as well as time is taken to run the training and image representation and QSVM multi-class prediction as mentioned in Table 2. Time is taken classification algorithm on IBM quantum to run the experiment on simulator almost the computers and simulator. In particular, discussed same in all experiment But it was different from and demonstrated the quantum circuit to encode the real quantum device and noticed it is not the MNIST handwritten image data into Quantum consistent and depends on the performance of computer and run the QSVM circuit to classify the IBM quantum API and network speed. Regarding digits. In the future, we wish to expand our scope accuracy on a quantum computer is very less than to more advanced computer vision tasks such as classical computers and one of the reason due to better quantum representations of images, object limitation imposed on the number of qubits that segmentation, and deep learning with the Fig. 7: Bloch Multivector for Digit 4

Fig. 9: QSVM histogram result

Fig. 10: State Vector and QASM Simulator QSVM result

Quantum States and Normalized Amplitude. arXiv preprint arXiv:1305.2251. [6] Rebentrost, P., Mohseni, M., and Lloyd, S. Quantum support vector machine for big data classification. Physical review letters 113, 13 (2014), 130503. [7] Harrow, A. W., Hassidim, A., and Lloyd, S. Fig. 8: Qsphere representation of a quantum state for solving linear systems of equations. Physical review letters for Digit 4 15, 103 (2009), 150502. [8] Learn Quantum Computation using Qiskit, 2020 https://community.qiskit.org/textbook/ [9] A collection of Jupyter notebooks showing how to use advancement in this field and it could be possible Qiskit that is synced with the IBM Q Experience, 2020 to demonstrate quantum supremacy. Our future https://github.com/Qiskit/qiskit-iqx-tutorials work is motivated by the promise of breaking the [10] Qiskit API documentation, 2020 https://qiskit.org/documentation/ limitations of the classical framework imposes and exploit the unique features of a quantum system discussed to efficiently run the computer vision tasks.

REFERENCES

[1] S.E. Venegas-Andraca and S. Bose, Storing, processing, and retrieving an image using quantum mechanics, in Proc. of SPIE Conf. Quantum [2] P.Q., Dong, F., Hirota, K.: A flexible representation of quantum images for polynomial preparation, image compression, and processing operations. Quantum Inform.Process. 10(1), 6384 (2011) [3] Y. Zhang, K. Lu, Y. Gao and M. Wang, NEQR: a novel enhanced quantum representation of digital images, Quantum Inf. Process., DOI 10.1007/s11128-013-0567-z. [4] H.-S. Li, Z. Qingxin, S. Lan, C.-Y. Shen, R. Zhou, and J.Mo, Image storage, retrieval, compression and segmentation in a quantum system, Quantum Inf. Process., vol. 12,pp. 2269-2290, 2013. [5] Srivastava, M., Moulick, S.R., and Panigrahi, P.K., 2013. Quantum Image Representation Through the TwoDimensional