ENPH 455 Final Report Ultrafast Digital Electronics and Analog Photonics (DEAP) for Convolutional Neural Networks and Image Proc
Total Page：16
File Type：pdf, Size：1020Kb
ENPH 455 Final Report Ultrafast Digital Electronics and Analog Photonics (DEAP) for Convolutional Neural Networks and Image Processing Viraj Bangari Supervisor: Dr. Bhavin J. Shastri 2019-03-27 Department of Physics, Engineering Physics & Astronomy Queen’s University Abstract Convolutions are computationally intensive mathematical operations used in convolutional neural networks (CNNs) and image processing. Convolution operations are typically delegated to GPUs due to their ability to highly paralleliZe matrix multiplication operations. In recent years, silicon photonics has shown promise in being the next generation of computing hardware that can operate at ultrafast speeds. In particular, the neuromorphic photonic broadcast-and- weight architecture has been able to implement recurrent neural networks while operating at gigahertZ frequencies. Inspired by the principles of broadcast-and-weight, this thesis proposes two photonic architectures that are capable of performing convolution operations. The first architecture, called DEAP, is specialiZed for implementing convolutional neural networks whereas the second, DEAP-GIP, is specialiZed for general-purpose image processing tasks. DEAP is estimated to perform convolutions between 2.8 and 14 times faster than a GPU while roughly using 0.75 times the energy consumption. Additionally, DEAP-GIP is estimated to operate 4.6 to 68 times faster than a GPU while using 2.3 times the energy consumption. The largest bottlenecks for both of these architectures are from the I/O interfacing with digital systems via digital-to-analog converters (DACs) and analog-to-digital converters (ADCs). If photonic DACs [1] and ADCs [2] are to be built with higher bit-precisions, the speedup over GPUs could be even higher. Overall, silicon photonics has the potential to outperform conventional electronic hardware for convolutions while having the ability to scale up in the future. Page i Contents Abstract ................................................................................................................................................. i List of Tables ...................................................................................................................................... iv List of Figures ..................................................................................................................................... iv Glossary of Terms ................................................................................................................................ v Acknowledgements ............................................................................................................................. vi 1 Introduction ........................................................................................................................ 1 2 Convolution Background .................................................................................................... 2 2.1 Convolutions for Image Processing ........................................................................................ 2 2.2 Convolutional Neural Networks ............................................................................................. 3 2.2.1 The Convolutional Layer using Matrix Multiplications ................................................................... 4 3 Silicon Photonics Background ............................................................................................ 5 3.1 Waveguides ............................................................................................................................. 5 3.2 Microring Resonators .............................................................................................................. 6 3.2.1 Wavelength Division Multiplexing and Optical Modulation ........................................................... 8 4 Dot Products using Photonics ............................................................................................. 9 4.1 Overall Architecture ................................................................................................................ 9 4.2 Representing Negative Inputs ............................................................................................... 11 4.3 Throughput and SiZe Estimation ........................................................................................... 12 5 Photonic Convolutions for CNNs ..................................................................................... 13 5.1 A Top-down View ................................................................................................................ 13 5.2 Producing a Single Convolved Pixel .................................................................................... 14 5.3 Performing a Full Convolution ............................................................................................. 18 5.4 Throughput and Energy Estimation ...................................................................................... 20 6 Photonic Convolutions for General Image Processing ..................................................... 21 6.1 Rationale for a second approach ........................................................................................... 21 6.2 Input Representation ............................................................................................................. 22 6.3 Kernel Representation ........................................................................................................... 25 6.4 Performing a Convolution ..................................................................................................... 27 6.5 Throughput and Energy Estimation ...................................................................................... 29 7 Benchmarks ...................................................................................................................... 30 7.1 Proof of Concept Simulation ................................................................................................ 30 7.2 Estimated DEAP Performance .............................................................................................. 31 7.3 Estimated DEAP-GIP Performance ...................................................................................... 33 Page ii 8 Conclusion ........................................................................................................................ 34 9 References ........................................................................................................................ 36 Appendix A.1 Benchmarking Calculations for DEAP and DEAP-GIP ............................................ 39 Appendix A.2 Selected DEAP simulator code .................................................................................. 39 Appendix B. A Note on Semiconductor Optical Amplifiers ............................................................. 40 Appendix C. Statement of Work ........................................................................................................ 41 Appendix D. Floating Point Arithmetic using IEEE 754 ................................................................... 41 Page iii List of Tables Table 1 - Summary of Convolutional Parameters ...................................................................... 4 Table 2 - DEAP bounding parameters ...................................................................................... 14 Table 3 - Required components for DEAP input representation .............................................. 16 Table 4 – DEAP required components for applying kernel ..................................................... 17 Table 5 – DEAP-GIP bounding parameters ............................................................................. 21 Table 6 - Required components for DEAP-GIP input representation ...................................... 24 Table 7 - DEAP-GIP required components for applying kernel .............................................. 27 Table 8 - Benchmarked GPUs with power consumption ......................................................... 31 Table 9 - Benchmarking parameters for DEAP ........................................................................ 39 Table 10 - Benchmarking parameters for DEAP-GIP .............................................................. 39 List of Figures Figure 1 - Convolutions in image processing ............................................................................. 3 Figure 2 - Performing a convolution as a matrix-matrix multiplication. Image was modified from [12]. .................................................................................................................................... 5 Figure 3 - Left: Scanning electron microscope image of silicon waveguide. Right: TE propagation of a similar waveguide. Both images taken from [13]. .......................................... 6 Figure 4 - All-pass and add-drop MRRs .................................................................................... 6 Figure 5 - Phase dependent transfer of MRR through port and drop port lines ......................... 8 Figure 6 – An electro-optic architecture for performing dot products ..................................... 11 Figure 7 - Block diagram of DEAP architecture ...................................................................... 14 Figure 8 – DEAP input representation with photonics ............................................................. 15 Figure 9 – Applying