eDRAM-CIM: Compute-In-Memory Design with Reconfigurable Embedded Dynamic Memory Array Realizing Adaptive Data Converters and Charge Domain Computing Shanshan Xie1, Can Ni1, Aseem Sayal1, Pulkit Jain2, Fatih Hamzaoglu2, Jaydeep P. Kulkarni1 1University of Texas at Austin, Austin, TX 2Intel, Hillsboro, OR © 2021 IEEE 16.2: eDRAM-CIM: Compute-In-Memory Design with Reconfigurable Embedded Dynamic Memory Array International Solid-State Circuits Conference Realizing Adaptive Data Converters and Charge Domain Computing 1 of 60 Self Introduction Education • B.S. 2018 Worcester Polytechnic Institute • Ph.D. student at University of Texas at Austin since 2018 • Circuit Research Lab: https://sites.utexas.edu/CRL/ • Advisor: Professor Jaydeep P. Kulkarni Experience • Analog Design Intern, Texas Instruments, 2019 • Application Engineering Intern, Analog Devices Inc., 2018 Shanshan Xie • Test Engineering Intern, Analog Devices Inc., 2017 • Product Engineering Intern, Analog Devices Inc., 2016 Email: [email protected] Research Area Website: http://cyanxie.com/ • Mixed signal approaches for ML accelerators • Compute-in-memory techniques © 2021 IEEE 16.2: eDRAM-CIM: Compute-In-Memory Design with Reconfigurable Embedded Dynamic Memory Array International Solid-State Circuits Conference Realizing Adaptive Data Converters and Charge Domain Computing 2 of 60 Outline n Compute-in-memory: motivation and challenges n Proposed eDRAM-CIM macro l Overall architecture l Digital-to-analog conversion (DAC) l Multiply-accumulate-averaging (MAV) computation l Analog-to-digital conversion (ADC) n Chip measurement and classification accuracy results n Summary © 2021 IEEE 16.2: eDRAM-CIM: Compute-In-Memory Design with Reconfigurable Embedded Dynamic Memory Array International Solid-State Circuits Conference Realizing Adaptive Data Converters and Charge Domain Computing 3 of 60 Outline n Compute-in-memory: motivation and challenges n Proposed eDRAM-CIM macro l Overall architecture l Digital-to-analog conversion (DAC) l Multiply-accumulate-averaging (MAV) computation l Analog-to-digital conversion (ADC) n Chip measurement and classification accuracy results n Summary © 2021 IEEE 16.2: eDRAM-CIM: Compute-In-Memory Design with Reconfigurable Embedded Dynamic Memory Array International Solid-State Circuits Conference Realizing Adaptive Data Converters and Charge Domain Computing 4 of 60 Deep Neural Network Applications AI High AI Cloud Edge Computing Edge Computing Performance Computing Face Detection Object Detection Data Center © 2021 IEEE 16.2: eDRAM-CIM: Compute-In-Memory Design with Reconfigurable Embedded Dynamic Memory Array International Solid-State Circuits Conference Realizing Adaptive Data Converters and Charge Domain Computing 5 of 60 DNN Model Size Challenges Total Number of MACs Energy Efficiency Challenges 341k LeNet-5 1 Data Intensive 724M AlexNet 2 Huge Data Movement 2.8G OverFeat 3 Data Transfer Latency 3.9G ResNet50 1.43G GoogleNet v1 4 Computation Energy Cost 15.5G VGG16 5 Memory Access Energy Cost Ref: Vivienne Sze, Proceedings of the IEEE 105.12 2017 © 2021 IEEE 16.2: eDRAM-CIM: Compute-In-Memory Design with Reconfigurable Embedded Dynamic Memory Array International Solid-State Circuits Conference Realizing Adaptive Data Converters and Charge Domain Computing 6 of 60 eDRAM-CIM Motivation Operation Energy(pJ) Computation Energy Cost Integer Add (32b) 0.1 Integer Multiply (32b) 3.1 Floating Point Add (32b) 0.9 Floating Point Multiply (32b) 3.7 Memory Access Energy Cost 8KB SRAM (64b) 10 ~650X 1MB SRAM (64b) 100 DRAM 2000 DRAM memory access is expensive: Ref: Mark Horowitz, ~650X higher than computation energy cost ISSCC 2014 © 2021 IEEE 16.2: eDRAM-CIM: Compute-In-Memory Design with Reconfigurable Embedded Dynamic Memory Array International Solid-State Circuits Conference Realizing Adaptive Data Converters and Charge Domain Computing 7 of 60 eDRAM Advantages for CIM ü Highest bitcell density ü High access speed ü Low read/write energy Desired attributes for CIM designs ü High write endurance ü High bandwidth Ref: Gregory Fredeman, JSSC 2015; Fatih Hamzaoglu, JSSC 2014; Nasser Kurd, JSSC 2015 © 2021 IEEE 16.2: eDRAM-CIM: Compute-In-Memory Design with Reconfigurable Embedded Dynamic Memory Array International Solid-State Circuits Conference Realizing Adaptive Data Converters and Charge Domain Computing 8 of 60 Our Approach: eDRAM based CIM MAV Compute Array • Directly perform computation inside embedded DRAM (eDRAM) • Reconfigure existing 1T1C eDRAM columns as charge domain computing units • Reduce off-chip memory access and latency © 2021 IEEE 16.2: eDRAM-CIM: Compute-In-Memory Design with Reconfigurable Embedded Dynamic Memory Array International Solid-State Circuits Conference Realizing Adaptive Data Converters and Charge Domain Computing 9 of 60 Outline n Compute-in-memory: motivation and challenges n Proposed eDRAM-CIM macro l Overall architecture l Digital-to-analog conversion (DAC) l Multiply-accumulate-averaging (MAV) computation l Analog-to-digital conversion (ADC) n Chip measurement and classification accuracy results n Summary © 2021 IEEE 16.2: eDRAM-CIM: Compute-In-Memory Design with Reconfigurable Embedded Dynamic Memory Array International Solid-State Circuits Conference Realizing Adaptive Data Converters and Charge Domain Computing 10 of 60 eDRAM CIM Overall Architecture: DAC Data Flow *DAC: Digital-to-Analog Converter 1. Load Data to eDRAM Load Input to DAC eDRAM array Bitcell Va & Va_bar 2. Generate Analog Values Va & Va_bar © 2021 IEEE 16.2: eDRAM-CIM: Compute-In-Memory Design with Reconfigurable Embedded Dynamic Memory Array International Solid-State Circuits Conference Realizing Adaptive Data Converters and Charge Domain Computing 11 of 60 eDRAM CIM Overall Architecture: MAV *DAC: Digital-to-Analog Converter *MAV: Multiply-Accumulation-Average 3. Load Weights to MAV Compute Array VMAV 4. Generate VMAV © 2021 IEEE 16.2: eDRAM-CIM: Compute-In-Memory Design with Reconfigurable Embedded Dynamic Memory Array International Solid-State Circuits Conference Realizing Adaptive Data Converters and Charge Domain Computing 12 of 60 eDRAM CIM Overall Architecture: Partial Sum *DAC: Digital-to-Analog Converter *MAV: Multiply-Accumulation-Average Store Partial Sum © 2021 IEEE 16.2: eDRAM-CIM: Compute-In-Memory Design with Reconfigurable Embedded Dynamic Memory Array International Solid-State Circuits Conference Realizing Adaptive Data Converters and Charge Domain Computing 13 of 60 eDRAM CIM Overall Architecture: Partial Sum *DAC: Digital-to-Analog Converter *MAV: Multiply-Accumulation-Average Pooling & ReLU © 2021 IEEE 16.2: eDRAM-CIM: Compute-In-Memory Design with Reconfigurable Embedded Dynamic Memory Array International Solid-State Circuits Conference Realizing Adaptive Data Converters and Charge Domain Computing 14 of 60 eDRAM CIM Overall Architecture: ADC *DAC: Digital-to-Analog Converter *SAR: Successive-Approximation *MAV: Multiply-Accumulation-Average *ADC: Analog-to-Digital Converter 5. Generate VREF 6. Final YOUT © 2021 IEEE 16.2: eDRAM-CIM: Compute-In-Memory Design with Reconfigurable Embedded Dynamic Memory Array International Solid-State Circuits Conference Realizing Adaptive Data Converters and Charge Domain Computing 15 of 60 eDRAM CIM Overall Architecture *DAC: Digital-to-Analog Converter *SAR: Successive-Approximation *MAV: Multiply-Accumulation-Average *ADC: Analog-to-Digital Converter 3. Load Weights to MAV Compute Array 5. Generate VREF 1. Load Data to eDRAM Bitcell 2. Generate Analog Values Va & Va_bar 4. Generate VMAV 6. Final YOUT © 2021 IEEE 16.2: eDRAM-CIM: Compute-In-Memory Design with Reconfigurable Embedded Dynamic Memory Array International Solid-State Circuits Conference Realizing Adaptive Data Converters and Charge Domain Computing 16 of 60 In-eDRAM Differential Digital-to-Analog Converter (DAC) © 2021 IEEE 16.2: eDRAM-CIM: Compute-In-Memory Design with Reconfigurable Embedded Dynamic Memory Array International Solid-State Circuits Conference Realizing Adaptive Data Converters and Charge Domain Computing 17 of 60 In-eDRAM DAC – Positive DAC eDRAM Bitcell Storage Constant “1” X3 X2 Input Data (4b) X1 X0 Constant “0” 32 capacitors charge share result 16×1 + 8×� + 4×� + 2×� + 1×� � = � ∗ # $ % & ! "" 32 © 2021 IEEE 16.2: eDRAM-CIM: Compute-In-Memory Design with Reconfigurable Embedded Dynamic Memory Array International Solid-State Circuits Conference Realizing Adaptive Data Converters and Charge Domain Computing 18 of 60 In-eDRAM DAC – Negative DAC eDRAM Bitcell Storage Constant “0” X3 X2 Complimentary to positive DAC X1 X0 Constant “1” 32 capacitors charge share result 8×� + 4×� + 2×� + 1×� + 1 � = � ∗ # $ % & !_(!) "" 32 © 2021 IEEE 16.2: eDRAM-CIM: Compute-In-Memory Design with Reconfigurable Embedded Dynamic Memory Array International Solid-State Circuits Conference Realizing Adaptive Data Converters and Charge Domain Computing 19 of 60 Differential Analog Voltages, Va and Va_bar VDD Va Positive DAC Zero Reference=VDD/2 Va_bar Negative DAC VSS • Prepare differential analog voltages (Va and Va_bar) for sign computation • Positive DAC: Va; Negative DAC: Va_bar • Depends on sign of the weight, choose either Va or Va_bar © 2021 IEEE 16.2: eDRAM-CIM: Compute-In-Memory Design with Reconfigurable Embedded Dynamic Memory Array International Solid-State Circuits Conference Realizing Adaptive Data Converters and Charge Domain Computing 20 of 60 Positive DAC Demonstration © 2021 IEEE 16.2: eDRAM-CIM: Compute-In-Memory Design with Reconfigurable Embedded Dynamic Memory Array International Solid-State Circuits Conference