A Packetized Display Protocol Architecture for Infrared Scene
Total Page:16
File Type:pdf, Size:1020Kb
A PACKETIZED DISPLAY PROTOCOL ARCHITECTURE FOR INFRARED SCENE PROJECTION SYSTEMS by Aaron Myles Landwehr A dissertation submitted to the Faculty of the University of Delaware in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Electrical & Computer Engineering Fall 2020 © 2020 Aaron Myles Landwehr All Rights Reserved A PACKETIZED DISPLAY PROTOCOL ARCHITECTURE FOR INFRARED SCENE PROJECTION SYSTEMS by Aaron Myles Landwehr Approved: Jamie D. Phillips, Ph.D. Chair of the Department of Electrical and Computer Engineering Approved: Levi T. Thompson, Ph.D. Dean of the College of Engineering Approved: Louis F. Rossi, Ph.D. Vice Provost for Graduate and Professional Education and Dean of the Graduate College I certify that I have read this dissertation and that in my opinion it meets the academic and professional standard required by the University as a dissertation for the degree of Doctor of Philosophy. Signed: Fouad E. Kiamilev, Ph.D. Professor in charge of dissertation I certify that I have read this dissertation and that in my opinion it meets the academic and professional standard required by the University as a dissertation for the degree of Doctor of Philosophy. Signed: Chase J. Cotton, Ph.D. Member of dissertation committee I certify that I have read this dissertation and that in my opinion it meets the academic and professional standard required by the University as a dissertation for the degree of Doctor of Philosophy. Signed: Xiaoming Li, Ph.D. Member of dissertation committee I certify that I have read this dissertation and that in my opinion it meets the academic and professional standard required by the University as a dissertation for the degree of Doctor of Philosophy. Signed: St´ephaneZuckerman, Ph.D. Member of dissertation committee ACKNOWLEDGMENTS I thank my committee for agreeing to be on my committee. I thank my col- leagues who helped with the endeavor of realizing PDP on actual hardware: Andrea, Chris, Daniel, and Tyler. I thank the rest of my colleagues past and current who helped in any way with PDP through their actions: Alex, Alexis, Andrew, Ben, Casey, Garret, Hamzah, Jaclyn, Jake, Jeff, Johnny, Jon, Josh, Kassem, Katie, Matt, Matt2, Mateo, Michelle, Miguel, Mike, Peyman, Rebekah, Rodney, Spencer, Tianne, Zack. I thank my friends who supported me over the years: Angela, Diego, Laura, Jose. I thank my family who supported me over the years: Joshua and my mom. I thank my therapist who supported me these past two years: Marcus. Finally, I thank these animals: Aurora, Chowder, Europa, Hal, Hazel, Kiddles, Kosmo, Mewist, Molly, Muffin, Pumpkin, Snickers, Tachi, and these unnamed animals: Birds, Fish, Furbies, Kittens, Mama Cat, Puppies, Random Animals, Tamagotchis. The work discussed within this dissertation was partially funded by (a) Air Force STTR Program AF18A-T017 `Next Generation Infrared Scene Projectors for Testing MWIR Systems' (Contract FA8650-19-C-1948), and (b) the Test Resource Management Center (TRMC) Test and Evaluation/Science & Technology (T&E/S&T) Program through the US Army Program Executive Office for Simulation, Training, and Instrumentation (PEO STRI) under Contract No. W900KK-17-C-0012. I thank ONSemiconductor for fabricating silicon arrays, Firefly Photonics and the University of Iowa for fabricating Infrared LED arrays, and Teledyne Scientific for hybridizing Silicon and LED arrays. Their fabrication effort enabled us to build and test the projector system(s) described in this dissertation. iv TABLE OF CONTENTS LIST OF TABLES :::::::::::::::::::::::::::::::: viii LIST OF FIGURES ::::::::::::::::::::::::::::::: ix ABSTRACT ::::::::::::::::::::::::::::::::::: xiii Chapter 1 INTRODUCTION :::::::::::::::::::::::::::::: 1 2 BACKGROUND ::::::::::::::::::::::::::::::: 5 2.1 IRLED Scene Projector History ::::::::::::::::::::: 5 2.2 IRLED Projection Process :::::::::::::::::::::::: 8 3 PROBLEM FORMULATION ::::::::::::::::::::::: 11 3.1 Display Protocol Limitations ::::::::::::::::::::::: 11 3.2 High-speed IRLED Scene Projector Systems :::::::::::::: 12 3.2.1 Hardware Limitations :::::::::::::::::::::: 13 3.2.2 Software Limitations ::::::::::::::::::::::: 13 3.3 Problem Statement :::::::::::::::::::::::::::: 14 3.4 Problem Solution ::::::::::::::::::::::::::::: 14 4 SYSTEM OVERVIEW ::::::::::::::::::::::::::: 18 4.1 Close Support Electronics :::::::::::::::::::::::: 18 4.2 Communication Flow ::::::::::::::::::::::::::: 21 4.2.1 Internal CSE Communication :::::::::::::::::: 23 4.2.2 External System Communication :::::::::::::::: 24 v 5 ARRAY WRITE PROCESS ::::::::::::::::::::::: 30 5.1 Array Interleaved Write Process ::::::::::::::::::::: 30 5.2 Data ordering ::::::::::::::::::::::::::::::: 37 6 DISPLAY PROTOCOLS :::::::::::::::::::::::::: 45 6.1 Conventional Display Protocols ::::::::::::::::::::: 45 6.2 Display Protocols within IRSP Technology ::::::::::::::: 51 7 PACKETIZED DISPLAY PROTOCOL :::::::::::::::: 53 7.1 Design Methodology ::::::::::::::::::::::::::: 53 7.2 Comparison :::::::::::::::::::::::::::::::: 55 7.3 Packet Format ::::::::::::::::::::::::::::::: 59 7.4 Packet Types ::::::::::::::::::::::::::::::: 60 7.5 PDP Stream Decoding :::::::::::::::::::::::::: 62 7.6 Overhead ::::::::::::::::::::::::::::::::: 65 7.7 Multi-frame Rate Performance :::::::::::::::::::::: 70 8 MACHINE MODEL :::::::::::::::::::::::::::: 74 8.1 Hardware Mapping :::::::::::::::::::::::::::: 74 8.2 Compositing :::::::::::::::::::::::::::::::: 77 9 IMPLEMENTATION :::::::::::::::::::::::::::: 82 9.1 HDMI Transport Layer :::::::::::::::::::::::::: 82 9.2 Abstract Architecture ::::::::::::::::::::::::::: 86 9.3 Frontend Architecture :::::::::::::::::::::::::: 87 9.4 Overall Backend Architecture :::::::::::::::::::::: 88 9.5 Synchronized Circular Buffer ::::::::::::::::::::::: 89 9.5.1 Controllers ::::::::::::::::::::::::::::: 89 9.5.2 Routing :::::::::::::::::::::::::::::: 90 9.5.3 Internal Buffer and Memory Synchronizer ::::::::::: 91 9.6 Array Emitter ::::::::::::::::::::::::::::::: 96 9.7 State Machines :::::::::::::::::::::::::::::: 97 9.8 Write Buffer :::::::::::::::::::::::::::::::: 99 vi 10 EXPERIMENTAL RESULTS ::::::::::::::::::::::: 101 10.1 Memory Synchronizer ::::::::::::::::::::::::::: 101 10.1.1 Simulation ::::::::::::::::::::::::::::: 101 10.1.2 Experimental Testing ::::::::::::::::::::::: 104 10.2 Firmware ::::::::::::::::::::::::::::::::: 106 10.2.1 Simulation ::::::::::::::::::::::::::::: 106 10.2.2 Characterization ::::::::::::::::::::::::: 107 10.2.2.1 Array Maps ::::::::::::::::::::::: 108 10.2.2.2 Fractional Difference Maps :::::::::::::: 109 10.2.2.3 Non-uniformity Corrected Imagery :::::::::: 114 10.2.2.4 Analog Bandwidth ::::::::::::::::::: 115 10.3 Packetized Operation ::::::::::::::::::::::::::: 116 10.3.1 Normal-speed ::::::::::::::::::::::::::: 117 10.3.2 High-speed :::::::::::::::::::::::::::: 118 10.3.3 Multi-frame Rate ::::::::::::::::::::::::: 119 10.4 Summary ::::::::::::::::::::::::::::::::: 120 11 CONCLUSION :::::::::::::::::::::::::::::::: 122 REFERENCES :::::::::::::::::::::::::::::::::: 125 vii LIST OF TABLES 3.1 Bandwidth requirements of a conventional display protocol ::::: 12 6.1 VESA Coordinated Video Timing (CVT) Modeline ::::::::: 50 7.1 Modeline Overhead ::::::::::::::::::::::::::: 57 7.2 List of PDP Packets :::::::::::::::::::::::::: 61 7.3 PDP Maximum Packet Overhead ::::::::::::::::::: 66 7.4 Multi-frame Rate Bandwidth Savings ::::::::::::::::: 72 9.1 PDP Select Communication APIs ::::::::::::::::::: 88 9.2 Full/Empty Memory Synchronizer State Transitions ::::::::: 94 viii LIST OF FIGURES 2.1 IRLED Scene Projector System :::::::::::::::::::: 6 2.2 IRLED Scene Projector Technology Development Timeline Overview7 2.3 SLEDs Array Pixel Ratios ::::::::::::::::::::::: 8 2.4 Typical IRLED Projection Process :::::::::::::::::: 9 3.1 Dynamic frame rate display with multiple regions updating at different frame rates :::::::::::::::::::::::::: 16 4.1 SLEDS System Block Diagram :::::::::::::::::::: 20 4.2 Example Hybrid Round Boards :::::::::::::::::::: 21 4.3 CSE Internals :::::::::::::::::::::::::::::: 22 4.4 CSE Externals and Empty Chassis :::::::::::::::::: 22 4.5 CSE Internal Communication Block Diagram :::::::::::: 24 4.6 CSE External Direct Communication Block Diagram :::::::: 26 4.7 CSE External Indirect API Communication Block Diagram :::: 27 4.8 CSE External Indirect API and Data Communication Block Diagram 28 5.1 TCSA, NSLEDS, and HDILED Array Quadrant Layouts :::::: 32 5.2 NSLEDS/HDILED Array Super Pixel Layout :::::::::::: 34 5.3 NSLEDS/HDILED Super Pixel Grid Layout ::::::::::::: 34 5.4 NSLEDS/HDILED Array Interleaved Pixel Mapping Per Write :: 36 ix 5.5 Bit-packing Format ::::::::::::::::::::::::::: 38 5.6 Image Encoding: Input Reordering :::::::::::::::::: 39 5.7 Image Encoding: Quadrant Reordering :::::::::::::::: 39 5.8 Image Encoding: Quadrant Reordering with Color Overlay ::::: 39 5.9 Image Encoding: Data Bit-Packing :::::::::::::::::: 41 5.10 Image Encoding: Data Reorder :::::::::::::::::::: 41 5.11 Image Encoding: Color Example 1 :::::::::::::::::: 43 5.12 Image Encoding: Color Example 2 :::::::::::::::::: 43 5.13 Image Encoding: Color Example 3 :::::::::::::::::: 43 5.14 Image Encoding: IR Example 1 :::::::::::::::::::: 44 5.15 Image Encoding: IR Example 2 :::::::::::::::::::: 44 6.1 Display Protocol Timing Overview :::::::::::::::::: 47 6.2 Display Protocol Horizontal Signal Cross Section Timing :::::: 48 6.3 Display Protocol Full Signal Cross Section Timing :::::::::: 48 6.4 Custom Synchronization Solution ::::::::::::::::::: 52 7.1