TERRAE: A Framework for Adaptive Hardware Concurrent Systems by Victor Gusev Lesau B.Eng. (Electrical), McMaster University, 2006 THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF APPLIED SCIENCE in the School of Engineering Science Faculty of Applied Sciences Victor Gusev Lesau 2012 SIMON FRASER UNIVERSITY Fall 2012 All rights reserved. However, in accordance with the Copyright Act of Canada, this work may be reproduced, without authorization, under the conditions for “Fair Dealing.” Therefore, limited reproduction of this work for the purposes of private study, research, criticism, review and news reporting is likely to be in accordance with the law, particularly if cited appropriately. Approval Name: Victor Gusev Lesau Degree: Master of Applied Science Title of Thesis: TERRAE: A Framework for Adaptive Hardware Concurrent Systems Examining Committee: Chair: Dr. Carlo Menon Assistant Professor, School of Engineering Science Dr. William A. Gruver Senior Co-Supervisor Professor Emeritus, School of Engineering Science Dorian Sabaz Senior Supervisor CTO, Holonic Technologies Inc. Dr. Richard Hobson Co-Supervisor Professor Emeritus, School of Engineering Science Dr. Craig Scratchley Internal Examiner Senior Lecturer, School of Engineering Science Date Defended/Approved: October 3, 2012 ii Partial Copyright Licence iii Abstract Dynamic Partial Reconfiguration (DPR) of Field Programmable Gate Arrays (FPGAs) is a technology that enables the development of embedded systems with hot swappable logic on the FPGA fabric. The advantage is that hardware logic can be swapped in and out “on-the-fly” while the rest of the system is operational. Since DPR is relatively new, tool support is still evolving. This thesis introduces new FPGA architectural tools and Linux OS modifications that aid in supporting DPR on FPGAs for concurrent control. It shows that control systems benefit from hardware concurrency, meaning that by moving the control intelligence into hardware, the negative effects inherent to threads and their scheduler are minimized. This leaves software with the role of a high-level administrator rather than an executor, thereby eliminating unnecessary bottlenecks. The tools described in this thesis enable the hardware engineer to develop DPR-FPGA systems more effectively for rapid control system development. Furthermore, the introduced Adaptive Hardware Concurrent System (AHCS) architecture illustrates how a designer can take operating systems to a new level of concurrency resulting in true deterministic concurrency implemented on DPR-enabled hardware platforms. Keywords: Adaptive Hardware; Embedded Linux; Hardware Concurrency; Field Programmable Gate Arrays; Dynamic Partial Reconfiguration; SoC Design iv Dedication To my family v Acknowledgements This thesis would not have been possible without the support of many people. First of all, I would like to thank the examining committee members for dedicating their time and energy in reviewing the thesis and providing the invaluable feedback. Also, I wish to express my gratitude to Dr. William A. Gruver for his thoughtful guidance, sincere advice and constructive feedback throughout my journey towards Masters. His wisdom and inspiration made a significant impact on this project. As well, a special thanks to Dorian Sabaz for being an unlimited fountain of ideas, which he shared with an open heart and without reservation, making it a truly joyful and inspirational learning experience. The long whiteboard sessions, countless Skype calls, and heated debates not only about technology, but also history, psychology, economics, and management played a key role in diversifying my thinking, and establishing a lifelong friendship. Furthermore, I would like to thank PMC-Sierra Inc. for providing support in completing this thesis, as well as the great people working there who shared my excitement and encouraged me throughout this project. I wish to thank George Babut for “infecting me with a technology bug” and fueling my initial curiosity in reconfigurable hardware and communication systems, which defined my career and academic paths to this point. Thanks to my grandfather, Victor Lesau, for influencing me from the childhood, and being a living example of a passionate researcher with an open sincere character. Finally, I wish to thank my family for giving me the opportunity getting to this point, supporting my values, and providing unconditional love. We made it! vi Table of Contents Approval .......................................................................................................................... ii Partial Copyright Licence ............................................................................................... iii Abstract .......................................................................................................................... iv Dedication ....................................................................................................................... v Acknowledgements ........................................................................................................ vi Table of Contents .......................................................................................................... vii List of Tables .................................................................................................................. xi List of Figures................................................................................................................xiv Glossary ....................................................................................................................... xvii 1. Introduction .......................................................................................................... 1 1.1. Motivation ............................................................................................................... 1 1.2. Contributions .......................................................................................................... 3 1.3. Organization ........................................................................................................... 3 2. Background .......................................................................................................... 5 2.1. Field Programmable Gate Arrays ........................................................................... 5 2.2. Dynamic Partial Reconfiguration ............................................................................. 7 2.3. Component-Based Design ...................................................................................... 9 2.4. Dynamically Reconfigurable Architecture ............................................................... 9 2.5. Linux and Embedded Devices .............................................................................. 10 2.6. Embedded Systems from Static FPGA to DPR-FPGA SoC .................................. 13 2.7. Current Operating System Support for DPR-FPGAs ............................................. 16 2.8. Issues with Threaded Architectures ...................................................................... 17 2.9. Adaptive Hardware Concurrent Control and DPR-FPGAs .................................... 18 3. System Architecture and Design ....................................................................... 20 3.1. System Requirements .......................................................................................... 20 3.2. Architecture and Design Discourse ....................................................................... 23 3.2.1. Single Bus Computer Architecture ............................................................ 23 3.2.2. Dual Bus Computer Architecture ............................................................... 25 3.2.3. PRM Connectivity and Component-Based Design .................................... 27 3.2.4. Processing Support ................................................................................... 29 3.2.4.1. Processor for Operating System ................................................. 29 3.2.4.2. Processor for DPR Subsystem ................................................... 30 3.2.5. Operating System ..................................................................................... 31 3.2.6. Software Systems ..................................................................................... 32 3.3. Decided Architecture ............................................................................................ 33 3.3.1. Intraframework for DPR Access ................................................................ 33 3.3.2. Computer Architecture of TERRAE ........................................................... 34 3.3.3. Software Components of TERRAE ........................................................... 37 3.3.3.1. PR API ....................................................................................... 39 3.3.3.2. TERRAE Connectivity ................................................................. 39 3.3.3.3. Packet Communication ............................................................... 40 3.3.3.4. System Packet Data Flow ........................................................... 41 vii 3.3.3.5. Synchronous Communication ..................................................... 42 3.3.3.6. Command Line Tool (CLT) ......................................................... 43 3.3.3.7. Daemon Process (DP) ...............................................................