Design and Implementation of a High-Speed PCI-Express Bridge Mandus Börjesson Håkan Gerner
Total Page:16
File Type:pdf, Size:1020Kb
LiU-ITN-TEK-A--19/012--SE Design and implementation of a high-speed PCI-Express bridge Mandus Börjesson Håkan Gerner 2019-06-05 Department of Science and Technology Institutionen för teknik och naturvetenskap Linköping University Linköpings universitet nedewS ,gnipökrroN 47 106-ES 47 ,gnipökrroN nedewS 106 47 gnipökrroN LiU-ITN-TEK-A--19/012--SE Design and implementation of a high-speed PCI-Express bridge Examensarbete utfört i Elektroteknik vid Tekniska högskolan vid Linköpings universitet Mandus Börjesson Håkan Gerner Handledare Qin-Zhong Ye Examinator Adriana Serban Norrköping 2019-06-05 Upphovsrätt Detta dokument hålls tillgängligt på Internet – eller dess framtida ersättare – under en längre tid från publiceringsdatum under förutsättning att inga extra- ordinära omständigheter uppstår. Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervisning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säkerheten och tillgängligheten finns det lösningar av teknisk och administrativ art. Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsmannens litterära eller konstnärliga anseende eller egenart. För ytterligare information om Linköping University Electronic Press se förlagets hemsida http://www.ep.liu.se/ Copyright The publishers will keep this document online on the Internet - or its possible replacement - for a considerable time from the date of publication barring exceptional circumstances. The online availability of the document implies a permanent permission for anyone to read, to download, to print out single copies for your own use and to use it unchanged for any non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional on the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility. According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement. For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its WWW home page: http://www.ep.liu.se/ © Mandus Börjesson , Håkan Gerner Abstract This master thesis will cover the prestudy, hardware selection, design and implemen- tation of a PCI Express bridge in the M.2 form factor. The thesis subject was proposed by WISI Norden who wished to extend the functionality of their hardware using an M.2 module. The bridge fits an M-Key M.2 slot and has the dimensions 80x22 mm. It is able to communicate at speeds up to 8 Gb/s over PCI Express and 200 Mbit/s on any of the 20 LVDS/CMOS pins. The prestudy determined that an FPGA should be used and a Xilinx Artix-7 device was chosen. A PCB was designed that hosts the FPGA as well as any power, debugging and other required systems. Associated proof-of-concept software was designed to verify that the bridge operated as expected. The software proves that the bridge works but requires improvement before the bridge can be used to translate sophisticated protocols. The bridge works, with minor hardware modifications, as expected. It fulfills all de- sign requirements set in the master thesis and the FPGA firmware uses a well-established protocol, making further development easier. Acknowledgments We would like to thank the supervisor and examiner who took part in this Master thesis. Your feedback has proven valuable throughout the entire thesis work, not only helping us overcome obstacles but also helping us improve the quality of the final report. We are grateful of our fellow students who helped motivate us, gave valuable input and were always eager to discuss ways to overcome our engineering challenges. We also wish to extend a special thank you to our mentor Jonas Åberg. You have shared your many years of experience in hardware design and helped us boost the quality of the project beyond our expectations. ii Contents Abstract i Acknowledgments ii Contents iii List of Figures v List of Tables vii 1 Introduction 1 1.1 Aim............................................ 1 1.2 Background . 2 1.3 Research questions . 3 1.4 Design requirements . 3 1.5 Delimitations . 4 2 Prestudy 5 2.1 System sketch . 5 2.2 Possible solutions . 6 3 Selection of hardware 10 3.1 FPGA selection . 10 3.2 Non-volatile memory selection . 13 3.3 Regulator selection . 13 4 System Description 16 4.1 M.2 Connector . 16 4.2 FPGA . 17 4.3 Programming and debugging . 18 4.4 Power management . 19 4.5 External connectors . 21 5 Hardware design and implementation 22 5.1 Stackup and design rules . 22 5.2 M.2 connector . 24 5.3 FPGA configuration banks . 25 5.4 FPGA I/O banks . 26 5.5 FPGA transceivers and PCI Express nets . 26 5.6 Power management . 26 5.7 Programming and debugging interface . 30 5.8 Decoupling . 31 6 Manufacturing 33 iii 6.1 Bill of materials . 33 6.2 Assembly and soldering . 33 6.3 Tests and inspection . 33 7 Software 37 7.1 EEPROM memory verification . 37 7.2 Flash memory interfacing . 38 7.3 FPGA firmware . 41 7.4 Platform application . 43 8 Results 44 8.1 PCB Layout . 44 8.2 Bridge performance . 45 8.3 Summary of hardware modifications . 46 8.4 Software design . 47 9 Discussion and conclusion 49 9.1 Design requirements and research questions . 49 9.2 Implementation . 49 Bibliography 51 iv List of Figures 2.1 High level sketch of the system. 6 2.2 System sketch of the solution using a CPU with native PCI Express support. 6 2.3 System sketch of the solution when using a CPU without native support for PCI Express. 7 2.4 System sketch of the solution when using a FPGA. 8 3.1 Total accumulated price for N number of units, assuming 100 units/year. 12 4.1 System layout on the M.2 module. 16 4.2 M.2 connector system connections. 17 4.3 FPGA system connections. 17 4.4 Programming and debugging system connections. 19 4.5 Power management system connections. 19 4.6 Power hierarchy option considered in the design. 20 4.7 M.2 connector system connections. 21 5.1 Illustration of design rules used during layout. 23 5.2 Open-drain logic level conversion. HV indicates High (3.3 V) logic level, LV indi- cates low (1.8 V) logic level. 24 5.3 Layout surrounding the M.2 connector on the top layer of the PCB. Notice the fencing and return path vias as well as the ground plane (yellow) on the layer below the PCI Express lanes. 25 5.4 Fanout of the SPI and JTAG nets (highlighted) under the Artix-7. 25 5.5 Power-on sequencing using RC networks. 27 5.6 I/O voltage sequencing and selectable I/O voltage solution. 28 5.7 Characteristic change of capacitance according to DC-voltage. 29 5.8 Artix-7 FPGA power pin mapping. 30 5.9 Programming and debugging interface with related components marked. 31 6.1 Load switch schematic . 34 6.2 Reference clock differential signal measured on the bridge. 36 6.3 PCI Express link lists the capabilities of the bridge. 36 7.1 Console output after a successful write to the EEPROM. 38 7.2 Enumeration of the SMBus of the bridge. Notice the SMBus-SPI bridge at ’0x28’ and the EEPROM at ’0x50’ and ’0x51’. 38 7.3 Ideal SPI data transfer. 39 7.4 SPI data transfer using the built-in CS pin feature. 40 7.5 Final SPI data transfer implementation. 40 7.6 Overview of the VHDL block design. 41 7.7 Platform application writing to register and validates its value. 43 8.1 Layout – Top Layer . 44 v 8.2 Layout – Inner Layer 1 . 45 8.3 Layout – Inner Layer 2 . 45 8.4 Layout – Bottom . 45 8.5 Output signal at 100 MHz. 46 8.6 Output signal at 200 MHz. 46 8.7 the nPor fix. Right: Altium designer screenshot with the modifications, Left: Same modification on the PCB. 47 8.8 The CLKREQ fix. Right: Altium designer screenshot with the modification, Left: Same modification on the PCB. 47 8.9 Signals; TX ready, data request, data in, data out, clock and state. 48 8.10 Output signals; data and clock. 48 vi List of Tables 1.1 PCI Express transfer characteristics. 2 3.1 Product series candidates. 10 3.2 Selected devices from the different product families. ..