The Design of an IEEE 1588 End-To-End Transparent Ethernet Switch
Total Page：16
File Type：pdf, Size：1020Kb
The Design of an IEEE 1588 End-to-end Transparent Ethernet Switch by Caleb Gordon A thesis submitted to the Victoria University of Wellington in fulfilment of the requirements for the degree of Master of Science in Electronic and Computer Systems Engineering. Victoria University of Wellington 2008 Abstract In measurement and control systems there is often a need to synchronise distributed clocks. Traditionally, synchronisation has been achieved using a dedicated medium to convey time information, typically using the IRIG-B serial protocol. The precision time protocol (IEEE 1588) has been designed as an improvement to current methods of synchronisation within a dis- tributed network of devices. IEEE 1588 is a message based protocol that can be implemented across packet based networks including, but not lim- ited to, Ethernet. Standard Ethernet switches introduce a variable delay to packets that inhibits path delay measurements. Transparent switches have been introduced to measure and adjust for packet delay, thus removing the negative effects that these variations cause. This thesis describes the hardware and firmware design of an IEEE 1588 transparent end-to-end Ethernet switch for Tekron International Ltd based in Lower Hutt, New Zealand. This switch has the ability to monitor all Ethernet traffic, identify IEEE 1588 timing packets, measure the delay that these packets experience while passing through the switch, and account for this delay by adjusting a time-interval field of the packet as it is leaving the switch. This process takes place at the operational speed of the port, and without introducing significant delay. Time-interval measurements can be made using a high-precision timestamp unit with a resolution of 1 ns. The total jitter introduced by this measurement process is just 4.5 ns through a single switch. Acknowledgments I would like to thank a number of people for their help and support through- out this project. Dale Carnegie my primary supervisor for keeping me on track and pro- viding advice at crucial times. Thanks for encouraging me to study further and expand my knowledge. Brian Smellie my co-supervisor and director of Tekron International Ltd who provided a huge amount of support and encouragement at all stages of this project. Thank you for providing me with a project and a job when I was unsure of the direction I wanted to take. Thank you for all the suggestions encouragement from your deep knowledge at all stages of the project. Ian Mills director of Tekron International, thank you for taking a risk and allowing me to study and learn through your business. To the rest of the Tekron team, Andrew, Thomas and Marcel for providing a great work envi- ronment. I would like to thank the Foundation for Research Science and Technology for providing me with a Technology for Industry Fellowship. This funding provided me the drive to succeed and the ability to dedicate the many hours that I have towards this project. i ii To Aleks and Thomas for being a source of encouragement and entertain- ment, it was invaluable to have friends going through the same process as me. To my family, Mum, Dad, Adrian and Tim for providing me with weekly dinners and an environment of support and encouragement. Thanks for always being there and helping to bring out the best in me. To Dad in par- ticular for all the late nights editing, thank you so much. To all my friends, you know who you are, for understanding when you didn’t see me for weeks and always being there when I needed a break. Thanks for helping me to still live a somewhat normal life through all of this. Finally, to my love, Kerryn. You are the greatest provider of encourage- ment, love, support and happiness in my life. I couldn’t have done this without you. iii Glossary ASIC Application Specific Integrated Circuit ATM Asynchronous Transfer Mode BGA Ball Grid Array surface mount IC package BMC Best Master Clock algorithm CAT-5 (CAT5e) Unshielded twisted pair type cable defined by the TIA/EIA- 568-B standard CRC Cyclic Redundancy Check, hash function to produce a checksum used in error detection DB9 9-pin D-subminiature connector DIN rail mount Standardized 35 mm wide metal rail with hat-shaped cross section EMI Electromagnetic Interference FIFO First In First Out data buffer Follow-up IEEE 1588 general messages containing precision timestamps of sync message transmission FPGA Field Programmable Gate Array GPIO General Purpose Inputs and Outputs GPS Global Positioning System GUI Graphical User Interface IC Integrated Circuit IEEE Institute of Electrical and Electronics Engineers iv IEC International Electrotechnical Commission IP Intellectual Property IPC Association Connecting Electronics Industries IPv4 Internet Protocol Version 4 (also IPv6) IRIG-B Inter-Range Instrumentation Group time code type B ISCPS International Symposium on Precision Clock Synchronisation ISO International Organization for Standardization JTAG Joint Test Action Group - IEEE 1149.1 - Standard Test Access Port and Boundary-Scan Architecture LAN Local area network MAC Media Access Control Mbps Mega-bits per second MII Media independent interface NIOS II 32-bit embedded processor designed specifically for Altera FPGAs NTP Network Time Protocol OS Operating System OSI Open Systems Interconnection PCB Printed Circuit Board PHY Physical Layer Transceiver PLL Phase-Locked Loop PTP Precision Time Protocol (IEEE 1588) QoS Quality of Service v RFC Request For Comments RMII Reduced Media Independent Interface SCADA Supervisory Control And Data Acquisition SMII Serial Media Independent Interface SNMP Simple Network Management Protocol Sync IEEE 1588 defined timing/event messages UDP Universal Datagram Protocol VCTCXO Voltage-Controlled Temperature-Compensated Oscillator VHDL VHSIC Hardware Description Language VHSIC Very High-Speed Integrated Circuit VLAN Virtual LAN Contents 1 Introduction 1 1.1 IEEE 1588 . 1 1.2 Ethernet Switching . 2 1.3 Transparent Switches . 3 1.4 Project Objectives . 4 1.5 Thesis Structure . 6 2 IEEE 1588 Overview 7 2.1 IEEE 1588 Operation . 7 2.1.1 Message-Based Synchronisation . 7 2.1.2 Best Master Clock Algorithm . 10 2.1.3 Switch Delays . 11 2.2 Components of an IEEE 1588 Network . 11 2.2.1 Grandmaster Clock . 12 2.2.2 Ordinary Clock . 13 2.2.3 Boundary Clock . 13 2.2.4 Transparent Switches . 13 3 Literature Review 15 3.1 Comparison to other protocols . 15 3.2 Network Synchronisation . 16 3.3 Applications of IEEE 1588 . 18 3.4 Ethernet Switches . 20 3.5 IEEE 1588 Switches . 24 vi CONTENTS vii 3.5.1 Boundary Clocks . 24 3.5.2 Transparent Switches . 27 3.5.3 End-to-end vs. Peer-to-peer . 28 3.5.4 One-step vs. Two-step . 28 3.6 Market Research . 29 3.6.1 Ruggedised Switches . 29 3.6.2 IEEE 1588 Switch Manufacturers . 30 3.6.3 Switch Accuracy . 31 3.6.4 IEEE 1588 Switch Development Board . 32 3.7 Summary . 33 4 Hardware Design 35 4.1 Overview . 35 4.2 System Design . 36 4.2.1 Hardware Requirements . 36 4.2.2 Hardware Timestamping . 38 4.2.3 MII vs.RMII . 39 4.2.4 Hardware Options . 40 4.3 Component Selection . 42 4.3.1 Switch-on-a-chip . 42 4.3.2 FPGA . 46 4.3.3 Physical Layer Transceiver . 50 4.3.4 Memory . 51 4.3.5 RS-232 . 51 4.3.6 FPGA Configuration . 52 4.4 Summary . 53 5 Design Implementation 54 5.1 Overview . 54 5.2 Schematic Design . 54 5.3 PCB Layout . 55 5.3.1 Component Placement . 55 5.4 Power Supply . 57 CONTENTS viii 5.4.1 Power Supply Requirements . 57 5.4.2 Galvanic Isolation . 59 5.4.3 Power Supply Options . 59 5.4.4 FPGA Power Planes . 61 5.4.5 Switch Power Planes . 62 5.4.6 Gigabit PHY Power Planes . 63 5.5 Layer Stack . 64 5.6 Clocks . 66 5.6.1 Oscillator Selection . 66 5.6.2 Clock Distribution . 67 5.6.3 Reset Circuit . 71 5.6.4 Signal Connections . 72 5.7 Signal Integrity . 73 5.7.1 IBIS (Input Output Buffer Information Specification) Models . 75 5.7.2 Altium Designer 2004 Signal Integrity Simulator . 76 5.7.3 Track Length Matching . 78 5.8 Prototype Manufacturing . 79 5.9 Summary . 79 6 Firmware Design 81 6.1 Overview . 81 6.2 FPGA Functional Hardware Components . 82 6.3 I2C Interface . 83 6.4 High-Precision Timestamp Unit . 85 6.4.1 Counter Speed Testing . 86 6.4.2 Interpolated Clocks . 87 6.4.3 Timestamp Retrieval . 89 6.5 Packet Monitoring and Timestamp Adjustment . 92 6.5.1 Time-Interval Calculation . 92 6.5.2 One-step Timestamping, On-the-fly . 93 6.5.3 Packet-Type Monitoring . 95 6.5.4 Time-Interval Field Adjustment . 98 CONTENTS ix 6.6 CRC Generator . 99 6.7 Switch Management . 102 6.7.1 CPU Interface . 102 6.7.2 Processor Firmware . 104 7 Results and Evaluation 106 7.1 Overview . 106 7.2 Hardware Testing . 106 7.2.1 Power Supply Testing . 106 7.2.2 FPGA Testing . 107 7.2.3 Memory Testing . 107 7.2.4 RS-232 Testing . 107 7.2.5 Clock Testing and Calibration . 107 7.2.6 Switch and PHY Testing . 108 7.2.7 General Packet Monitoring . 109 7.3 Firmware Design Testing . 109 7.3.1 IEEE 1588 Packet Monitoring . 109 7.3.2 CRC Adjustment . 110 7.3.3 IEEE 1588 Time-Interval Adjustment . 110 7.3.4 High-Precision Timestamp Unit Testing . 110 7.4 Switch Testing Environment . 111 7.5 Transparent Switch Jitter . 113 7.6 Packet Delay Measurements . 115 7.6.1 Master and Slave Direct Connection Delay .