USB4TM Time Synchronization, Host Interface, Connection Manager and ThunderboltTM 3 Compatibility Uri Hermoni – Architect, Intel Corporation Gal Yedidia – Architect, Intel Corporation USB Developer Days 2019 – Taipei, Taiwan November 20, 2019 USB Type-C®, USB-C® and USB4™ are trademarks of the Universal Serial Bus Implementers Forum (USB-IF). Thunderbolt™ is a trademark of Intel Corporation. All product names are trademarks, registered trademarks, or service marks of their respective owners. USB Implementers Forum © 2019 • Time Synchronization • Host Interface Agenda • Connection Manager • ThunderboltTM 3 Compatibility 2 USB Implementers Forum © 2019 • Time Synchronization • Host Interface Agenda • Connection Manager • ThunderboltTM 3 Compatibility 3 USB Implementers Forum © 2019 • Why Time Synchronization is needed? • Time Synchronization Mechanism Time Sync • Time Sync Operation Modes • Time Calculation Agenda • Recursive Propagation • Use of Time Synchronization in tunneled protocols 4 USB Implementers Forum © 2019 Why Time Synchronization is Needed? • DisplayPort Tunneling • Link Clock restoration in the Monitor side based on the GPU side • Need a Grandmaster Time to measure the two clocks and adjust • USB3 Tunneling • Isochronous Timestamp Packets correction • Delta should include the tunnel delay • PCIe Tunneling • Precision Time Measurement messages requires time sense • Handshake transforms to use the Grandmaster Time 5 USB Implementers Forum © 2019 Time Synchronization Mechanism Grandmaster • Each Router has its own clock (>125MHz±100ppm) Host Router M M • Host Router holds the Grandmaster Time S S Router A Router B • Each Router holds a Local Time Counter and need to M M M M calculate • TimeOffsetFromGM S S • FreqOffsetFromGM Router C Router D M M M M S S Router E Router F M M 6 USB Implementers Forum © 2019 Time Sync Operation Modes • There are two types Resolution Uni-directional Bi-directional • Bi-directional LowRes TSNOS every 1ms N/A • Uni-directional HiFi TSNOS every 16us 2-way handshake every 16us • There are two resolutions • HiFi • LowRes • Time Sync Notification Ordered Set (TSNOS) tells the Router when to timestamp • Information than sent on the Follow Up Packet 7 USB Implementers Forum © 2019 Bi-directional Time Sync • Initiated by Slave • Periodically send TSNOS every 16us • Slave timestamp when TX and RX TSNOS • Master timestamp when RX, immediately responds and timestamp when TX • Follow Up packet holds information from Master to Slave 8 USB Implementers Forum © 2019 Bi-directional Time Sync • Static latency between Slave and Master 푇 − 푇 − (푇 − 푇 ) 퐷 = 4 1 3 2 2 • Time offset between Master and Slave 푇푖푚푒푂푓푓푠푒푡 = 푇3 − 푇4 + 퐷 • Frequency ratio between Master and Slave 푇 푛 − 푇 [푚] 퐹푟푒푞푢푒푛푐푦푅푎푡푖표 = 4 4 푇3 푛 − 푇3[푚] • Frequency offset between Master and Slave 퐹푟푒푞푢푒푛푐푦푂푓푓푠푒푡 = 퐹푟푒푞푢푒푛푐푦푅푎푡푖표 − 1 × 241 9 USB Implementers Forum © 2019 Uni-directional Time Sync • Initiated by Master • Periodically send TSNOS every 16us/1ms • Master timestamp when TX TSNOS • Slave timestamp when RX TSNOS • Follow Up packet holds information from Master to Slave • T2 is empty 10 USB Implementers Forum © 2019 Uni-directional Time Sync • Static latency between Slave and Master can’t be measured • Time offset between Master and Slave 푇푖푚푒푂푓푓푠푒푡 = 푇3 − 푇4 + 퐷 • Frequency ratio between Master and Slave 푇 푛 − 푇 [푚] 퐹푟푒푞푢푒푛푐푦푅푎푡푖표 = 4 4 푇3 푛 − 푇3[푚] • Frequency offset between Master and Slave 퐹푟푒푞푢푒푛푐푦푂푓푓푠푒푡 = 퐹푟푒푞푢푒푛푐푦푅푎푡푖표 − 1 × 241 11 USB Implementers Forum © 2019 Recursive Propagation • Each Slave Router calculate offsets from Master Router • OffsetFromGM is sent as part of the handshake • TimeOffsetFromGM • FreqOffsetFromGM • Each Router can calculate offset from Grandmaster • In Inter-Domain systems there is a special handshake to allow Time Sync between two Hosts 12 USB Implementers Forum © 2019 Time Accuracy & Convergence • Accuracy depends on TSPacketInterval (1ns/4ns) • Convergence should be within tConvergeTime • Results are filtered to improve accuracy 13 USB Implementers Forum © 2019 • Why Time Synchronization is needed? • Time Synchronization Mechanism Time Sync • Time Sync Operation Modes • Time Calculation Agenda • Recursive Propagation • Use of Time Synchronization in tunneled protocols 14 USB Implementers Forum © 2019 DisplayPort tunneling: Link Clock Restoration • DisplayPort is tunneled from GPU to Monitor over USB4 • The video stream is restored at the Monitor side as it is sampled from GPU • In order to recreate the video after tunneling, time sense is required Router A Router B Router C DP Source DP Sink Main-Link Main-Link AUX CH DP IN DP OUT AUX CH DPTX DPRX Adapter Adapter HPD HPD USB4 Link USB4 Link TMU TMU TMU PLL DP Link DP Link 15 USB Implementers Forum © 2019 DP Link Link Clock Restoration Process • TMU establish time synchronization over USB4 Link • Grandmaster Time propagate through the entire USB4 tree Router A Router B Router C DP Source DP Sink Main-Link Main-Link AUX CH DP IN DP OUT AUX CH DPTX DPRX Adapter Adapter HPD HPD USB4 Link USB4 Link TMU TMU TMU PLL DP Link DP Link 16 USB Implementers Forum © 2019 DP Link Link Clock Restoration Process • DisplayPort Link clock is measured at the GPU side based on TMU events Router A Router B Router C DP Source DP Sink Main-Link Main-Link AUX CH DP IN DP OUT AUX CH DPTX DPRX Adapter Adapter HPD HPD USB4 Link USB4 Link TMU TMU TMU PLL DP Link DP Link 17 USB Implementers Forum © 2019 DP Link Link Clock Restoration Process • USB4 packet with the measurement result is sent over USB4 network to the Router at the Monitor side Router A Router B Router C DP Source DP Sink Main-Link Main-Link AUX CH DP IN DP OUT AUX CH DPTX DPRX Adapter Adapter HPD HPD USB4 Link USB4 Link TMU TMU TMU PLL DP Link DP Link 18 USB Implementers Forum © 2019 DP Link Link Clock Restoration Process • DisplayPort Link clock is measured at the Monitor side based on TMU events • The two measurements are compared • Error correction is computed Router A Router B Router C DP Source DP Sink Main-Link Main-Link AUX CH DP IN DP OUT AUX CH DPTX DPRX Adapter Adapter HPD HPD USB4 Link USB4 Link TMU TMU TMU PLL DP Link DP Link 19 USB Implementers Forum © 2019 DP Link Link Clock Restoration Process • DisplayPort Link clock frequency in the Monitor side is shifted according to the error correction result • Entire mechanism is executed periodically Router A Router B Router C DP Source DP Sink Main-Link Main-Link AUX CH DP IN DP OUT AUX CH DPTX DPRX Adapter Adapter HPD HPD USB4 Link USB4 Link TMU TMU TMU PLL DP Link DP Link 20 USB Implementers Forum © 2019 DP Link PCIe Precision Time Measurement • Supporting components determine the relationship between their local time and a PTM Master Time • PTM Handshake is used to determines the Link latency • PTM ResponseD holds the PTM Master Time (t2’) and the Propagation Delay (t3-t2) 푡4−푡1 − 푡3−푡2 • PTM Master Time at 푡1′ = 푡2′ − 2 21 USB Implementers Forum © 2019 PCIe Tunneling: PTM Time Restoration PCIe Root Complex • Precision Time Measurement (PTM) message is tunneled over USB4 Control Adapter Downstream PCIe Switch USB4 PTM PCIe Adapter • TMU PTM messages need USB4 time sense in order to recreate the PTM Router Master Time at the Endpoint side USB4 Host USB4 Port • The Tunneled PTM holds two parameters to describe the relation between the PTM Master Time and the USB4 TMU Grandmaster USB4 Port USB4 PTM Upstream PCIe Time Adapter Control Adapter • The parameters replace the PTM Master Time and Propagation PCIe Switch Router TMU Downstream Delay fields in the PTM message USB4 PTM PCIe Adapter • PCIe Endpoint can restore the PTM Master Time from the TMU USB4 Hub USB4 Port USB4 Port Grandmaster Time and generated parameters USB4 Port USB4 PTM Upstream PCIe Adapter Control Router Adapter PCIe Switch TMU PCIe Endpoint USB4 Device 22 USB Implementers Forum © 2019 TMU to PTM Parameters • The parameters represent the linear relationship between the PTM Master Time and TMU Grandmaster Time PTM • The parameters Generator is the Host Router connected to the PCIe Time Root Complex ptm1 • TMU_to_PTM_A represent the slope of the linear line TMU_to_PTM_A • TMU_to_PTM_B represent the intercept of the linear line ptm0 • Ideally: ptm0 = TMU_to_PTM_A * tmu0 + TMU_to_PTM_B TMU_to_PTM_B ptm1 = TMU_to_PTM_A * tmu1 + TMU_to_PTM_B tmu0 tmu1 TMU Time • Due to quantization error a more robust method described in the spec • It is recommended to calculate the parameters periodically 23 USB Implementers Forum © 2019 PTM Master Time Reconstruction • PCIe Endpoint can use the TMU Grandmaster Time and generated parameters to reconstruct the PTM Master Time PTM Master Time(t) = TMU_to_PTM_A * TMU Grandmaster Time(t) + TMU_to_PTM_B 24 USB Implementers Forum © 2019 USB Tunneling: ITP Timestamp Correction • USB3 uses Isochronous Timestamp Packets to deliver timestamps from Host to all active Devices • The Isochronous Timestamp field holds the Bus Interval counter and Delta 25 USB Implementers Forum © 2019 ITP Timestamp Correction • Timestamp is taken before the ITP enters the tunneling and is added to the tunneled packet 26 USB Implementers Forum © 2019 ITP Timestamp Correction • Another timestamp is taken when ITP exits the tunnel • Both timestamps are in TMU Grandmaster Time • An Upstream USB3 Adapter that receives an ITP tunneled packet updates the Delta field according to the calculated propagation delay (in tIsochTimeStampGranularity resolution) • It also update the CRC-16 field Bus Delta before Delta after Bus Interval tunneling tunneling Interval N N+1 Original Tunneling Delta Propagation