The lpGBT: a radiation tolerant ASIC for Data, Timing, Trigger and Control Applications in HL-LHC Paulo Moreira, CERN On behalf of the lpGBT team Topical Workshop on Electronics for Particle Physics 02 – 06 September 2019, Santiago de Compostela – Spain lpGBT Team Design/Test • CERN David Porret, Daniel Hernandez, Julian Mendez, Jose Fonseca, Ken Wyllie, Nour Guettouche, Paulo Moreira, Pedro Leitao, Rui Francisco, Sophie Baron, Stefan Biereigel, Szymon Kulis • AGH UST Marek Idzik, Miroslaw Firlej, Jakub Moroń, Tomasz Fiutowski, Krzysztof Swientek • KU Leuven Bram Faes, Jeffrey Prinzie, Paul Leroux • UNL – FCT João Carvalho, Nuno Paulino • SMU Physics Datao Gong, Di Guo, Dongxu Yang, Jingbo Ye, Quan Sun, Tiankuan Liu, Wei Zhou • SMU Engineering Tao Zhang, Ping Gui Building blocks • Czech Technical University Prague Miroslav Havranek, Tomas Benka • CERN Stefano Michelis, Iraklis Kremastiotis, Alessandro Caratelli, Kostas Kloukinas 2 lpGBT Link Architecture High radiation doses No or small radiation doses LHC: up to 100 Mrad (1014 1MeV n/cm2) HL – LHC: up to 1 Grad (1016 1MeV n/cm2) Short distance optical links: 50 to 300 m FPGA Timing & Trigger Timing & Trigger TIA PD DAQ lpGBT DAQ LD LD Slow Control Slow Control Custom ASICs On-Detector Off-Detector Radiation Hard Electronics Commercial Off-The-Shelf (COTS) Electrical links to the frontend modules. Lengths: cm to few m Custom optoelectronics 3 More than a “Communications ASIC” • Capable of • 5.12 or 10.24 Gbps (for uplinks) • 2.56 Gbps (for downlinks) • Enables the implementation of RadTol links • DAQ • Trigger (constant and deterministic latency) • Experiment control [slow control] • Implements Control and Monitoring Functions • Three I2C Masters • 16 – bit General Purpose I/O port • Output reset pin • 10 – bit ADC (8 multiplexed inputs) Pin count: 289 (17 x 17) • 8 – bit voltage DAC Pitch: 0.5 mm • 8 – bit current DAC Size: 9 mm x 9 mm x 1.25 mm • Temperature sensor • Designed for radiation hardness • M. Firlej, et al., “An lpGBT sub-system for Total Ionizing Dose (TID): 200 Mrad environmental monitoring and control of • Extensive SEU protection (TMR, FEC) experiments” (Fri @ 09:50) 4 Communicates with • The counting room • Optical fibre links • The FE modules / ASICs • Electrical links (eLinks) • The Number and Bandwidth of eLinks is programmable • For Down eLinks • Bandwidth: 80/160/320 Mbps • Count: 16/8/4 • For Up eLinks 5 eLinks Special Features • Ad hoc signalling “standard” (CLPS) • Differential for noise immunity and EMI • 100 W for good signal integrity • Low voltage swing for low power • eLink drivers (eTx) • Programmable amplitude • Programmable pre-emphasis • Data invert function • eLink Receivers (eRx) • Programmable equalization (3 settings) • Data invert function D. Guo, et al., “The eTx line driver and the eRx line receiver: two building blocks • Hi – Z or 100 W termination for data and clock transmission using the • External biasing for DC coupling CLPS standard” (Tue @ 17:20) • Internal biasing for AC Coupling • Phase – aligner mechanisms for error S. Kulis, et al. “A multi-channel multi-data free data reception rate circuit for phase alignment of data in • manual / semi-automatic / automatic the lpGBT” (Fri @ 11:30) 6 High – Speed links Special Features • Transmitter RX Continuous Time Linear Equalizer • 5.12 / 10.24 Gbps • Programmable amplitude • Programmable pre-emphasis • Receiver • 2.56 Gbps • Programmable equalization • Reference and Reference-less locking • Built-In test features • HS – Loopback (2.56 Gbps) • PRBS generators / checkers • Eye Opening Monitor Q Sun, et al., “Downlink Equalization and • FEC error counter Eye Opening Monitor in the lpGBT” (Thu @ 10:00) 7 Timing Functions • Deterministic & fixed latency • For clocks • For data (up and down links) • Programmable frequency clock outputs • 29 outputs • 40 / 80 / 160 / 320 / 640 / 1280 MHz • Programmable phase/frequency clock outputs • 4 outputs • 50 ps phase-resolution • 40 / 80 / 160 / 320 / 640 / 1280 MHz S. Biereigel, et al., “The lpGBT PLL and • Low jitter CDR Architecture, Performance and SEE • RMS < 5 ps @ 40 MHz Robustness” (Fri @ 09:00) 8 Built In Test Features • Ring oscillators All curves normalized to CH1 at 1.2V • Four instances VDD = 1.08 / 1.2 / 1.32 V • One in each “corner” of the ASIC “Shades” correspond to the four channels • The chip can measure their oscillation frequency T = 10˚ C f ≈ 40 MHz @ 1.2 V • To estimate the “process corner” nom • To track the degradation with TID • Not a “dosimeter”! • SEU Monitoring Memory • 2500 Self Resetting Memory cells • Upset rate estimation! • TMR Testing • Enables/disables clock TMR • Verification of TMR during production 9 The lpGBT is Low-Power! • CMS PS Module (10Gbs, FEC5) 410 mW (@ Vdd = 1.2V) • Uplink 14 x 640 Mbps • Downlink 2 x 320 Mbps • eClocks 2 x 320 MHz + 2 x 640 MHz • CMS 2S Module (5Gbs, FEC12) 330mW (@ Vdd = 1.2V) • Uplink 10 x 320 Mbps • Downlink 2 x 320 Mbps • eClocks 2 x 320 MHz • CMS Pixels (10Gbs, FEC5) 320mW (@ Vdd = 1.2V) • Uplink 7 x 1.28 Gbps • Downlink 7 x 160 Mbps • eClocks disabled • ATLAS ITK Strips (10Gbs, FEC5) 380 mW (@ Vdd = 1.2V) • Uplink 14 x 640 Mbps • Downlink 8 x 160 Mbps • eClocks 4 x 160MHz • Low speed system (10Gbs, FEC5) 490 mW (@ Vdd = 1.2V) • Uplink 28 x 320 Mbps • Downlink 16 x 80 Mbps • eClocks 16 x 40 MHz 10 lpGBT Prototyping • Tapeout • 25/07/2018 • 1st Prototypes • 300 Pieces • Foundry done: 12/10/2018 (die at CERN) • Packaging done: 21/11/2018 (packaged chips at CERN) • Evaluation testing • Chips proved “99.9% functional”! • A few “functions” need performance improvement! • 2nd Prototype batch • 300 chips ordered Dec 2018 • 1st + 2nd batch • 50 “naked” chips for SEU and TID testing • 150 ASICs distributed to the users • Currently 400 tested chips still available for distribution • 3rd Prototype batch • 400 chips ordered July 2019 11 Testing • Test system Evaluation Testing @ CERN • Based on the Xilinx Virtex-7 FPGA VC707 Evaluation Kit Testing started November 2018 and a custom mezzanine board • Test cycle: • More than 1000 parameters measured at 7 supply SEU – Heavy Ions Heavy Ion Facility (HIF) voltages (including [rough] characterization of some March 2019 @ UCL Louvain, Belgium circuits: BERT, ADC, DAC,…) • All this in ≈ 220 s SEU – Two Photon Testing Two Photon Absorption (TPA) facility May 2019 @ KU Leuven, Belgium TID – X-Ray (up to 500 Mrad) X – Ray facility May / June 2019 @ CERN TID – Annealing @ CERN July 2019 Temperature (-25 ˚C to 60 ˚C) @ CERN J. Mendez, et al., “LpGBT Tester: an FPGA S. Biereigel, et al., “Methods for Clock July 2019 based test system for the lpGBT ASIC” Signal Characterization using FPGA (Tue @ 17:20) Peripherals” (Thru @ 16:55) 12 Performance (1/…) Function Circuit Functionality Performance TID SEU ASIC Control I2C Slave (1) IC channel N.A. EC port N.A. Configuration memory X eFuses X N.A. Startup & WD Power ON RST Brown-Out Detector X X N.A. (1) See note (7) Legend – Functionality/performance good! – Functionality/performance satisfactory but some aspects of it need improvement! X – Functionality/performance unsatisfactory must be reviewed! N.A. – Does Not Apply or wasn’t specifically tested. 13 Performance (2/…) Function Circuit Functionality Performance TID SEU HS Transmitter PLL/CDR (2) Tx @ 5.12 Gbps Tx @ 10.24 Gbps PRBS Generators N.A. HS loopback N.A. N.A. HS Receiver Equalizer CDR Frame Aligner (3) Rx @ 2.56 Gbps Eye Opening Monitor N.A. N.A. PRBS Checkers N.A. N.A. (2) See: S. Biereigel, et al., “The lpGBT PLL and CDR Architecture, Performance and SEE Robustness” (Fri @ 09:00) (3) Frame aligner fails to capture some of the unlock conditions making for a relatively long relocking time after an SEU 14 Performance (3/…) Function Circuit Functionality Performance TID SEU ePortRx Deserializer (4) Phase-Aligner PRBS Checkers N.A. N.A. ePortTx Serializer PRBS Generators N.A. N.A. Mirror function eTx eTx + pre-emphasis eRx eRx + equalization (4) See: S. Kulis, et al. “A multi-channel multi-data rate circuit for phase alignment of data in the lpGBT” (Fri @ 11:30) 15 Performance (4/…) Function Circuit Functionality Performance TID SEU Experiment control and monitoring I2C master (x3) (5) N.A. GPIO REST OUT Reference voltage N.A. Temperature sensor N.A. 8[12]-bit Voltage DAC N.A. 8-bit Current DAC N.A. 10-bit ADC N.A. (5) The Yield of the I2C masters is low. Particularly, in about 43% of the ASICs at least one of the I2C masters fails. This is due to a timing problem (identified) in the logic. The yield of master “0” is the lowest. “Curiously”, some of the channels might “popup back into life” with Total Irradiation Dose! 16 Performance (5/…) Function Circuit Functionality Performance TID SEU Clocks eClocks (6) Phase-Shifter Process Monitors and Testing Ring oscillator (x4) N.A. SEU Monitoring Memory N.A. TMR Testing (7) N.A. N.A. N.A. Power Consumption (6) Measurements show the presence of systematic jitter (approaching ±10 ps P-P) in all the clocks except at 40 MHz. Additionally, significant duty cycle distortion is present in the high frequency clocks. This phenomena is not yet well understood, although the systematic jitter can be correlated with digital activity in the ASIC. (7) A logic mistake (identified) prevents accessing the ASIC through the I2C interface once one of the TMR clocks (40 MHz) is disabled! 17 TID and Power Consumption @ 10 Gbps & FEC 12 1.32 V 1.20 V 1.08 V VDD (“All that is digital + I/O”) VDDTX (HS TX) VDDRX (HS RX + PLL + CLK GEN) 18 eFuses • At least for lpGBTs serving as masters, configuration data is read from the eFuses at Power-Up Reading the eFuses: Error Probabilities • This has to be done reliably so that communications can be established with the frontend modules and the counting room • However, X-ray irradiation tests have proven the fuse reading process to degrade with TID • At around 150 Mrad, “bit flips” can be observed during the read process! • Solutions to be implemented • Redesign the sense amplifier to avoid an offset buildup with TID • Add CRC to the configuration.