Argonne TDAQ/Electronics Roadmap
Total Page:16
File Type:pdf, Size:1020Kb
Argonne TDAQ/electronics roadmap A. Paramonov Argonne National Laboratory High Energy Physics Division 13 October 2017 Brief Overview of Argonne HEP Electronics Group ‣ 8 Staff with variety of skills: 5 engineers, CAD Layout, Instrument Repair, 3 technicians • Provides electronics design & support for Argonne HEP experiments • Supports many projects at the laboratory ‣ Group expertise • High Speed Digital Design: Data acquisition systems, Trigger Processors, Custon-to-commodity networks interface, High Speed Communication (copper and optical links) • Front End Design: Charge Amplifiers, Digitizers, Discriminator, HV, and Switching Power Supplies, - No IC design tools available at Argonne • Reliability testing for radiation tolerance - Failure mode analysis - single event upsets - Design for radiation environement • R&D: Wireless DAQ, Digital HCAL Electronics 2 Recently completed and ongoing projects 1. ATLAS experiment at CERN ongoing and completed • FTK to Level-2 Interface Card (firmware, and boards) • FELIX (firmware) • Hardware Track trigger interface (HTT IF) (firmware) • FELIX for the HL-LHC upgrade of ATLAS (firmware and board production) • Concluded work on Tile Cal upgrade (LVPS, HV_OPTO, QIE FEB, QIE MB, project engineering) 2. proto DUNE ongoing • Front-end electronics for Photon detectors (design of the boards and firmware) 3. CTA all completed • L2 trigger (boards + firmware) 4. g-2 all completed • Trolley motor control • Field measurement electronics 5. R&D Development of wireless techniques in data and power transmission application for particle-physics detectors. 6. Electronics for quantum computers at the University of Chicago ongoing • Digital synthesizer daughter board 7. GRETINA TDAQ System 3 Recent and current projects Synthesizer for a quantum computer QIE front-end and HV distribition systems for ATLAS Tile Calorimeter 4 NMR Readout for Fermilab g-2 Experiment ‣ g-2 depends on two numbers: spin precession frequency & B field seen by muons • 70 ppb measure of B required to achieve experiment target accuracy • Achieved 300 parts-per-trillion! measure of B with the redesigned trolley electronics ‣ Field measured with array of 17 NMR probes as trolley moved through storage ring • Refurbish & upgrade BNL g-2 trolley • Digitize NMRs, Control NMR sequence, and readout external sensors (barcode for positional location, temperature, pressure,…) ‣ Special requirements • N2 atmosphere in trolley cylinder; vacuum outside • difficult heat dissipation; low power for components in trolley; <1-1.5W • Minimize distortion of ring magnetic field; non- magnetic components • Minimize cables going around ring 5 Trolley Electronics Upgrade for Fermilab g-2 ‣ Replace obsolete microcontroller with low-power SmartFusion chip (ARM-3 + FPGA) for high speed communication ‣ Full digitization of NMR signal for consistent readout of all field probes 6 The main motion controller for the trolley . 6U rack mount chassis will include: – Galil motion controller – Custom made interface board from Argonne • 8 motor channels have interface to Shinsei driver, 2 limit switches and 1 encoder • 8 analog channels with differential to single ended receivers connected to Galil ADC – Power supply Motor channel Analog channel 7 Photon detector readout for proto-DUNE . The initial requirements for the readout system targeted high-performance . The goal is to understand performance of the Silicon Photo-multiplier photo-detectors . The challenge is to readout the SiPMs though long (~20m) cables. 8 Diagram of the photo-detector readout system 9 Photo-detector readout system status . The boards were manufactured in the spring of 2017. – Power supplied and everything else are contended in the box. – 1 box = 12 channels. Initial on-the desk tests to characterize the digital readout (1G Ethernet) and the analog performance. The integration with the rest of the system is ongoing. 10 Electronics for Dynamic Quantum Computing Experiments at the University of Chicago • Typical qubit measurement involves sending a microwave pulse to a readout resonator, recording a reflected or transmitted signal, and reducing the record to a binary decision regarding the qubit state. • FPGAs are needed for qubit control since CPUs do not give desirable latency. • Cheap and user-friendly. • 1-channel system. ADC is desired to be faster. • Multi-channel system is planned. Red Pitaya board 125 Msps ADC AD9914 Closed-source 3.5 Gbps direct digital platform for synthesizer measurements and instrument control 11 QIE12 ASIC for ATLAS Tile Calorimeter ‣ Gated integrator ‣ Superior handling of the out-of-time pulses that shaper-based front-ends ‣ Front End Board • Designed, fabricated, assembled • 3.9fC least count (shunt on) super low noise. • 17 bits dynamic range • Analog current integrator (uses programmable shunt from QIE) • 16-bit DACs for calibration (current injection) ‣ Main board • fully passive • extensive RF shielding for switching POLs QIE Front End Board 100 GeV electron beam 12 Overview of HVOpto Board (Cont.) • Basic Layout Power & Data Connection to Daughter Card “A” HV Connections to PMTs - Side “A” HV Input Data Connector Power Supplied By the Interface CH00 Circuitry CH01 CH04 CH03 CH02 Power CH05 Split Plane Daughter Board Regulators Isolates Power & Ground Isolates “A” & “B” Circuits CH06 CH08 CH07 CH11 CH10 CH09 Uses Interface Redundancy Circuitry Power HV Output Data Connector of the Regulators (to next board) Power System Power & Data Connection to HV Connections to PMTs - Side “B” Daughter Card “B” 13 ATLAS Trigger & DAQ 1. Fast Tracker for Phase-I (FTK) • Hardware global tracking @ 100kHz - pattern recognition with Associative Memory - track fit with digital signal processor - Argonne work - sytem design and performance study - FTK Lvl 2 Interface Crate (FLIC) - commission production boards at CERN Fall 2015 - System wide installation & testing - Integration with TDAQ and Detectors • Primary vertexing with FTK tracks • Lepton isolation with FTK tracks • Jet selection with FTK tracks 2. Front-End Link Exchange for Phase-I (FELIX) • Bridge between the custom fully synchronous system and Ethernet • Timing Trigger and Control (TTC) - Development environment & testbed at ANL 3. Level-1 Track Trigger (L1Track) R&D for Phase-II • FTK-like Level 1 Track Trigger • Phase-1 Lvl 1 becoming Phase-II Lvl 0 • Working on TDAQ architecture - Trigger strategy and DAQ for iTK and full calorimeter granularity - Initial design report (2016) and TDR (2017) 14 15 Production is complete. Boards are being integrated into the full FTK system 16 Hardware track trigger for the HL-LHC program of the ATLAS experiment • The hardware track processor will be an attachment to the central trigger system. • Will reconstruct tracks with pT>1 GeV. Regional tracking at 1 MHz. Global tracking at 100 kHz. • Argonne is working on delivering data between the commodity network and the HTT boards (HTT IF). • The HTT will use associative memory and FPGAs to identify and fit tracks. • Currently we are detailing architecture specifications for the HTT system. • Can not be replaced with contemporary commodity CPU/GPU-based system because of cost, power consumption, and space. 17 FELIX • Enabling transition from custom hardware to COTS as early as possible • Using high level switch protocols of high speed and large bandwidth • Configurable and flexible data routing and error handling, without relying on detector specific hardware • Direct low latency paths between links • Universal ATLAS-wide TTC/BUSY handling as for Run 1&2 • Command scheduling with guaranteed timing for calibration 18 FELIX as a System FELIX Host Hardware Firmware Detector FE FELIX I/O card Electronics GBT Software I/O I/O Buffer TTC PCIe Buffer BUSY TTC PCIe Gen3 16 lanes BUSY Handler SWROD/ROS CPU running FELIX Application 100 Gbs NIC 100 Gbs NIC 19 FELIX Development TTCfx Custom FMC accepting TTC input Outputing TTC clock and CH A/B info Hitech Global HTG- 710 2 CXP cages Virtex-7 X690T PCIe Gen 3 x 8 lanes BNL-711 Xilinx VC-709 4 miniPOD RX & Tx 4 SFP+ connectors Kintex Ultrascale Mellanox Dual-port 40 Gb Virtex-7 X690T PCIe Gen 3 x 16 Ethernet or Infiniband PCIe Gen 3 x 8 lanes lanes 20 Future directions • Future plan: • The work for ATLAS will continue with focus on DAQ. We will leverage our expertise in FELIX to develop interfaces (e.g. HTT IF) between the custom and commodity systems • There is a need to tailor FELIX to the inner tracker detectors (Pixel and Strip). Argonne is working on integrating FELIX with Pixel readout (RD53A and beyond). High-speed I/O and system complexity is a challenge. • The need for readout electronic for quantum computing will grow as the experiments become more sophisticated. • Electronics for next-gen South Pole Telescope? • A Discussion on DUNE Photon-Detector Electronics is starting now, and the input from ProtoDUNE will be important • Future collider experiments (e.g. for HE-LHC) will require custom electronics (front-end and DAQ). More channels and higher bandwidth than ATLAS or CMS. • Expertise needed to support future experiments • Currently, digital design is in strong demand. The demand will not go away. PCB layout for high speed serial (10… 25Gbps) is a challenge. • The future experiments (e.g. for HE-LHC) will being opportunities to design front-end electronics. The expertise in analog design needs to be maintained. • A lot of on-detector electronics requires IC design; must also interface to the off-detector DAQ 21 Summary ‣ Argonne HEP Electronics Group supports HEP Division experimental