Particle Tracking and Identification on Multi- and Many-Core Hardware
Total Page:16
File Type:pdf, Size:1020Kb
PARTICLETRACKINGANDIDENTIFICATIONONMULTI-AND MANY-COREHARDWAREPLATFORMS by plácido fernández declara A dissertation submitted by in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science and Technology Universidad Carlos III de Madrid Advisors: José Daniel García Sánchez Niko Neufeld September 2020 This thesis is distributed under license “Creative Commons Attribution – Non Commercial – Non Derivatives”. To my parents. A mis padres. We have seen that computer programming is an art, because it applies accumulated knowledge to the world, because it requires skill and ingenuity, and especially because it produces objects of beauty. — Donald E. Knuth [87] ACKNOWLEDGMENTS Since I finished my master’s degree and started working inthe aerospace industry with GMV I have been interested in pursuing a PhD. But I did not find the right challenge or topic I was looking for. One day I received a call from my professor from Computer Science and Engineering degree and director of the master’s degree I would later study in Leganés. He let me know about an opening position to do a PhD in High Performance Computing at CERN, in Switzerland. It was just what I was looking for; an incredible computing challenge working in the Large Hadron Collider. I applied for it and some interviews later I got the job. I have to thank first and foremost my parents. It is thanks tothem that I was able to purse any dream and challenge I ever imagined and I am ever thankful for their love and support. To my love and friend Afri, who helps me overcome any situation and makes any adventure an amazing journey to enjoy life. I would like to thank my friends in Madrid for endless hours of explorations, discussions and joy in the good and bad times. To my childhood friends Javi and Josemari for every minute in the holm oak woods and the surrounding neighbourhoods. To Miguel, Guille, Centeno and all the group for adventure, climbing and nights in the mountains. To all the people that supported me in GMV... To Jordi, Albano, Patricio, Puma, Aven, Romo, Oso and Abel for many, many hours of rehearsal, all the concerts around Spain and experiences. To Cristian who took care of me in the river and plains. To Jorge for we supported each other to improve and finish our studies. To Auro and Álex who showed and accompany us on discovering the Alps, the lakes and the area. I am eternally grateful to my family who always have been supportive and encouraged me to achieve the career and life I wanted. I am grateful to Niko and Omar for showing me the way in my doc- toral studies. To Jose Daniel for the advice, guiding and conversations all along my career. To Daniel who made the experience at CERN so much more enjoyable. To Luismi for the discussions at CERN and exploring the mountains together. To Dorothea, Flavio, Rainer and Roel for all the help to make Allen an amazing thing. Thank you to all the people from the LHCb experiment and CERN that I worked with, iv for showing me the way of physics, for the BBQs at Point 8, endless Factorio games and all the great discussions. Thanks to my colleagues at Universidad Carlos III de Madrid in the ARCOS research group. Firstly, I would like to thank to David del Río, Manuel Dolz and Javier Fernández for their help with the integration with GrPPI. Secondly, I would like to also thank Javier Garcia-Blas who helped me with the experiments in GPU architectures. Thanks to the Spanish MINISTERIO DE ECONOMÍA Y COMPETI- TIVIDAD though project grant TIN2016-79637-P TOWARDS UNIFI- CATION OF HPC AND BIG DATA PARADIGMS, the EU Project ICT 644235 "REPHRASE: REfactoring Parallel Heterogeneous Resource- Aware Applications", Spanish "Ministerio de Economía y Competitivi- dad" under the project grant TIN2016-79637-P "Towards Unification of HPC and Big Data Paradigms", and Madrid Regional Government, CABAHLA-CM (ConvergenciA Big dAta-Hpc: de Los sensores a las Aplicaciones) grant number S2018/TCS-4423. v PUBLISHEDANDSUBMITTEDCONTENT Publications and research contributions from the author included as part of this thesis: Plácido Fernández et al. «Parallelizing and Optimizing LHCb- Kalman for Intel Xeon Phi KNL Processors.» In: 2018 26th Eu- romicro International Conference on Parallel, Distributed and Network- based Processing (PDP). IEEE. 2018, pp. 741–750. doi: 10.1109/ PDP2018.2018.00121 • This publication is wholly included in this thesis in Chap- ter 6. • The material from this source included in this thesis is not singled out with typographic means and references. Placido Fernandez Declara et al. «A parallel-computing algo- rithm for high-energy physics particle tracking and decoding using GPU architectures.» In: IEEE Access 7 (2019), pp. 91612– 91626. issn: 2169-3536. doi: 10.1109/ACCESS.2019.2927261 • This publication is wholly included in this thesis in Chap- ter 7. • The material from this source included in this thesis is not singled out with typographic means and references. Placido Fernandez Declara and J. Daniel Garcia. «Compass SPMD: a SPMD vectorized tracking algorithm.» 2020. Accepted for publication at CHEP 2019 Proceedings, Adelaide, Australia • This publication is partly included in this thesis in Chap- ter 8. • The material from this source included in this thesis is not singled out with typographic means and references. vi OTHERRESEARCHMERITS A list of publications and research contributions produced during the work of this thesis is presented here: Publications: Plácido Fernández et al. «Parallelizing and Optimizing LHCb- Kalman for Intel Xeon Phi KNL Processors.» In: 2018 26th Eu- romicro International Conference on Parallel, Distributed and Network- based Processing (PDP). IEEE. 2018, pp. 741–750. doi: 10.1109/ PDP2018.2018.00121 Placido Fernandez Declara et al. «A parallel-computing algo- rithm for high-energy physics particle tracking and decoding using GPU architectures.» In: IEEE Access 7 (2019), pp. 91612– 91626. issn: 2169-3536. doi: 10.1109/ACCESS.2019.2927261 Placido Fernandez Declara and J. Daniel Garcia. «Compass SPMD: a SPMD vectorized tracking algorithm.» 2020. Accepted for publication at CHEP 2019 Proceedings, Adelaide, Australia Roel Aaij et al. «Allen: A High-Level Trigger on GPUs for LHCb.» In: Computing and Software for Big Science 4.7 (2020). doi: 10. 1007/s41781-020-00039-7 Posters: Placido Fernandez Declara. «CompassUT: study of a GPU track reconstruction for LHCb upgrades.» 2019. url: https://cds. cern.ch/record/2665033. Poster presented at Winter LHCC sessions, CERN, Switzerland Placido Fernandez Declara. «Compass SPMD: a SPMD vector- ized tracking algorithm.» 2019. url: https://cds.cern.ch/ record / 2699802. Poster presented at CHEP 2019, Adelaide, Australia Talks at international conferences: Placido Fernandez Declara. Fast Kalman Filtering: new approaches for the LHCb upgrade. Tech. rep. 2018. url: http://cds.cern.ch/ record/2631784 Technical reports: LHCb Collaboration. LHCb Upgrade GPU High Level Trigger Technical Design Report. Tech. rep. CERN-LHCC-2020-006. LHCB- TDR-021. Geneva: CERN, 2020. url: https://cds.cern.ch/ record/2717938 vii I am also a co-author in a number of LHCb related papers. The full list can be found at: Placido Fernandez Declara. Google Scholar profile. https : / / scholar.google.com/citations?user=Ygkq_7YAAAAJ. 2020 viii ABSTRACT Particle track reconstruction in high-energy physics is used for pur- poses that include simulation, particle analysis, and particle collision filtering and recording. The LHCb experiment at CERN is undergoing an upgrade where improvements are being applied to the detectors, hardware and software. One of the main changes of this upgrade includes the removal of the "hardware filter" which used custom elec- tronics to filter particle collisions, and its substitute: the "software filter". This new filter is composed of commodity hardware which must be able to process a data rate of 40 Tb per second in real-time. Different architectures are being considered to achieve this goal, and the software used to compute it must be optimized and improved to achieve the target data throughput. This software filter is used to reconstruct particle collisions, also known as events, including the trajectories of the resulting particles, which later are analyzed and used to help explain problems like the matter-antimatter asymmetry. This thesis explores different opportunities with multi and many- core architectures, to improve the throughput processing of particle collisions, and the maintainability and improvement of the source code used for it. The Kalman filter algorithm is widely used in high-energy physics for particle reconstruction, and the Intel Xeon Phi KNL processor offers a many-core x86 architecture that is well suited for parallel workloads. Performance and maintainability improvements are discussed, where optimization are targeted towards the Intel Xeon Phi processor. GPU architectures are a good fit for high-energy physics workloads, where its highly parallel architecture can benefit the throughput pro- cessing of it. A GPU framework for event filtering is discussed, in particular the optimizations and changes implemented to a tracking algorithm to deliver high-throughput. Finally vectorization opportunities for CPUs are explored by using data-oriented algorithms and constructs that benefit the vector units found in x86 processors. A SPMD compiler that resembles program- ming for GPUs is used to improve the readability and performance of these algorithms. ix RESUMEN La reconstrucción de trayectorias en física de partículas se usa con distintos fines entre los que se incluyen la simulación, el análisis yel filtrado y recogida de las colisiones entre protones. El experimento LHCb del CERN se encuentra en pleno proceso de actualización en el que cambios y mejoras serán aplicadas a los detectores, los procesadores y el software. Uno de los principales cambios incluye la eliminación del "filtro hardware" basado en circuitos integrados específicos para estas aplicaciones, por un "filtro software".