The ATLAS (Offline) Software Attila Krasznahorkay Preliminaries • ATLAS’s software infrastructure is quite complicated • On the whole there’s no one software to talk about • This is an introduction to the ATLAS offline software infrastructure • The analysis software will be discussed on Tuesday • And the trigger software on Thursday • This talk is not specific to Run 2 • While many aspects of the offline software changed for Run 2, the basics didn’t. Which is what will be discussed here. 2 Athena • Is the name of the software framework in which (offline) data processing is performed • Is based on Gaudi, originally developed by the LHCb collaboration • Is used for nearly all aspects of offline (and even online) data processing • Simulation, reconstruction, the High Level Trigger, and even data analysis tasks • Most of the code is written in C++ • But all of the scripting and configuration (and even some algorithmic code) is in Python Athena: Greek goddess of wisdom, war, the arts, industry, justice and skill; sprang fully grown out of her father’s (Zeus’s) head Gaudi: Antoni Gaudi, Barcelona architect 1852 – 1926, famed for the design of Barcelona’s La Sagrada Familia, an immense Basilica which has been under construction since the beginning of the 20th century and is not scheduled for completion until 2026 3 Software Framework • A skeleton into which software developers plug in their code • Defines a high level architecture for the organisation of the software • Provides functionality common to several applications • Provides communication between different components of the software • Controls the configuration, loading and execution of the code that the software developer provides • In the end and absolute necessity once >10 developers have to write code for a common goal… • Some generic description of the basic idea is under: • http://en.wikipedia.org/wiki/Software_framework 4 What Are We Doing? • We are dealing with a lot of structured data • We have individual events that can be treated independent of each other, and need to go through several steps during the data processing • Simulation, reconstruction, analysis • We have metadata (data about data) for the events that we simulate and record • Which trigger stream the event came from • Is it simulated data? • Which trigger menu was used • Which detector geometry was used • … • So the framework needs to load the correct metadata for each processed event, and then process it with the code that the developers wrote 5 Basic Concepts 6 Algorithm • Is pretty much the simplest concept: • A piece of code that initialises itself before the event processing (initialize) • Some code that does something using the event data during event processing (execute) • Finally, some code that possibly does something after the event processing has finished (finalize) 7 Algorithm • In Athena we implement algorithms in C++ very simply like: class MyAlgorithm : public AthAlgorithm { public: MyAlgorithm( const std::string& name, ISvcLocator* svcLoc ) : AthAlgorithm( name, svcLoc ) {} StatusCode initialize() { // Do something… return StatusCode::SUCCESS; } StatusCode execute() { // Do something… return StatusCode::SUCCESS; } StatusCode finalize() { // Do something… return StatusCode::SUCCESS; } }; 8 Algorithm • Is implemented with a C++ class that inherits from one of the algorithm base classes • Generic algorithms inherit from AthAlgorithm • Event filtering algorithms inherit from AthFilterAlgorithm • Histogram filling algorithms inherit from AthHistogramAlgorithm • A few more base classes can be found in the AthenaBaseComps package (Control/AthenaBaseComps) • The Athena framework takes care of: • Running the initialize() function once before the event loop starts • Running the execute() function once per event • Running the finalize() function once after the event loop is finished 9 Reporting Back • Most functions in Athena tell their caller whether they succeeded or not, by returning a StatusCode object • A function will not always be able to do what was asked of it. If it can fail, it needs to report back about its success or failure using StatusCode. • StatusCode is a very simple C++ class that has a few states, and knows whether it was checked or not • Not explicitly checking the StatusCode returned by a function is an error. Which stops the execution of an Athena job. • Checking can be done by using one of the object’s functions (isSuccess()), or comparing it to a value (in an if(…) statement) • The possible values of a StatusCode are: • StatusCode::SUCCESS: The function succeeded • StatusCode::FAILURE: The function failed, and the job needs to stop • StatusCode::RECOVERABLE: The function failed, but the job could continue with the next event. (After discarding the current one.) 10 Reporting Back • Most functions in Athena tell their caller whether they succeeded or not, by returning a StatusCode object • A function will not always be able to do what was asked of it. If it can fail, it needs to report back about its success or failure using StatusCode. • StatusCode is a very simple C++ class that has a few states, and knows whether it was checked or not • Not explicitly checking the StatusCode returned by a function is an error. Which stops the execution of an Athena job. • Checking can be done by using one of the object’s functions (isSuccess()), or comparing it to a value (in an if(…) statement) • The possible values of a StatusCode are: • StatusCode::SUCCESS: The function succeeded • StatusCode::FAILURE: The function failed, and the job needs to stop • StatusCode::RECOVERABLE: The function failed, but the job could continue with the next event. (After discarding the current one.) 10 Hardly ever used… Algorithm Sequence • All the code is never put into a single algorithm • Instead we write multiple algorithms that need to be executed one after the other • Possibly running the same algorithm in multiple instances with different configurations • Is pretty much just a list of algorithms that Athena runs in the specified order • The order applies to the execution of all 3 main functions of the algorithms • An Athena job means more or less the execution of the main algorithm sequence (called “top sequence”) 11 Algorithm Sequence CaloClusterRecoAlg Top Sequence Top IDTrackRecoAlg ElectronRecoAlg 12 Algorithm Sequence • The algorithm sequence class (AthSequencer) is itself an algorithm! • You can add an algorithm sequence to another algorithm sequence Alg1 Top Sequence Top Filter Seq. FilterAlgorithm ConditionalAlgorithm Alg2 13 Algorithm Sequence • The algorithm sequence class (AthSequencer) is itself an algorithm! • You can add an algorithm sequence to another algorithm sequence Alg1 Top Sequence Top Filter Seq. FilterAlgorithm ConditionalAlgorithm STOP ConditionalAlgorithm doesn’t run if FilterAlgorithm didn’t Alg2 “accept” the event 13 Whiteboard • Algorithms don’t know about each other, they don’t talk directly to each other • How do they exchange information then? • Via a whiteboard • All (event) data is published on a central whiteboard, including the data read in from an input file • Each algorithm can retrieve objects that they need from the whiteboard (retrieve) • If an algorithm produces new data, it puts it on the whiteboard as well (record) • In Athena the class implementing the whiteboard is StoreGateSvc • There are multiple instances of the whiteboard for different purposes • “StoreGateSvc” holds event data, and is cleaned at the end of each event • “DetectorStore” holds detector information, and keeps its contents between events • “MetaDataSvc” holds metadata about the event being processed at the moment, and normally remains unchanged for many events 14 Whiteboard • In an algorithm’s code you use StoreGateSvc like: StatusCode MyAlgorithm::execute() { const CaloClusterContainer* clusters = 0; ATH_CHECK( evtStore()->retrieve( cluster, “Clusters” ) ); MyObject* obj = new MyObject( 3.141592 ); ATH_CHECK( evtStore()->record( obj, “MySuperObject” ) ); const DetectorInfo* info = 0; ATH_CHECK( detStore()->retrieve( info, “SomeDetectorInfo” ) ); return StatusCode::SUCCESS; } 15 Whiteboard • In an algorithm’s code you use StoreGateSvc like: Convenient accessors to the StatusCode MyAlgorithm::execute() { different StoreGateSvc instances const CaloClusterContainer* clusters = 0; ATH_CHECK( evtStore()->retrieve( cluster, “Clusters” ) ); MyObject* obj = new MyObject( 3.141592 ); ATH_CHECK( evtStore()->record( obj, “MySuperObject” ) ); const DetectorInfo* info = 0; ATH_CHECK( detStore()->retrieve( info, “SomeDetectorInfo” ) ); return StatusCode::SUCCESS; } 15 Whiteboard • In an algorithm’s code you use StoreGateSvc like: Convenient accessors to the StatusCode MyAlgorithm::execute() { different StoreGateSvc instances const CaloClusterContainer* clusters = 0; ATH_CHECK( evtStore()->retrieve( cluster, “Clusters” ) ); MyObject* obj = new MyObject( 3.141592 ); ATH_CHECK( evtStore()->record( obj, “MySuperObject” ) ); const DetectorInfo* info = 0; ATH_CHECK( detStore()->retrieve( info, “SomeDetectorInfo” ) ); return StatusCode::SUCCESS; } Helper macro to check the returned StatusCode objects 15 M Re:! Data Flow • t ! ! D:! • B 16 Karsten Köneke 19 Interface/Factory • In ATLAS we adopted a component-based software engineering model • See: http://en.wikipedia.org/wiki/Component- based_software_engineering • This allows us to reduce the dependency between different parts of the software • We define a generic interface class in a central place • All clients only make use of this generic interface, and ask the framework