8 Interfacing Gaudi with Python Gaudi Framework Tutorial, 2004 Objectives After completing this lesson, you should be able to: • Understand how Gaudi is interfaced to Python • Convince you that Python is a very good programming language for Rapid Prototyping • Write simple Python scripts to access LHCb data 8-2 Gaudi Framework Tutorial, 2004 Python Language Python is an Open Source programming language created by Guido van Rossum. While it has been available since 1990, it has recently seen a dramatic increase in popularity in a wide variety of domains Python is simple, elegant, powerful and reliable Features: • Variables & Arithmetic expressions • String manipulations • Conditionals (if/else statements), Loops (for/while) • Functions, Lists, Dictionaries, Classes (Objects), Exceptions , Modules 8-3 Gaudi Framework Tutorial, 2004 Why Python? Free, Open-Source, Maintainable Cross-Platform, Portable General-Purpose, High-level, Object-Oriented Interpreted • However it is quite fast (byte code idea from Java) Dynamically typed, Introspective • Not need to declare any variable Simple syntax, Robust • Emphasis by the author to minimize typing Variety of Shells Powerful built-in types and modules Î Ideal for Scripting and Prototyping 8-4 Gaudi Framework Tutorial, 2004 Python as a “Glue” Extension Modules • Very easy to interface to C++ classes (C-API) Collaborative • Extension modules can be plugged and used together Very rich set LHC modules specialized generic modules Several GUI toolkits EDG API PVSS XML Database GUIGUI Python shell JPE PyROOT gaudipython mathmath Gateways to other frameworks Java Root Gaudi Very rich set Classes Classes Framework of Python standard 8-5 Gaudi Framework Tutorial, 2004 modules Python Binding Techniques Python provides a very complete C-API • By hand bindings are possible but even for medium-sized projects, it becomes cumbersome and unmaintainable. Developing Python bindings • Several existing options: SWIG, Boost.Python, SIP PyLCGDict: Python binding to the LCG Dictionary • It allows the user to interact with any C++ class for which the LCG Dictionary has been generated • With this module, there is no need to generate specialized Python bindings or wrapper code to interact with C++ classes 8-6 Gaudi Framework Tutorial, 2004 PyLCGDict: Mode d’emploi From class definitions (.h files) a “dictionary” library is select.xml produced • Description of the class MyClass.h lcgdict • “stub” functions to class MyDict.so methods MyClass.so load Absolutely non-intrusive The PyLCGDict module does PyLCGDict the adaptation between Python objects and C++ objects in a generic way Python MyScript.py • It works for any interpreter dictionary 8-7 Gaudi Framework Tutorial, 2004 PyLCGDict: Supported Features Conversion of C++ and Python primitive types C++ classes • Mapped to Python classes and loaded on demand. Templated classes supported. C++ namespaces • Mapped to python scopes. The "::" separator is replaced by the python "." separator Class methods • Static and non static class methods are supported. Default arguments. • Method arguments are passed by value or by reference • The return values are converted into Python types and new Python classes are created if required. Dynamic type returned if possible. • Method overloading works by dispatching sequentially to the available methods with the same name until a match with the provided arguments is successful. 8-8 Gaudi Framework Tutorial, 2004 PyLCGDict: Supported Features Class data members • Public data members are accessible as Python properties Emulation of Python containers • Container C++ classes (std::vector, std::list, std::map like) are given the behavior of the Python collections to be use in iterations and slicing operations. Operator overloading • Standard C++ operators are mapped to the corresponding Python overloading operators 8-9 Gaudi Framework Tutorial, 2004 PyLCGDict: Example (1) #include <iostream> namespace Example { MyClass.h class MyClass { public: MyClass() : m_value(0) {} MyClass(const MyClass& m ) : m_value(m.m_value) {} ~MyClass() {} int doSomething(const std::string& something ) { std::cout << “I am doing something with “ << something << std::endl; return something.size(); } int value() { return m_value; } void setValue(int v) { m_value = v; } private: int m_value; public: float fprop; std::string sprop; }; } 8-10 Gaudi Framework Tutorial, 2004 PyLCGDict: Example (2) > python Python 2.2.2 (#1, Feb 8 2003, 12:11:31) [GCC 3.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import PyLCGDict >>> PyLCGDict.loadDict('MyClassDict') >>> m = Example.MyClass() >>> m.doSomething('care') I am doing something with care 4 >>> m.setValue(99) >>> print m.value() 99 >>> n = Example.MyClass(m) # copy constructor >>> print n.value() 99 >>> n.fprop = 1.2 >>> n.sprop = 'String property' >>> print n.sprop, n.fprop String property 1.2 8-11 Gaudi Framework Tutorial, 2004 Another Example: ROOT The ROOT.py loads the ROOTDict.so and initializes the ROOT system >>> from ROOT import gRandom, TCanvas, TH1F >>> c1 = TCanvas('c1','Example',200,10,700,500) >>> hpx = TH1F('hpx','px',100,-4,4) >>> for i in xrange(25000): ... px = gRandom.Gaus() ... hpx.Fill(px) ... >>> hpx.Draw() >>> c1.Update() 8-12 Gaudi Framework Tutorial, 2004 GaudiPython Enabling the interaction of Gaudi components from Python interpreted • Configuration, Interactivity, etc. Starting from Gaudi v14r1, GaudiPython has been re-implemented using PyLCGDict • Generated dictionaries for most common Gaudi “Interfaces” and “Base classes” (~80 classes) • Not need to generate dictionaries for all classes (in particular the implementations) 8-13 Gaudi Framework Tutorial, 2004 GaudiPython The end-user module “gaudimodule.py” hides some of the technicalities and adds some handy functionality • Very easy to extern/modify/adapt since is written in Python • Basically backward compatible with previous version 8-14 Gaudi Framework Tutorial, 2004 Using GaudiPython Add in the requirements file: use GaudiPython v* 8-15 Gaudi Framework Tutorial, 2004 Starting Methods Starting a Gaudi application from Python Shell >>> import gaudimodule >>> gaudi = gaudimodule.AppMgr() >>> gaudi.run(100) ... >>> gaudi.exit() Starting a Python interpreted from a Gaudi application Optional ApplicationMgr.DLLs += { "GaudiPython“ }; initial script ApplicationMgr.Runable = "PythonScriptingSvc"; PythonScriptingSvc.StartupScript = "../options/AnalysisTest.py"; 8-16 Gaudi Framework Tutorial, 2004 Configuration Instead of using “JobOptions” files we could use Python to completely configure Gaudi applications • ATLAS is going in this direction “Properties” are mapped to native Python types (int, float, str, list, etc) >>> gaudi = gaudimodule.AppMgr() >>> gaudi.EvtMax = 100 >>> gaudi.TopAlg = [‘Alg1’, ‘Alg2’] >>> alg1 = gaudi.algorithm(‘Alg1’) >>> alg1.EnergyCut = 10.7 >>> msgsvc = gaudi.service('MessageSvc') >>> msgsvc.OutputLevel = 3 ... 8-17 Gaudi Framework Tutorial, 2004 Interacting with the Event Since we have the complete set of Dictionaries we can interact with any Event class References to Event objects are obtained by asking the Event data service ( AppMgr.evtSvc() ) • There are currently some small limitations (i.e. handling directly KeyedContainer<T>, etc.) >>> evt = gaudi.evtSvc() >>> header = evt['Header'] >>> print ‘Run number = ‘, header.runNum() >>> mcparts = evt['MC/Particles'] >>> for p in mcparts.containedObjects() : ... print p.momentum().px() 8-18 Gaudi Framework Tutorial, 2004 Interacting with Histograms It is possible to book and fill histograms using the Histogram data service • Same interface as in C++ The complete AIDA Histogram interfaces are available to Python >>> import random >>> his = gaudi.histsvc() >>> h1 = his.book('h1',‘Histogram title', 40, 0, 1000) >>> for v in range(1000) : ... h.fill(random.uniform(0,1000)) >>> print h1.mean() 8-19 Gaudi Framework Tutorial, 2004 Complete Example: Read a DST file import gaudimodule gaudi = gaudimodule.AppMgr(outputlevel=5, joboptions='$EVENTSYSROOT/options/PoolDicts.opts') sel = gaudi.evtsel() sel.open(['PFN:rfio:/castor/cern.ch/lhcb/DC04/00000541_00000001_10.dst']) evt = gaudi.evtsvc() his = gaudi.histsvc() h1 = his.book('h1','# of MCParticles', 40, 0, 5000) class DumpAlg(gaudimodule.PyAlgorithm): def execute(self): evh = evt['Header'] mcps = evt['MC/Particles'] print 'event # = ',evh.evtNum() h1.fill(mcps.size()) gaudi.addAlgorithm(DumpAlg()) Ex/PythonExample gaudi.run(10) print h1, h1.contents() 8-20 Gaudi Framework Tutorial, 2004 Complete Example: Read a DST file EventSelector SUCCESS Reading Event record 0. Record number within stream 0: 0 event # = 1 event # = 2 event # = 3 event # = 4 event # = 5 event # = 6 event # = 7 event # = 8 event # = 9 event # = 10 Histogram 1D "# of MCParticles" 40 bins [0.000000,5000.000000] [5, 0, 0, 0, 1, 1, 0, 0, 1, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] 8-21 Gaudi Framework Tutorial, 2004 Building on GaudiPython It is fairly easy to extern the functionality provided by GaudiPython There are already several “products” which are build on top of GaudiPython Bender • Physics Analysis on Python Panoramix • Detector and Event Visualization 8-22 Gaudi Framework Tutorial, 2004 Summary This is an attempt to convince you that Python is a very good programming language for Rapid Prototyping • Efficient way to test your Physics Analysis ideas Interfacing C++ classes to Python is fairly easy • Several well established technologies exists (Boost.Python, etc.) • PyLCGDict is another way of doing it • PyLCGDict exploits the LCG dictionaries that are already available due to object persistency (POOL) GaudiPython • Is the package that enables the interaction of Gaudi from Python • Latest version of GaudiPython is based on PyLCGDict Examples of the available functionality has been shown 8-23 Gaudi Framework Tutorial, 2004.