arXiv:1010.4891v1 [cs.SE] 23 Oct 2010 hti Mayavi? is What Introduction nqeitgainwt cetfi workflow scientific a with integration unique A FRANCE nvri´,4reJ oo,983OsyFRANCE Orsay 91893 Monod, J. rue Universit´e, 4 ubi NI 0 076 400 - INDIA Mumbai, et ftesinicwrflw ihinteractive as- rich various A in workflow. scientific fit the to of pects application, dialogs, an widgets, and library, the plugins, to pro- it addition since in library vides, visualization is a project just the than important, more visualiza- most complex and Finally, handle also tions. to and users expectation, expert user’s allows naive fit a that with objects closely and tools problems. similar re- exposes Mayavi solve different Second, to since need often important fields is search This ab- all, general-purpose, stractions. reusable, cases of build to domain-specific First strives avoids and project factors. Mayavi tools of the visualization combination other a most through unlike is Mayavi few a discuss applications. in finally novel made and decisions Mayavi, engineering insight implementing and provide design then the var- we on the Mayavi, of of overview features an present ious we article, this general- In a with framework. ap- work application-building domain-specific purpose that plugins a and in these plication, assembling with for associated feature-rich objects a widgets and hierarchy; scripting object easy on inter- focused MATLAB-like a face both with library application; Python several interactive a work- provides full-blown Mayavi a user’s purpose, points: scientific this entry For visual- the data with flow. for fit tools that pro- interactive ization to and seeks easy It 3D vide package. general-purpose, visualization open-source, scientific an is Mayavi aai akg o Dvsaiaino scientific of visualization 3D for package a Mayavi: ∗ ‡ † erSi/E aly ˆt15 19 Gif-sur-Yvette Bˆat 91191 Saclay, 145, NeuroSpin/CEA aitlta,IRASca l eFac,Pr Orsay Parc France, de Ile Saclay INRIA team, Parietal eateto eopc niern,ITBombay, IIT Engineering, Aerospace of Department rbuRamachandran Prabhu data ∗ 1 sacmoett raecso ilg htem- that dialogs custom used create be to can component object a each as which reflex- in a model on object in- ive relies Mayavi for Second, convenient well work. highly integrates teractive it Mayavi makes that IPython, fact with the with along lz Ddt ecie by described visu- data to 3D functions alize convenient provides interface yhnpoet.A ealdblw h entire and be- arrays the conversion VTK implicit tween below, with wrapped detailed is API As VTK scientific main the projects. in Python used structure data core the rpia sritrae scmoet First on . naturally components operates Mayavi all, of as its all interfaces exposes user and graphical structures data familiar uses it computation: a typical numerical for a by Python of using tools workflow scientist the these with integration from support tighter or differs datasets. 4D Mayavi tools as [6] well However, these as ParaView Mayavi, visualization data Unlike parallel as [7]. such VisIt interfaces, high-quality expose 3D Python that general-purpose programs excellent visualization several are There tightly- this environment. to integrated visualization power- data scientific brings 3D data- Mayavi ful and scientific for development. ideal simultaneous processing is with that 5] use [4, interactive plotting it to Indeed, sig- itself garnered lends scientists. recently among has mind-share Python li- nificant scientific 3], yet high-quality [2, powerful, of braries number a growing is a Thanks to it [1]. as language multi- programming cases, easy-to-learn, the answering use of to scientific use central ple The is language interface. user Python a without be driven can and visualizations processing Mayavi batch use, enable non-interactive to the addition, suit In tweak to visually, data. visualization data the build the and examine parameters can user interactively, the done as best is datasets complex visualiza- of because tion important full- is but scripting simple featured with along application, graphical Ga¨el Varoquaux numpy †‡ numpy rasadthe and arrays ras This, arrays. numpy arrays, mlab bed a Mayavi visualization. (ETS [8]) enables chy that maps with some simple rules to VTK users to create rich, interactive scientific applica- and forms the foundation of Mayavi. tions that support 2D and 3D plotting requiring TraitsUI/Envisage : The user interface of the only the knowledge of Python and object-oriented end-user application relies entirely on Traits programming. Thus Mayavi fills a valuable need and optionally on Envisage. Envisage is in the scientific computing ecosystem. an application-building framework, `ala the Eclipse framework , which is used for the ap- Powerful underlying technologies plication. At its core, Envisage is a system Mayavi is part of the ETS [8] and builds upon a for defining, registering, and using plugins. powerful stack of existing libraries. In this section Both Traits and Envisage can use wxPython we provide a very brief overview of these. or PyQt as a backend. Thus Mayavi can be used with both toolkits, although currently the VTK : The Visualization ToolKit [9] is one of wxPython backend is more developed and thus the best, actively-developed, general-purpose, more mature. open-source, visualization and graphics li- braries available. Packaging and licensing Numpy : The workhorse of scientific comput- A simple pipeline model for visualization ing with Python is the numpy array structure. VTK relies on an elaborate pipeline model that is This multidimensional numerical array trans- assembled to create a visualization. In an effort to forms Python into a high-level array language, to the user, Mayavi exposes a basic pipeline inter- similar to MATLAB. It is used as a common face: data is obtained from a data source, and the data container in most scientific libraries and user can add visualization modules to display the projects relying on Python. data in various ways, or filters to modify the data Traits : The Traits library before eventual visualization. Mayavi connects (http://code.enthought.com/projects/traits), the various components of the pipeline and, to an is the cornerstone of the ETS. Traits extend extent, also takes care of which components can Python object attributes and provide an be inter-connected. For example, a cloud of scat- elegant mechanism for attribute initialization, tered points cannot be visualized with a grid plane validation, delegation, notification (efficient since there is no fixed grid for these points. The callbacks on attribute modification), and visu- different sources, modules and filters are traited alization (through dialogs using wxPython or objects that all have associated dialogs and meth- Qt4). Henceforth, we refer to an object with ods that expose the underlying TVTK objects traits as a “traited object”. as much as possible without requiring a detailed knowledge of VTK. The Mayavi pipeline collapses TVTK : The Traited VTK library (TVTK) is an together several elements of a VTK pipeline, for automatically generated wrapper of VTK com- instance the Mayavi modules are made of VTK bined with Traits This is done by wrapping mappers, actors, and eventually widgets or even VTK-Python objects and providing a traits- filters when their use helps representing the data. enabled API with a more “Pythonic” feel . Finally, although loops in the pipeline are possi- All C++-style getters and setters are replaced ble, Mayavi the pipeline as a tree rather than by traits. VTK has its own C++-based ar- a graph (see Figure 1), to facilitate its manipula- ray structures, and the VTK-Python bindings tion, both through the interactive user interface, require tedious manipulations to copy or refer- and programmatically. ence the data from numpy arrays to VTK ar- rays. TVTK provides a seamless API where VTK arrays are converted dynamically from History of the project and to numpy arrays, using a view of the same Mayavi was created in 2001 [10] by P. Ramachan- memory where possible. In addition, because dran as an end-user application for scientific visu- of the use of Traits, all TVTK objects pro- alization. The name is a Sanskrit word meaning vide a default dialog to edit their properties. “magician”. The application was appreciated for TVTK is thus a very powerful object hierar- its ease of use and interactivity. However, it was

2 not easy to script from Python. During 2004– 2005, Enthought Inc. hired P. Ramachandran to write TVTK [11] and start work on a new ver- sion of Mayavi. The new version, “Mayavi2” [12], uses the tools developed at Enthought to focus on reuse and embedding. In 2007, G. Varoquaux joined the project. The project is rapidly gaining features and documentation while the usability is improved based on user feedback. In early 2008, TVTK and Mayavi were each awarded FOSS In- dia awards.

Using Mayavi Mayavi can be used in different ways, and as such has several entry points.

The mayavi2 application The interactive application, mayavi2, is an end- user tool that can be used without any program- ming knowledge. It provides an interface with Figure1: A view of the mayavi2 application showing menus and several panels to guide the user while the Mayavi logo: a Boy mathematical surface. On creating a visualization (see Figure 1). As de- the bottom right, an interactive IPython prompt, for scribed earlier in the overview, Mayavi presents prototyping scripts or exploring objects, can be seen. a simplified pipeline view of the visualization. On the upper left stands the tree representing the vi- Through the menus, data can be loaded from files, sualization pipeline. It features a source, here a grid of points created by numpy arrays, a PolyDataNormal or created with predefined objects such as a lay- filter used to calculate normals of the surface for a out of the Earth’s continents, parametric surfaces smooth rendering, and the Surface module, with defining for example a Klein bottle, etc. Subse- its corresponding module-manager node, used to quently, optional filtering may be performed on change the colors and legend of the surface. The the data, and visualization modules are added to dialog below the tree can be used to modify the se- create the visualization. lected node, i.e. colors and legend properties in the The various pipeline components (sources, mod- present case. ules and filters) appear on a tree view (see the left panel in Figure 1), and more can be added ilarly, the Mayavi modules are a single point of through menus and dialogs. In particular, contex- entry to changing all the properties of an object tual menus suggest to the user the filters or mod- displayed on the visualization, and gather all the ules that are applicable to a given data source. VTK sub-objects in one object and one dialog. The pipeline may also be reorganized using drag- An exception to this rule is that the color maps and-drop operations on the tree nodes. Objects and the legends can be shared between modules, selected on the pipeline view can be edited in an- and thus can be represented as a separate node in other panel (left side of the bottom panel in Fig- the pipeline. ure 1) and modifications are immediately applied Mayavi offers through its primary tree-based in- to the visualization. It is to be noted that al- terface only a limited subset of VTK’s filters, and though the pipeline shown in the Figure 1 is ex- the Mayavi sources cannot cover all possible ways tremely simple, more complex pipelines can also to create VTK datasets. This is why Mayavi of- be setup. fers a UserDefined filter to plug in almost any While a raw VTK dataset is a versatile data struc- VTK filter in the Mayavi pipeline by specifying ture describing data embedded in a 3D space, a its name, and a VTKDataSource class to create a Mayavi source tries to expose to the user a sim- Mayavi source from any VTK dataset. ple interface to importing data in Mayavi. Sim- The entire visualization can be saved to disk in

The mayavi2 application 3 a Mayavi-specific format although, this is one as- Mayavi’s mlab scripting interface is a set of pect of Mayavi that needs to be more robust. Al- Python functions that work with numpy arrays ternatively, the data generated at any point of the and draw some inspiration from the MATLAB pipeline can be saved to VTK files. . and matplotlib plotting functions. It can be used The mayavi2 application has a couple of features interactively in IPython, or inside any Python to help create Python scripts from a visualiza- script or application. The following example gen- tion. First, the application displays an interac- erates iso-contours of a mathematical function, tive Python shell, where Python commands can sampled on a regular grid. The resulting visu- be entered for immediate execution. The scripting alization can be seen in Figure 2. API described later in this article can be used to from enthought.mayavi import mlab create or modify visualizations. Objects dragged from numpy import ogrid from the pipeline tree to the shell appear in it x, y, z = ogrid[-10:10:100j, -10:10:100j, -10:10:100j] as Python objects for exploration or modifica- ctr = mlab.contour3d(0.5*x**2 + y**2 + 2*z**2) tion. Second, the pipeline tree view features a mlab.show() record button (the red button on the toolbar in the left panel of Figure 1). When the record mode Simple plotting commands operating on numpy is turned on, any modification to objects in the arrays, such as mlab.contour3d, used in the pipeline, or any object added to the pipeline auto- previous example, build a complete visualiza- matically generates the necessary lines of Python tion pipeline. These simple commands hide the code to reproduce the action. Fully-functional pipeline model from the user, for simple use cases. Python scripts are thus created when the record They accept a large number of extra arguments mode is turned on before adding any object to the to control the properties of the visualization cre- pipeline, although the generated code is not al- ated. In addition, as they return the Mayavi mod- ways the simplest possible code. In addition, the ules created, more properties can be changed by record mode is also an extremely valuable learning modifying their attributes. tool for scripting Mayavi or TVTK objects. In- Alternatively, mlab offers a direct control of the deed, VTK is a very rich visualization library and Mayavi pipeline through separate creation of Mayavi’s object hierarchy can be deep and com- sources, filters and modules. Thus the call to plex. Thus, it can be hard to find the method or mlab.contour3d in the previous example can be attribute corresponding to a given feature. Con- replaced by two commands, one to create a source sequently, the record feature is highly convenient object from the regularly-spaced volumetric data even for experienced VTK users. in a numpy array, and a second to apply an iso- An extensive user manual is shipped with surface module on it: src = mlab.pipeline.scalar_field(0.5*x**2 + y**2 + 2*z**2) Mayavi [13] and is accessible from the ap- ctr = mlab.pipeline.iso_surface(src) plication. The user guide is rendered using Sphinx (http://sphinx.pocoo.org/) and em- Manually populating the pipeline requires an un- beds a search bar and an index. derstanding of the pipeline model. It is also more powerful as it gives access to a wider range of Simple Python scripting possibilities through custom-made pipelines. The As we have seen, Mayavi can be used in a fully- names of the mlab.pipeline functions to create interactive manner, by a non-programmer, using objects are lower-case-with-underscores versions the mayavi2 application. However, Mayavi can of the camel-case names of the classes represented, also be used through a simple and yet power- as they appear by default in the pipeline view. ful scripting API, providing a workflow similar Thus, going from a pipeline built interactively to to that of MATLAB or Mathematica. Many sci- a script is very easy. entists use Python for their computational work. Although the window used to display visualiza- Visualization is a very important component of tion is very simple, the full power of Mayavi is still such work and is most effective when used inter- available. Clicking the button with the Mayavi actively, as an exploratory and debugging tool. A logo on the left of the visualization-window tool- good example of this is the typical use of mat- bar displays a dialog containing the same pipeline plotlib [5] and IPython [4]. tree view as in the Mayavi application (see Fig-

4 function embedded in a 3-Dimensional space as a curved line. The mathematical curve, defined by the curve function, takes one parameter – the number of minor rotations in the transverse di- rection. The dialog (Figure 3) enables the modi- fication of this parameter with an immediate vi- sualization of the results: from numpy import linspace, pi, cos, sin from enthought.traits.api import HasTraits, Range, Instance, \ on_trait_change from enthought.traits.ui.api import View, Item, HGroup from enthought.mayavi.core.ui.api import SceneEditor, \ MlabSceneModel

Figure 2: Working with Mayavi in IPython. The def curve(n_turns): terminal on the background runs the IPython ses- phi = linspace(0, 2*pi, 2000) sion from which the visualization window on the return [ cos(phi) * (1 + 0.5*cos(n_turns*phi)), foreground (right) was created. The pipeline dialog sin(phi) * (1 + 0.5*cos(n_turns*phi)), 0.5*sin(n_turns*phi)] editing the different visualization objects was cre- ated by clicking on the button with the Mayavi icon class Visualization(HasTraits): on the left of the visualization window’s toolbar. n_turns = Range(0, 30, 11) scene = Instance(MlabSceneModel, ())

def __init__(self): ure 1). All the interactive functionality of the HasTraits.__init__(self) mayavi2 application is accessible in this dialog: x, y, z = curve(self.n_turns) the buttons on the toolbar provide access to help, self.plot = self.scene.mlab.plot3d(x, y, z) script recording, or object-creation. Clicking on @on_trait_change(’n_turns’) the pipeline nodes creates dialogs that allow mod- def update_plot(self): x, y, z = curve(self.n_turns) ification of the object properties. The pipeline can self.plot.mlab_source.set(x=x, y=y, z=z) be populated and modified by the context menus view = View(Item(’scene’, height=300, show_label=False, accessible with a right-click on the nodes. editor=SceneEditor()), HGroup(’n_turns’), resizable=True)

Animating data and building interactive dialogs Visualization().configure_traits() Objects created by the mlab functions expose an In the above code example, the Visualization mlab source attribute, which gives access to the class defines a few traits including the scene trait numpy arrays used to create the dataset. Assign- which is an instance of MlabSceneModel. The ing new arrays to the mlab source triggers an configure traits() call at the end of the code update of the visualization. The name of the at- creates a dialog representing the object, the lay- tribute one needs to modify on the mlab source out of which is given by the view defined at the object is the name given to the corresponding end of the class. This view exposes the scene argument in the function signature documenta- trait using a SceneEditor in the dialog, and the tion, for instance the mlab.contour3d function n turns attribute as a slider. On creation of signature is contour3d(scalars, ...), where the Visualization object, the curve is plotted scalars is a 3D array. Thus, we can animate the in the embedded scene with the plot3d mlab contour object created previously by modifying in call. When the n turns attribute is modified, the place the scalars it represents: update plot method is called, curve data is re- from time import sleep for i in range(1, 10): computed, and the plot object is modified using sleep(1) the mlab source attribute. ctr.mlab_source.scalars = 0.5*x**2 + y**2 + i*z**2 In general, the different properties of the objects In-place modifications are also useful when em- used in visualizations, such as sources, filters, bedding a visualization in an interactive applica- modules, or even the scenes, can be modified in a tion. A Mayavi scene can be displayed as part of script, with instantaneous or delayed update of a traits-based user interface. The following exam- the scene, by simply setting the corresponding ple displays a dialog visualizing a 1D parametric trait. Moreover, the different dialogs that form

Animating data and building interactive dialogs 5 import wx

class MainWindow(wx.Frame): def __init__(self, parent, id): wx.Frame.__init__(self, parent, id, ’Mayavi in Wx’) self.visualization = Visualization() self.control = self.visualization.edit_traits( parent=self, kind=’subpanel’).control self.Show()

app = wx.PySimpleApp() frame = MainWindow(None, wx.ID_ANY) app.MainLoop()

In the above example, the edit traits method is passed the wxPython frame into which the dialog is embedded. The control trait of the object pro- duced by the edit traits call is the wxPython object containing the widget. Similarly, dialogs can be embedded in a PyQt application, as de- tailed in the user guide. Any Mayavi dialog can be embedded similarly Figure 3: The dialog created by the code example. In addition to the interaction with the visualization, in more complex applications. For instance the dragging the slider bar modifies the visualization in- various Mayavi pipeline objects also provide an teractively. edit traits method to edit their properties. Thus, the work invested in developing powerful widgets for the Mayavi application, such as the Mayavi, such as the tree view, or the dialog edit- pipeline tree view, is readily available to the ap- ing an object, can be embedded in a user’s appli- plication builder. cation. Callbacks between these different dialogs and the scene are already wired. As such, Mayavi forms more than a visualization library; it can be Extending the mayavi2 application used as a set of interactive components to provide Instead of creating a new application, one can live visualization to a domain-specific application, extend the already-powerful mayavi2 application requiring little knowledge of GUI programming or by adding to it custom functionality or domain- event-loops. specific elements. The application is built by inte- grating the 3D visualization provided by Mayavi with other functionality, such as a Python shell, Embedding in existing applications via Envisage plugins. Using the same mechanism, Although Traits is a very powerful tool for devel- one may put together other applications or extend oping interactive applications, most existing ap- the mayavi2 application. A discussion of the En- plications are developed using a raw GUI toolkit. visage application-building framework is beyond It is thus important to integrate the dialogs pro- the scope of this article. duced from the code in the previous paragraphs in In addition, Mayavi has a mechanism to regis- a non Traits-aware GUI. Traits has a wxPython ter new types of data sources, filters or mod- and a Qt4 backend. While the configure traits ules. These are automatically added to the var- method used in the above example to create the ious menus and the mlab.pipeline interface. A dialog creates a full wxPython application and common use case is to add domain-specific file starts the main event loop, the HasTraits class readers or visualization modules. For example, also provides an edit traits method that only the following code can be placed in a module im- creates and returns a panel or dialog. Below is an ported in ~/.mayavi2/user mayavi.py to define example showing how the Visualization class a reader using numpy to load arrays stored in a defined earlier, and the corresponding dialog, can text file: be embedded in a wxPython application:

6 scripting API controls the engine to create visu- alizations. As all objects rely on the Traits li- brary, UI panels or widgets can be created using the TraitsUI package. The different panels, as well as the core functionality of Mayavi, can be combined with other Envisage plugins to create applications such as the mayavi2 application.

The Engine as a pipeline warden The Mayavi engine maintains a tree structure of pipeline objects. Each pipeline object main- Figure 4: Architecture diagram of Mayavi. The tains references to its parents and children, and green nodes are external Python modules, the blue exposes a list of inputs and outputs. The out- nodes are the VTK-based object hierarchy, and the put list contains the TVTK datasets flowing be- yellow ones form the Mayavi functionality. tween the pipeline elements. The input objects are Mayavi pipeline objects. The Mayavi pipeline from enthought.mayavi.core.api import registry, \ objects have callbacks to rewire the underly- SourceMetadata, PipelineInfo from enthought.mayavi.sources.api import ArraySource ing VTK pipeline if their inputs change. They import numpy as np also feature two events, pipeline changed and data changed, which are propagated down the def array_reader(fname, engine): return ArraySource(scalar_data=np.loadtxt(fname)) pipeline to update it. The engine manages the life-cycle of the pipeline objects, i.e. it manages registry.sources.append(SourceMetadata( factory = __name__ + ’.array_reader’, their addition and removal. menu_name = "Array text files", The pipeline of the iso-surface example intro- extensions = [’txt’], wildcards = ’TXT files (*.txt)|*.txt’, duced with the mlab API in the beginning of the output_info = PipelineInfo(datasets=[’image_data’], article can be explicitly built with the following attribute_types=[’any’], code: attributes=[’any’]), )) from enthought.mayavi.core.api import Engine from enthought.mayavi.sources.api import ArraySource from enthought.mayavi.modules.api import IsoSurface Of course, proper use of Mayavi as a platform for from enthought.pyface.api import GUI domain-specific applications requires a good un- from numpy import ogrid derstanding of the finer details of Mayavi, which x, y, z = ogrid[-10:10:100j, -10:10:100j, -10:10:100j] is beyond the scope of this article, but is detailed engine = Engine() in the user manual [13]. engine.start() engine.new_scene()

src = ArraySource(scalar_data=(0.5*x**2 + y**2 + 2*z**2)) Mayavi architecture and software design engine.add_source(src) In this section we provide a broad overview of the engine.add_module(IsoSurface()) GUI().start_event_loop() architecture and software design of Mayavi. In the above, the engine manages the connec- Architecture overview tion between the source and modules internally: The general software architecture of Mayavi is it maintains the context of the visualization. For summarized in the diagram on Figure 4. The vi- instance the new scene method can be overridden sualization layer of Mayavi relies on TVTK ob- in a subclass to create an embedded scene, as done jects. The Mayavi pipeline objects use TVTK in the mayavi2 application, or a separate window, objects and have methods that help wire them as when using mlab in IPython. Other subclasses together and simplify building the VTK pipeline. for off-screen rendering are also available. In ad- A central object, the Mayavi Engine, manages all dition, the engine provides context-dependent ac- the pipeline objects making up the visualization. tions which can be useful to drive an interactive The set of pipeline objects and the engine form application in a manner similar to spreadsheet the core of the Mayavi functionality. The mlab scripting. In the above, the add module call

The Engine as a pipeline warden 7 implicitly adds the IsoSurface instance to the of the Model View Controller (MVC) design pat- ArraySource source src. It is also possible to tern [14] is not complete and there is some mixing build the above pipeline explicitly by selecting of the view and model. However, while this can and connecting the different objects rather than be reduced in the future, we believe that the de- delegating the task to the engine: sign is already very reusable, offering us most of scene = engine.current_scene the advantages of MVC. src = ArraySource(scalar_data=(0.5*x**2 + y**2 + 2*z**2)) The “message-passing” style that replaces scene.add_child(src) src.add_child(IsoSurface()) method calls by trait assignments is thread- friendly and can be used to avoid dealing with We note that the Engine is not global. While the GUI event loops. The code updating the UI mayavi2 application and mlab provide default en- for instance is not exposed to the user. As we gines which suffice for most of the use cases, a mostly rely on Traits for views, these are defined user can create many different engines for differ- in a declarative way (as in the interactive dialog ent needs. As a result, the scope, or context, of example above). Consequently, no application the different visualizations and actions can be lim- logic can be found in the view-related code. ited and controlled. This isolation is important This is very important for a clean scripting API when one wishes to avoid side-effects in a large and also enables scripts to run in a headless application. It also makes Mayavi much easier to (off-screen) mode with no modifications . reuse and test. For example, if the Engine was global then any changes to it in a test suite would Testing and scriptable APIs influence other tests. However, since one may cre- ate as many engines as one desires it is easy to As seen above, Mayavi strives to be highly write tests that avoid unnecessary side-effects. reusable in a variety of contexts for the user and developer. One of the development strategies that really helped make the API reusable and the code A central registry to avoid duplication reliable is our focus on being able to script the Mayavi provides a large, and growing, list of API as much as possible. This was achieved in pipeline objects. They are exposed through many part by resorting to unit testing, integration test- different interfaces, both graphical, and program- ing, and some Test-Driven Development (TDD) ming APIs. In order to avoid code duplication, [15]. Testing and good example scripts forced us all the different sources, filters, and modules are to make the API highly reusable and resulted in specified in a central registry along with informa- increased reliability and clean code. This experi- tion describing their functionality. The metadata ence corroborates with the advantages claimed by information in the registry is used to generate the practitioners of TDD. different menus of the user interfaces, as well as many of the mlab.pipeline functions, thus en- Summary of key design choices forcing consistency throughout Mayavi. A sam- ple of this information can be seen in the example We believe some architectural decisions are key to provided in sectionwhere Mayavi is extended with the success of Mayavi’s reusability: a new source by adding it to the registry. Multiple abstraction layers, as summarized by Figure 4. This layered functionality enforces separation Model-view separation of concerns and enables sharing code between As much as possible, Mayavi uses a reactive pro- different entry points or APIs to address gramming style employing callbacks on trait mod- various use cases. ifications. UIs are created using representations A central, well-defined object: the Engine. of the objects’ traits. This programming style al- It coordinates the visualization, provides a lows for a very clean separation of the model from context, and is thus important in establishing the view. The model can be fully described by the a consistent view of the application, both in traits of all the objects on the pipeline. For exam- the Mayavi code, and in the scripting APIs. ple, the script-recording functionality described In addition, this central object helps avoiding earlier is implemented in large part by tracking globals. We note that any number of Engines the modifications to these traits. Mayavi’s use

8 can be created and used simultaneously, thus the Engine does not preclude the possibility of data-parallel execution. The use of Traits. Traits’ powerful object model leads to a good design through reactive pro- gramming and strong model-view separation, in addition to providing multiple-backend UIs with little effort. Model/view separation and loose coupling. It is well known that GUI-related code should be Figure 5: Four Mayavi applications controlled via well separated from the core application logic. the network from a central program providing a syn- However, in addition to this, we find that all chronized view. helper-code unrelated to the core functionality which caters to common end-user needs, such as provided by the mlab API, should also be many mundane tasks. The automatic script- separate from the core. ing mode of Mayavi is used to record UI ac- tions and generates human readable Python code. Automatically-created objects and functions. This code is then hand-edited to produce vi- The TVTK wrapper code is entirely automat- sualizations of the weather data. Thus, with- ically generated. Large parts of the UI-related out a direct knowledge of Mayavi’s internals or boilerplate code, and some of the APIs, are even a good knowledge of Python, the scien- auto-generated. This reduces duplication and tists at NAL are able to generate fully-working thus makes the interfaces more consistent, and Python scripts and tailor them to their needs. the code easier to maintain. For the interactive display of the different atmo- Focus on the API. Striving for a simple API spheric fields, each field is displayed on a separate greatly improves the developers mental rep- computer running a Mayavi script. The cam- resentation of the library and application’s era position of each Mayavi application is con- model, and as a result its architecture. The trolled by an in-house OpenGL application used API should answer common use-cases and be for display of cloud data obtained from satel- consistent across the various needs. Interac- lite images. In order to do this, we wrote a tion and feedback from users, whose needs simple TCP and UDP server program that lets sometimes differ from that of the developers, a user send Python statements across the net- has proven priceless. work which are interpreted by the running Mayavi Testing and scripting. Unit tests, TDD and ex- application. We used the excellent Twisted ample scripts for users are invaluable in creat- (http://twistedmatrix.com/trac/) library for ing a truly reusable tool. These practices are this. The size of the resulting server module was also a great way to notice unwanted tight cou- about 90 lines of code (without the documenta- pling in the object model. tion/comments). Using the server required two additional lines of code in the existing scripts at Some real-word applications NAL. The hardware setup at NAL is shown in Figure 5. The left-most screen shows the appli- Weather visualization using Mayavi cation controlling the view of the 4 other Mayavi The FloSolver division at NAL (National applications via the network. The right side is Aerospace Laboratories, Bangalore) uses Mayavi a visualization wall consisting of 4 separate LCD to visualize data produced by their weather mod- panels put together as one. This would not be eling code which is used to primarily forecast the possible but for the powerful libraries available monsoon in India. Mayavi is used both as a with Python. display device and more importantly to help re- fine the weather models. Data generated from weather simulations is rendered interactively us- ing Mayavi scripted from Python to automate

Weather visualization using Mayavi 9 Web-browser based usage of Mayavi using Python as a natural, easy to use, 3D visu- In a recent development, O. Certik and P. Ra- alization environment. machandran worked together to setup a Sage (http://www.sagemath.org) notebook working References with Mayavi in order to perform visualizations [1] Computing in Science and Engineering special is- on the web. Sage provides, among many other sue on scientific computing with Python, vol. 09, things, a powerful environment to do mathemat- no. 3, May/June, 2007. ics on the web through a notebook interface. This [2] T. E. Oliphant, “A Guide to NumPy”, Trelgol interface essentially provides a powerful Python- publishing, http://numpy.scipy.org, 2006. capable web page where users can interact with a Python interpreter, and embed the results along [3] E. Jones, T. E. Oliphant, P. Peterson, “SciPy: with images and text seamlessly from a browser Open source scientific tools for Python”, window. http://www.scipy.org, 2001. In order to have Mayavi working in the Sage [4] F. Perez, B. E. Granger, “IPython: A System for notebook, we built VTK with support for us- Interactive Scientific Computing”, Computing in ing Mesa’s (http://www.mesa3d.org) OSMesa li- Science & Engineering, 2007. brary for pure off-screen rendering. Then by [5] J. D. Hunter, “Matplotlib: A 2D Graphics Envi- using the existing support in Mayavi for off- ronment”, Computing in Science & Engineering, screen rendering we are able to render im- 2007. ages and generate X3D files, displayed interac- tively by the Sage notebook. The web page [6] Andy Cedilnik, Berk Geveci, Kenneth Moreland, http://lab.femhub.org/home/pub/39 demon- James Ahrens, and Jean Favre, “Remote Large Data Visualization in the ParaView Framework”, strates the resulting Sage notebook. The page In Eurographics Parallel Graphics and Visualiza- uses Mayavi’s mlab interface to generate a visu- tion pp 163–170, May 2006. alization and produce an X3D file that may be visualized interactively using a browser plugin. [7] Hank Childs, Eric Brugger, Kathleen Bonnell, Jeremy Meredith, Mark Miller, Brad Whitlock, Nelson Max, “A Contract Based System For Conclusions Large Data Visualization”, 16th IEEE Visualiza- There is a growing trend of moving more and more tion 2005, pp. 25, 2005. computational code to high-level environments. Python is an increasingly popular high-level lan- [8] Enthought, Inc, Austin, “Enthought Tool Suite”, http://www.enthought.com/products/ets.php guage for scientific computing because it has the potential to unite a variety of modules into a ho- [9] W. , K. Martin, W. Lorensen, “The Visualization mogeneous environment. Toolkit”, Kitware, 4th edition, 2006. Mayavi provides a rich and powerful 3D data visu- [10] P. Ramachandran, “MayaVi: A free tool for CFD alization package that tightly integrates the vari- data visualization.” In 4th Annual CFD Sympo- ous aspects of scientific computing and scientific- sium, Bangalore, India. Aeronautical Society of In- application development in Python. It strongly dia, August 2001. focuses on being reusable. In particular, it is well- suited to applied science and engineering prob- [11] P. Ramachandran, “TVTK A Pythonic VTK”, EuroPython Conference Proceedings, Goteborg, lems for which building custom visualization tools Sweden June 2005. is an important challenge. It tries to match the different scientific computing work flows: interac- [12] P. Ramachandran. “MayaVi2: The next genera- tively with an end-user application, in scripts, or tion”, EuroPython Conference Proceedings, Gote- in custom applications and also for pure off-screen borg, Sweden June 2005. rendering. The visualization model is consistent [13] P. Ramachandran, G. Varo- throughout the various entry points and the in- quaux, “Mayavi Users Guide”, teractive application can be used to prototype http://code.enthought.com/projects/mayavi, visualizations that can be easily converted into 2008. code embedded in scripts or applications. Mayavi dovetails nicely into a rich set of scientific tools

10 [14] E. Gamma et al, “Design Patterns - Elements of Reusable Object-Oriented Software”, Addison Wesley, 1994.

[15] K. Beck, “Test-Driven Development by Exam- ple”, Addison Wesley, 2003.

REFERENCES 11