D5.3 Report on the Implementation of the Three Vir- Tual Laboratories

Total Page:16

File Type:pdf, Size:1020Kb

D5.3 Report on the Implementation of the Three Vir- Tual Laboratories PaN-data ODI Deliverable D5.3 Report on the implementation of the three vir- tual laboratories Grant Agreement Number RI-283556 Project Title PaN-data Open Data Infrastructure Title of Deliverable Report on the implementation of the three virtual laboratories Deliverable Number D5.3 Lead Beneficiary STFC Deliverable Public Dissemination Level Deliverable Nature Report Contractual Delivery Date 31 Aug. 2014 Actual Delivery Date XX Oct. 2014 The PaN-data ODI project is partly funded by the European Commission under the 7th Framework Programme, Information Society Technologies, Research Infrastructures. PaN-data ODI Deliverable: D5.3 Abstract This report demonstrates and evaluates the services being developed within the PaN-data ODI project. A survey of the implementation status of these services at all participating institutes is in- cluded. The metadata catalogue plays a central role in the PaN-data project. Therefore it is dis- cussed how scientists benefit from this service. The status of the tomography database is re- viewed. Details of the NeXus deployment at various partners are presented. Eventually, three data processing frameworks, which are developed within the PaN-data community, are compared by implementing two scientific use cases. Keyword list PaN-data ODI, virtual laboratories, metadata, data catalogue, standard data format Document approval Approved for submission to EC by all partners on XX.10.2014 Revision history Issue Author(s) Date Description 1.0 Thorsten Kracht Aug 2014 First version 1.1 Jan Kotanski Aug 2014 NeXus deployment at DESY 1.2 Gero Flucke Aug 2014 Comparison of data processing frameworks 1.3 Eugen Wintersberger Aug 2014 Status of the NeXus library 1.4 Mark Koennecke Jul 2014 NeXus deployment at PSI-SINQ 1.5 Jens-Uwe Hoffmann Aug 2014 NeXus deployment at HZB 1.6 Guifre Cuni Aug 2014 NeXus deployment at ALBA 1.7 Frank Schluenzen Aug 2014 Science3D section 1.8 Tobias Richter Sep 2014 NeXus deployment at DLS 1.9 Jean-Francois Perrin Sep 2014 NeXus deployment at ILL 1.10 Tom Griffin Sep 2014 NeXus deployment at ISIS Page 2 of 107 PaN-data ODI Deliverable: D5.3 Table of contents Page 1 INTRODUCTION ..................................................................................................................... 6 1.1 THE AMENDMENT OF THE DESCRIPTION OF WORK .................................................................... 6 1.2 OVERVIEW OF THE DELIVERABLE ............................................................................................ 7 2 SURVEY OF THE IMPLEMENTATION OF THE PAN-DATA ODI SERVICES ....................... 9 2.1 SCIENTIFIC DATA POLICY ....................................................................................................... 9 2.2 COMMON IDENTITY MANAGEMENT SYSTEM UMBRELLA ........................................................... 10 2.3 COMMON DATA FORMAT NEXUS/HDF5 ................................................................................. 10 2.4 METADATA CATALOGUE ICAT-4 ........................................................................................... 12 3 DEMONSTRATION AND EVALUATION OF THE PAN-DATA SERVICES .......................... 15 3.1 INTRODUCTION.................................................................................................................... 15 3.2 VL1: POWDER DIFFRACTION ................................................................................................ 15 3.3 VL2: SMALL ANGLE SCATTERING .......................................................................................... 18 3.4 ISIS ICAT .......................................................................................................................... 19 3.5 SUMMARY ........................................................................................................................... 24 4 SCIENCE 3D – THE TOMOGRAPHY DATABASE .............................................................. 25 4.1 THE SCIENCE 3D KICK-OFF WORKSHOP ............................................................................... 26 4.2 CONCEPT OF THE SCIENCE 3D OPEN ACCESS CATALOGUE .................................................... 26 4.3 CURRENT STATUS OF THE SCIENCE 3D OPEN ACCESS CATALOGUE ....................................... 29 4.3.1 Schema ...................................................................................................................................... 30 4.3.2 Catalog ....................................................................................................................................... 30 4.3.3 Documentation & Dissemination ................................................................................................ 31 4.3.4 Summary .................................................................................................................................... 31 5 NEXUS DEPLOYMENT ........................................................................................................ 33 5.1 INTRODUCTION.................................................................................................................... 33 5.2 STATUS OF THE NEXUS LIBRARY .......................................................................................... 33 5.2.1 Why implementing a new Nexus library? ................................................................................... 33 5.2.2 Project status ............................................................................................................................. 34 5.2.3 Performance markers for libpnicore ........................................................................................... 35 5.2.4 Performance markers for libpniio ............................................................................................... 36 5.3 EXTERNAL FILTERS.............................................................................................................. 37 5.3.1 First implementation of an external filters infrastructure ............................................................ 38 5.3.2 External filters and user applications ......................................................................................... 39 5.3.3 External filters with Python ......................................................................................................... 40 5.4 NEXUS INTEGRATION AT DESY ........................................................................................... 41 5.4.1 Overview .................................................................................................................................... 41 5.4.2 The configuration procedure ...................................................................................................... 42 Page 3 of 107 PaN-data ODI Deliverable: D5.3 5.4.3 Component selection ................................................................................................................. 45 5.4.4 Mandatory metadata .................................................................................................................. 47 5.4.5 The data acquisition process ..................................................................................................... 48 5.4.6 Generalizing the DESY NeXus implementation......................................................................... 49 5.5 NEXUS INTEGRATION AT ALBA ............................................................................................ 49 5.5.1 Overview .................................................................................................................................... 49 5.5.2 Application definitions ................................................................................................................ 51 5.5.3 Data and metadata sources selection ....................................................................................... 51 5.6 NEXUS INTEGRATION AT DLS .............................................................................................. 53 5.6.1 Overview .................................................................................................................................... 53 5.6.2 Device selection ......................................................................................................................... 53 5.6.3 Configuration .............................................................................................................................. 54 5.6.4 Default metadata ........................................................................................................................ 54 5.6.6 Application definitions ................................................................................................................ 55 5.7 NEXUS INTEGRATION AT HZB .............................................................................................. 55 5.7.1 Overview .................................................................................................................................... 55 5.7.2 Mandatory metadata .................................................................................................................. 56 5.7.3 Generalizing the HZB NeXus implementation ........................................................................... 57 5.8 NEXUS INTEGRATION AT ILL ................................................................................................ 57 5.8.1 Configuration .............................................................................................................................
Recommended publications
  • The Qtiplot Handbook I
    The QtiPlot Handbook i The QtiPlot Handbook The QtiPlot Handbook ii Copyright © 2004 - 2011 Ion Vasilief Copyright © 2010 Stephen Besch Copyright © 2006 - june 2007 Roger Gadiou and Knut Franke Legal notice: Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documen- tation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. The QtiPlot Handbook iii COLLABORATORS TITLE : The QtiPlot Handbook ACTION NAME DATE SIGNATURE WRITTEN BY Ion Vasilief and 22 February 2011 Stephen Besch REVISION HISTORY NUMBER DATE DESCRIPTION NAME The QtiPlot Handbook iv Contents 1 Introduction 1 1.1 What QtiPlot does...................................................1 1.2 Command Line Parameters..............................................1 1.2.1 Specify a File.................................................1 1.2.2 Command Line Options...........................................2 1.3 General Concepts and Terms.............................................2 1.3.1 Tables.....................................................4 1.3.2 Matrix.....................................................5 1.3.3 Plot Window.................................................6 1.3.4 Note......................................................7 1.3.5 Log Window.................................................8 1.3.6 The Project Explorer.............................................9 2 Drawing plots with QtiPlot 10 2.1 2D
    [Show full text]
  • Elenco Di Programmi Open Source
    Elenco di programmi open source Da Wikipedia, l'enciclopedia libera. Questo è un elenco di programmi open source: software rilasciato sotto una licenza open source. Programmi che rispettano la definizione di open source possono essere chiamati anche Software Libero; in particolare il progetto GNU è contrario all'uso di open source per indicare i suoi prodotti. Per ulteriori informazioni sul background filosofico del software open source, vedi movimento open source e movimento Software Libero. Tuttavia, quasi ogni programma che rispetti la Definizione di Open Source è anche Software Libero, e viceversa, e dunque è elencato qui. Indice 1 Campi applicativi o 1.1 CAx . 1.1.1 CAD (Computer-Aided Drafting/Design) . 1.1.2 CAE (Computer-Aided Engineering) o 1.2 Finanza - Gestionali o 1.3 Matematica o 1.4 Scienza . 1.4.1 Sistema informativo geografico . 1.4.2 Plotting . 1.4.3 Scanning probe microscopy . 1.4.4 Fisica o 1.5 E.R.P. (Enterprise Resource Planning) - Sistemi Informativi Aziendali 2 Accessibilità 3 Raccolta e gestione dei dati o 3.1 Software di backup e salvataggio dati o 3.2 Archiviatori - Compressione dei dati o 3.3 Sistemi di gestione di database (compresi tool di amministrazione) o 3.4 Gestione infrastruttura IT o 3.5 Data mining 4 Programmi per la tipografia digitale (document editing) o 4.1 Suite office o 4.2 Word processing o 4.3 Applicazioni per appunti o 4.4 PDF o 4.5 DjVu (visualizzatori) o 4.6 Editor di testo matematici/scientifici o 4.7 Fogli di calcolo o 4.8 Editor di testo o 4.9 Editor web o 4.10 Impaginazione o 4.11
    [Show full text]
  • The Qtiplot Handbook I
    The QtiPlot Handbook i The QtiPlot Handbook The QtiPlot Handbook ii Copyright © 2004 - 2011 Ion Vasilief Copyright © 2010 Stephen Besch Copyright © 2006 - june 2007 Roger Gadiou and Knut Franke Legal notice: Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documen- tation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. The QtiPlot Handbook iii COLLABORATORS TITLE : The QtiPlot Handbook ACTION NAME DATE SIGNATURE WRITTEN BY Ion Vasilief and 22 February 2011 Stephen Besch REVISION HISTORY NUMBER DATE DESCRIPTION NAME The QtiPlot Handbook iv Contents 1 Introduction 1 1.1 What QtiPlot does...................................................1 1.2 Command Line Parameters..............................................1 1.2.1 Specify a File.................................................1 1.2.2 Command Line Options...........................................2 1.3 General Concepts and Terms.............................................2 1.3.1 Tables.....................................................4 1.3.2 Matrix.....................................................5 1.3.3 Plot Window.................................................6 1.3.4 Note......................................................7 1.3.5 Log Window.................................................8 1.3.6 The Project Explorer.............................................9 2 Drawing plots with QtiPlot 10 2.1 2D
    [Show full text]
  • Scientific Software Useful for the Undergraduate Physics Laboratory
    Scientific software useful for the Undergraduate Physics Laboratory Windows Microcal Origin Origin is quite well distributed within the scientific community and is used in some of the research groups in the Faculty of Physics and Geosciences in Leipzig. There is a license available for students. Origin 8 and 7.5 might be downloaded from http://research.uni-leipzig.de/zno/Software/ . This website is accessible from your computer at home via a VPN connection (see https://www.urz.uni-leipzig.de/hilfe/anleitungen-a-z/vpn/ ). Running Origin requires a connection to the license server; this is unproblematic, if you computer is logged in into the University network; from home, it should sufficient to run the VPN client on your computer. Alternatively, you might bring your notebook to the University, connect to the wireless, start Origin and borrow the license for a maximum of 150 days. Please note that the license is automatically returned the next time you open Origin, when connected to the University network. For other options see Mac and Linux. Mac IGOR This is software from Wavemetrics; IGOR runs also under Windows. License for students? No idea. For other options see Linux. Linux Qtiplot Qtiplot is free software (in principle) and quite similar to Origin. It runs under Linux, Windows and Mac. The official site is (http://soft.proindependent.com/qtiplot.html). Qtiplot is provided in some GNU/Linux distributions (e.g., in the official repositories of Debian and Ubuntu). Windows binaries might be legally downloaded from http://www.cells.es/Members/cpascual/docs/unofficial-qtiplot- packages-for-windows.
    [Show full text]
  • A Review on the Comparative Roles of Mathematical Softwares in Fostering Scientific and Mathematical Research
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Covenant University Repository Global Journal of Pure and Applied Mathematics. ISSN 0973-1768 Volume 11, Number 6 (2015), pp. 4937-4948 © Research India Publications http://www.ripublication.com A Review On The Comparative Roles Of Mathematical Softwares In Fostering Scientific And Mathematical Research Emetere Moses Eterigho1 and Sanni Eshorame Samuel2 1Physics Department, Covenant University, P.M.B. 1023, Ota, Nigeria 2Chemical Engineering Department, Covenant University, P.M.B. 1023, Ota, Nigeria [email protected] Abstract Mathematical software tools used in science, research and engineering have a developmental trend. Various subdivisions for mathematical software applications are available in the aforementioned areas but the research intent or problem under study, determines the choice of software required for mathematical analyses. Since these software applications have their limitations, the features present in one type are often augmented or complemented by revised versions of the original versions in order to increase their abilities to multi-task. For example, the dynamic mathematics software was designed with integrated advantages of different types of existing mathematics software as an improved version for understanding numerical related problems for advanced mathematical content (advanced simulation). In recent times, science institutions have adopted the use of computer codes in solving mathematics related problems. The treatment of complex numerical analysis with the aid of mathematical software is currently used in all branches of physical, biological and social sciences. However, the programming language for mathematics related software varies with their functionalities. Many invaluable researches have been compromised within the confines of unacceptable but expedient standards because of insufficient understanding of the valuable services the available variety of mathematical software could offer.
    [Show full text]
  • Sportscience In-Brief 2011
    SPORTSCIENCE · sportsci.org News & Comment / In Brief • The Best Graphing Software. Solving the problems with Office. • Updates: Sample size; Clinical inferences; Validity and reliability Reprint pdf · Reprint doc The Best Graphing Software Will G Hopkins, Sport and Recreation, AUT University, Auckland, New Zealand. Email. Sportscience 15, i-iii, 2011 (sportsci.org/2011/inbrief.htm#graphs). Published June 2011. ©2011 Update Aug 2018. Bill Microsoft Gates has at 2010 has unacceptable bugs for such pro- last solved the problem (detailed below) of cessing, and my IT people are telling me I have lines turning into tram tracks when graphics to give up Excel and Powerpoint 2003 when from Excel are pasted into Powerpoint as a they install Windows 7 on my laptop soon. So metafile and ungrouped for further editing. I am looking for graphing software that will Thanks, Bill, you got there in the end! The make graphs I can modify with the latest Pow- latest version of Powerpoint in Office 2016 also erpoint. In any case, I am interested in software has a wonderful new feature for lining up ob- that does a better job than Excel, although I still jects, whereby dashed lines appear when edges have to use Excel: all my spreadsheet resources of an object you are moving line up with edges are in Excel, so anyone using the spreadsheets, of other nearby objects. Now, Bill, please rein- including me, will still want to transfer figures state Recent Folders in the Save As… windows from Excel into Powerpoint. in Windows 10. Solution #1: Keep Using Office 2003 Update Feb 2018.
    [Show full text]
  • Technical Notes All Changes in Fedora 13
    Fedora 13 Technical Notes All changes in Fedora 13 Edited by The Fedora Docs Team Copyright © 2010 Red Hat, Inc. and others. The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/. The original authors of this document, and Red Hat, designate the Fedora Project as the "Attribution Party" for purposes of CC-BY-SA. In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version. Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law. Red Hat, Red Hat Enterprise Linux, the Shadowman logo, JBoss, MetaMatrix, Fedora, the Infinity Logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries. For guidelines on the permitted uses of the Fedora trademarks, refer to https:// fedoraproject.org/wiki/Legal:Trademark_guidelines. Linux® is the registered trademark of Linus Torvalds in the United States and other countries. Java® is a registered trademark of Oracle and/or its affiliates. XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries. All other trademarks are the property of their respective owners. Abstract This document lists all changed packages between Fedora 12 and Fedora 13.
    [Show full text]
  • 1. Why POCS.Key
    Symptoms of Complexity Prof. George Candea School of Computer & Communication Sciences Building Bridges A RTlClES A COMPUTER SCIENCE PERSPECTIVE OF BRIDGE DESIGN What kinds of lessonsdoes a classical engineering discipline like bridge design have for an emerging engineering discipline like computer systems Observation design?Case-study editors Alfred Spector and David Gifford consider the • insight and experienceof bridge designer Gerard Fox to find out how strong the parallels are. • bridges are normally on-time, on-budget, and don’t fall ALFRED SPECTORand DAVID GIFFORD • software projects rarely ship on-time, are often over- AS Gerry, let’s begin with an overview of THE DESIGN PROCESS bridges. AS What is the procedure for designing and con- GF In the United States, most highway bridges are budget, and rarely work exactly as specified structing a bridge? mandated by a government agency. The great major- GF It breaks down into three phases: the prelimi- ity are small bridges (with spans of less than 150 nay design phase, the main design phase, and the feet) and are part of the public highway system. construction phase. For larger bridges, several alter- There are fewer large bridges, having spans of 600 native designs are usually considered during the Blueprints for bridges must be approved... feet or more, that carry roads over bodies of water, preliminary design phase, whereas simple calcula- • gorges, or other large obstacles. There are also a tions or experience usually suffices in determining small number of superlarge bridges with spans ap- the appropriate design for small bridges. There are a proaching a mile, like the Verrazzano Narrows lot more factors to take into account with a large Bridge in New Yor:k.
    [Show full text]
  • Evaluation of the Usability Areasof the Open Source and Public Domain Applicationsduring the Creating of E-Learning Courses
    EduAkcja. Magazyn edukacji elektronicznej nr 2 (12)/2016, str. 51—69 Ocena obszarów stosowalności aplikacji Open Source oraz Public Domain podczas tworzenia kursów e-learningowych Barbara Dębska Politechnika Rzeszowska [email protected] Lucjan Dobrowolski Politechnika Rzeszowska [email protected] Karol Hęclik Politechnika Rzeszowska [email protected] Streszczenie: W artykule przedstawiono wyniki oceny obszarów stosowalności wybranych aplikacji Open Source oraz Public Domain, wykorzystywanych podczas wytwarzania kursów e-learningowych. Omówiono ich funkcje i ergonomię pracy oraz niedostatki. Uwzględniono przeszkody, na jakie napotykają osoby przygotowujące doku- menty, przeznaczone do umieszczenia w materiałach e-learningowych. Opisane aplikacje pogrupowano tematycz- nie wg rodzajów przetwarzanych treści, a następnie oceniono pod kątem możliwości wykorzystywania nowych technologii internetowych w e-kursach. Zwrócono uwagę na trudności, jakie mogą sprawiać niektóre nowe tech- nologie teleinformatyczne podczas przygotowywania treści multimedialnych. W szczególności dotyczy to animacji realizowanych przy pomocy HTML5, CSS3, JavaScript i SVG. Słowa kluczowe: e-learning, open source, narzędzia pomocnicze, multimedia 1. Wprowadzenie 1.1. Treści kursów e-learningowych Obecnie coraz częściej strony kursów e-learningowych zawierają nie tylko tekst i grafikę statycz- ną, ale także treści multimedialne (Rys. 1). Przygotowanie takich materiałów wymaga użycia do- datkowych aplikacji, bowiem podstawowe narzędzia do przygotowania e-kursów nie
    [Show full text]
  • The Qtiplot Handbook
    The QtiPlot Handbook Ion Vasilief, Roger Gadiou, and Knut Franke April 10, 2008 Contents 1 Introduction 1 1.1 What QtiPlot does ....................................................... 1 1.2 Command Line Parameters .................................................. 1 1.2.1 Specify a File ..................................................... 1 1.2.2 Command Line Options ............................................... 2 1.3 General Concepts and Terms ................................................. 2 1.3.1 Tables ......................................................... 3 1.3.2 Matrix ......................................................... 4 1.3.3 Plot Window ..................................................... 5 1.3.4 Note .......................................................... 6 1.3.5 Log Window ..................................................... 6 1.3.6 The Project Explorer ................................................. 6 2 Drawing plots with QtiPlot 8 2.1 2D plots ............................................................ 8 2.1.1 2D plot from data. .................................................. 8 2.1.2 2D plot from function. ................................................ 10 2.1.2.1 Direct plot of a function. ......................................... 10 2.1.2.2 Filling of a table with the values of a function. .............................. 11 2.2 3D plots ............................................................ 12 2.2.1 Direct 3D plot from a function ............................................ 13 2.2.2 3D plot
    [Show full text]
  • The Scidavis Handbook
    The SciDAVis Handbook Ion Vasilief, Roger Gadiou, Knut Franke, Fellype Nascimento, and Miquel Garriga June 25, 2018 Contents 1 Introduction1 1.1 What is SciDAVis?...........................1 1.2 Command Line Parameters.......................2 1.2.1 Specify a File..........................2 1.2.2 Command Line Options....................3 1.3 General Concepts and Terms......................3 1.3.1 Tables..............................5 1.3.2 Matrix..............................7 1.3.3 Plot Window..........................8 1.3.4 Note...............................8 1.3.5 Log Window..........................9 1.3.6 The Project Explorer...................... 10 2 Drawing plots with SciDAVis 11 2.1 2D X-Y plots.............................. 11 2.1.1 2D plot from data........................ 11 2.1.2 2D plot from function...................... 15 2.1.2.1 Direct plot of a function............... 15 2.1.2.2 Filling of a table with the values of a function.... 17 2.1.3 The different types of 2D X-Y plots.............. 18 2.1.4 Customization of a 2D plot................... 18 2.1.4.1 "Plot details" window................ 19 2.1.4.1.1 Options for the layer........... 19 2.1.4.1.2 Custom curves for data series....... 20 2.1.5 Changing default 2D plot options............... 22 2.1.5.1 Modification of default options........... 23 2.1.6 Working with templates.................... 24 2.2 Other special 2D plots......................... 25 2.2.1 Pie plots............................. 25 2.2.1.1 Formatting of pie plots............... 25 2.2.2 Vectors plots.......................... 26 2.2.2.1 Formatting of vector plots.............. 27 2.3 Statistical plots............................. 28 2.3.1 Box plots...........................
    [Show full text]
  • Introduction to DFT Simulations of Molecules and Solids
    HybridD3 Theory Training Workshop: Introduction to DFT simulations of Molecules and Solids September 28, 2018 A quick summary of the exercises A guideline through the tutorial This tutorial aims to give a basic introduction to electronic structure calculations for very simple systems. As every DFT code has its own philosophy, this tutorial should also familiarize you with fundamental aspects of using FHI-aims. The goal of the first section is to explain the basic inputs of FHI-aims and to demonstrate that DFT calculations can have predictive power for many observable quantities. The practice session consists of two parts: Part I: Basic electronic structure with FHI-aims Problem I: The hydrogen atom Problem II: Hydrofluoric acid (HF): bond length and dipole moment Part II: Periodic systems Problem I: Generation and visualization of bulk structures Problem II: Energy convergence tests Problem III: Phase stability and cohesive properties Problem IV: Unit cell relaxation Problem V: Electronic band structure & density of states Problem VI: The effects of spin-orbit coupling on a free atom Problem VII: The effects of spin-orbit coupling on periodic materials As first step please copy the folder tutorial_1 from $HandsOn to your working directory. For every exercise, we also provide solutions and sample input files. They can be found in $HandsOn/tuto- rial_1/solutions and $HandsOn/tutorial_1/skel/exercise_X/templates, respectively. However, we strongly recom- mend to use the provided input files only in case of time shortage. You will maximize your learning progress by trying to generate the input files on your own. In case you get stuck with a particular problem, do not hesitate to ask one of the tutors.
    [Show full text]