DOCSLIB.ORG

Digitalcommons@USU

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Digitalcommons@USU Utah State University DigitalCommons@USU All Graduate Theses and Dissertations Graduate Studies 12-2009 Parallelization of Performance Limiting Routines in the Computational Fluid Dynamics General Notation System Library Kyle Horne Utah State University Follow this and additional works at: https://digitalcommons.usu.edu/etd Part of the Mechanical Engineering Commons Recommended Citation Horne, Kyle, "Parallelization of Performance Limiting Routines in the Computational Fluid Dynamics General Notation System Library" (2009). All Graduate Theses and Dissertations. 522. https://digitalcommons.usu.edu/etd/522 This Thesis is brought to you for free and open access by the Graduate Studies at DigitalCommons@USU. It has been accepted for inclusion in All Graduate Theses and Dissertations by an authorized administrator of DigitalCommons@USU. For more information, please contact [email protected]. PARALLELIZATION OF PERFORMANCE LIMITING ROUTINES IN THE COMPUTATIONAL FLUID DYNAMICS GENERAL NOTATION SYSTEM LIBRARY by Kyle Horne A thesis submitted in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE in Mechanical Engineering Approved: Dr. Thomas Hauser Dr. Robert Spall Major Professor Committee Member Dr. Heng Ban Dr. Byron Burnham Committee Member Dean of Graduate Studies UTAH STATE UNIVERSITY Logan, Utah 2009 ii Copyright © Kyle Horne 2009 All Rights Reserved iii Abstract Parallelization of Performance Limiting Routines in the Computational Fluid Dynamics General Notation System Library by Kyle Horne, Master of Science Utah State University, 2009 Major Professor: Dr. Thomas Hauser Department: Mechanical and Aerospace Engineering The Computational Fluid Dynamics General Notation System provides a unified way in which computational fluid dynamics data can be stored, but does not support the parallel I/O capabilities now available from version five of the Hierarchical Data Format library which serves as a back end for the standard. To resolve this deficiency, a new parallel extension library has been written and benchmarked for this work which can write files compliant with the standard using parallel file access modes. When using this new library, the write performance shows an increase of four- fold in some cases when compared to the same hardware operating in serial. Additionally, the use of parallel I/O allows much larger cases to be written since the problem is scattered across many nodes of a cluster, whose aggregate memory is much greater than that found on a single machine. These developments will allow computational fluid dynamics simulations to execute faster, since less time will be spent waiting for each time step to finish writing, as well prevent the need for lengthy reconstruction of data after the completion of a simulation. (293 pages) iv Acknowledgments Firstly, I would like to thank my wife, Lydia, for supporing me in the completion of this project. Her efforts to make our humble apartment our home have made all the difference. Secondly, I want to thank my parents for encouraging my curiosity and scientific interest from an early age, and giving me the freedom to become who I am. I would also like to thank Dr. Thomas Hauser for employing me since my senior year of undergraduate studies. He has funded my research and provided me with opportunities to grow that I had not even considered before. Additionally, I would thank the faculty for their efforts in making the undergraduate and graduate programs at Utah State University environments in which students can succeed. Much credit must also go to Bonnie Ogden for making sure that graduate students get all their paperwork done. Additionally, I would thank Mr. Mike Kennedy, Mrs. Vanessa Liveris, Ms. Nicole Burt, and Mr. Gussman from Neuqua Valley High School for the sound foundation in science and mathemat- ics which I received there. Kyle Horne v Contents Page Abstract ......................................................... iii Acknowledgments .................................................. iv List of Tables ..................................................... viii List of Figures ..................................................... ix 1 Introduction ................................................... 1 1.1 Scientific Computing . 1 1.1.1 Data Storage History . 1 1.1.2 System Level I/O . 2 1.1.3 Hierarchical Data Format . 3 1.1.4 Supercomputing . 3 1.1.5 Vector Computers . 3 1.1.6 Cluster Computing . 4 1.2 Computational Fluid Dynamics . 5 1.2.1 Problem Formulation . 5 1.2.2 Solution Algorithms . 7 1.2.3 Parallel Implementations . 8 1.2.4 Storage Concerns . 8 2 Problem Description .............................................. 13 2.1 Problem Introduction . 13 2.2 Problem Background . 16 2.3 Computational Fluid Dynamics General Notation System . 18 2.3.1 Standard Interface Data Structures . 18 2.3.2 Advanced Data Format . 18 2.3.3 Hierarchical Data Format version 5 . 20 vi 2.3.4 Mid-level Library . 20 3 Literature Review ............................................... 21 3.1 Previous Work . 21 3.2 Parallel Input/Output . 22 3.2.1 Message Passing Interface Standard . 22 3.2.2 Network Attached Storage . 24 3.2.3 Message Passing Interface-I/O . 24 3.2.4 Parallel File Systems . 24 3.2.5 Hierarchical Data Format version 5 . 27 4 Parallel Implementation for the CFD General Notation System ................ 28 4.1 Design . 28 4.2 Features . 29 5 Results ....................................................... 31 5.1 Hardware Setup . 31 5.2 Benchmark Selection . 33 5.2.1 IOR . 33 5.2.2 pCGNS . 33 5.3 Benchmark Results . 34 5.3.1 IOR . 35 5.3.2 pCGNS . 38 6 Summary and Conclusion .......................................... 48 Bibliography ...................................................... 51 Appendix ........................................................ 58 A pCGNS Source Code & Documentation ................................ 59 A.1 Data Structure Index . 59 A.1.1 Data Structures . 59 A.2 File Index . 59 A.2.1 File List . 59 A.3 Data Structure Documentation . 60 vii A.3.1 base_s Struct Reference . 60 A.3.2 coords_s Struct Reference . 63 A.3.3 file_s Struct Reference . 64 A.3.4 iter_s Struct Reference . 69 A.3.5 section_s Struct Reference . 70 A.3.6 slice_s Struct Reference . 72 A.3.7 sol_s Struct Reference . 74 A.3.8 zone_s Struct Reference . 76 A.4 File Documentation . 80 A.4.1 benchmark.c File Reference . 80 A.4.2 benchmark.c . 88 A.4.3 open_close.c File Reference . 94 A.4.4 open_close.c . 96 A.4.5 pcgns_util.c File Reference . 97 A.4.6 pcgns_util.c . 119 A.4.7 pcgns_util.h File Reference . 140 A.4.8 pcgns_util.h . 163 A.4.9 pcgnslib.c File Reference . 168 A.4.10 pcgnslib.c . 186 A.4.11 pcgnslib.h File Reference . 228 A.4.12 pcgnslib.h . 249 A.4.13 test_base.c File Reference . 252 A.4.14 test_base.c . 254 A.4.15 test_queue.c File Reference . 256 A.4.16 test_queue.c . 258 A.4.17 test_unstructured.c File Reference . 260 A.4.18 test_unstructured.c . 262 A.4.19 test_zone.c File Reference . 265 A.4.20 test_zone.c . 266 A.4.21 thesis_benchmark.c File Reference . 268 A.4.22 thesis_benchmark.c . 274 viii List of Tables Table Page 4.1 API of the pCGNS library as implemented in pcgnslib.c and declared in pcgnslib.h. Software using the library to access CGNS files in parallel must do so using the routines listed here. 30 ix List of Figures Figure Page 1.1 A generalized diagram of a cluster computer (Upper) and a shared-memory com- puter (Lower). The abbreviations used in the diagram are as follows: CPU=Central Processing Unit; NIC=Network Interface Card; HD=Hard Disk. The dashed line connecting all the components of each computer represents the primary data bus of the machine. The simplicity of the shared-memory computer architecture com- pared to that of the cluster is apparent, making shared-memory computers desirable when possible. Unfortunately, the performance required from the main data bus in a shared-memory computer limits the scalability of the platform. 6 1.2 Mesh partitioned for parallel computation using Gmsh [1]. The mesh has been split into four partitions algorithmically, with the intent of minimizing the number of boundary edges between each partition. This minimization lowers the communica- tions overhead incurred when executing a CFD simulation on the mesh. 9 1.3 Mesh from a CFD solution of the common driven cavity problem on an unstructured grid. The mesh was generated using Gmsh [1], and is composed of triangles and quadrilaterals on the interior, with one dimensional cells all around the boundary. 11 1.4 Velocity field from a CFD solution of the common driven cavity problem on an unstructured grid. The solution was obtained using a code written by the author and is used only as an example of CFD results. The color corresponds to the velocity magnitude at each cell center and the lines portray both the direction and magnitude of the velocity. 11 1.5 Pressure field from a CFD solution of the common driven cavity problem on an unstructured grid. The solution was obtained using a code written by the author and is used only as an example of CFD results. The color corresponds to the relative pressure at each cell center. 12 1.6 General transported quantity field from a CFD solution of the common driven cavity problem on an unstructured grid. The solution was obtained using a code written by the author and is used only as an example of CFD results. The color corresponds to the magnitude of the quantity phi at each cell center. ..
Load more

Recommended publications
  • Introduction CFD General Notation System (CGNS)
    CGNS Tutorial Introduction CFD General Notation System (CGNS) Christopher L. Rumsey NASA Langley Research Center Outline • Introduction • Overview of CGNS – What it is – History – How it works, and how it can help – The future • Basic usage – Getting it and making it work for you – Simple example – Aspects for data longevity 2 Introduction • CGNS provides a general, portable, and extensible standard for the description, storage, and retrieval of CFD analysis data • Principal target is data normally associated with computed solutions of the Navier-Stokes equations & its derivatives • But applicable to computational field physics in general (with augmentation of data definitions and storage conventions) 3 What is CGNS? • Standard for defining & storing CFD data – Self-descriptive – Machine-independent – Very general and extendable – Administered by international steering committee • AIAA recommended practice (AIAA R-101A-2005) • Free and open software • Well-documented • Discussion forum: [email protected] • Website: http://www.cgns.org 4 History • CGNS was started in the mid-1990s as a joint effort between NASA, Boeing, and McDonnell Douglas – Under NASA’s Advanced Subsonic Technology (AST) program • Arose from need for common CFD data format for improved collaborative analyses between multiple organizations – Existing formats, such as PLOT3D, were incomplete, cumbersome to share between different platforms, and not self-descriptive (poor for archival purposes) • Initial development was heavily influenced by McDonnell Douglas’ “Common
    [Show full text]
  • Development of a Coupling Approach for Multi-Physics Analyses of Fusion Reactors
    Development of a coupling approach for multi-physics analyses of fusion reactors Zur Erlangung des akademischen Grades eines Doktors der Ingenieurwissenschaften (Dr.-Ing.) bei der Fakultat¨ fur¨ Maschinenbau des Karlsruher Instituts fur¨ Technologie (KIT) genehmigte DISSERTATION von Yuefeng Qiu Datum der mundlichen¨ Prufung:¨ 12. 05. 2016 Referent: Prof. Dr. Stieglitz Korreferent: Prof. Dr. Moslang¨ This document is licensed under the Creative Commons Attribution – Share Alike 3.0 DE License (CC BY-SA 3.0 DE): http://creativecommons.org/licenses/by-sa/3.0/de/ Abstract Fusion reactors are complex systems which are built of many complex components and sub-systems with irregular geometries. Their design involves many interdependent multi- physics problems which require coupled neutronic, thermal hydraulic (TH) and structural mechanical (SM) analyses. In this work, an integrated system has been developed to achieve coupled multi-physics analyses of complex fusion reactor systems. An advanced Monte Carlo (MC) modeling approach has been first developed for converting complex models to MC models with hybrid constructive solid and unstructured mesh geometries. A Tessellation-Tetrahedralization approach has been proposed for generating accurate and efficient unstructured meshes for describing MC models. For coupled multi-physics analyses, a high-fidelity coupling approach has been developed for the physical conservative data mapping from MC meshes to TH and SM meshes. Interfaces have been implemented for the MC codes MCNP5/6, TRIPOLI-4 and Geant4, the CFD codes CFX and Fluent, and the FE analysis platform ANSYS Workbench. Furthermore, these approaches have been implemented and integrated into the SALOME simulation platform. Therefore, a coupling system has been developed, which covers the entire analysis cycle of CAD design, neutronic, TH and SM analyses.
    [Show full text]
  • Integrated Tool Development for Used Fuel Disposition Natural System Evaluation –
    Integrated Tool Development for Used Fuel Disposition Natural System Evaluation – Phase I Report Prepared for U.S. Department of Energy Used Fuel Disposition Yifeng Wang & Teklu Hadgu Sandia National Laboratories Scott Painter, Dylan R. Harp & Shaoping Chu Los Alamos National Laboratory Thomas Wolery Lawrence Livermore National Laboratory Jim Houseworth Lawrence Berkeley National Laboratory September 28, 2012 FCRD-UFD-2012-000229 SAND2012-7073P DISCLAIMER This information was prepared as an account of work sponsored by an agency of the U.S. Government. Neither the U.S. Government nor any agency thereof, nor any of their employees, makes any warranty, expressed or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness, of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. References herein to any specific commercial product, process, or service by trade name, trade mark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the U.S. Government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the U.S. Government or any agency thereof. Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy‘s National Nuclear Security Administration under contract DE-AC04-94AL85000. Integrated Tool Development for UFD Natural System Evaluation 9/1/2012 iii Integrated Tool Development for UFD Natural System Evaluation iv 9/1/2012 Integrated Tool Development for UFD Natural System Evaluation 9/1/2012 v Executive Summary The natural barrier system (NBS) is an integral part of a geologic nuclear waste repository.
    [Show full text]
  • Seven Keys for Practical Understanding and Use of CGNS
    Seven keys for practical understanding and use of CGNS Marc Poinot∗ Christopher L. Rumseyy SAFRAN NASA Langley Research Center [email protected] [email protected] We present key features of the CGNS standard, focusing on its two main elements, the data model (CGNS/SIDS) and its implementations (CGNS/HDF5 and CGNS/Python). The data model is detailed to emphasize how the topological user oriented information, such as families, are separated from the actual meshing that could be split or modified during the CFD workflow, and how this topological information is traced during the meshing process. We also explain why the same information can be described in multiple ways and how to handle such alternatives in an application. Two implementations, using HDF5 and Python, are illustrated in several use examples, both for archival and interoperability purposes. The CPEX extension formalized process is explained to show how to add new features to the standard in a consensual way; we present some of the next extensions to come. Finally we conclude by showing how powerful a consensual public approach like CGNS can be, as opposed to a stand-alone private one. All throughout the paper, we demonstrate how the use of CGNS could be of great benefit for both the meshing and CFD solver communities. I. Introduction The CFD General Notation System12 (CGNS) is a public standard for the CFD community. It has had more than 20 years of feedback from users throughout the world in industry, universities, and government research labs. The CGNS name is now well known in the CFD community, but it is often only associated with the CGNS library (CGNS/MLL) used by CFD tools.
    [Show full text]
  • Summary of Software Using HDF5 by Name (December 2017)
    Summary of Software Using HDF5 by Name (December 2017) Name (with Product Open Source URL) Application Type / Commercial Platforms Languages Short Description ActivePapers (http://www.activepapers File format for storing .org) General Problem Solving Open Source computations. Freely available SZIP implementation available from Adaptive Entropy the German Climate Encoding Library Special Purpose Open Source Computing Center IO componentization of ADIOS Special Purpose Open Source different IO transport methods Software for the display and Linux, Mac, analysis of X-Ray diffraction Adxv Visualization/Analysis Open Source Windows images Computer graphics interchange Alembic Visualization Open Source framework API to provide standard for working with electromagnetic Amelet-HDF Special Purpose Open Source data Bathymetric Attributed File format for storing Grid Data Format Special Purpose Open Source Linux, Win32 bathymetric data Basic ENVISAT Toolbox for BEAM Special Purpose Open Source (A)ATSR and MERIS data Bers slices and holomy Bear Special Purpose Open Source representations Unix, Toolkit for working BEAT Visualization/Analysis Open Source Windows,Mac w/atmospheric data Cactus General Problem Solving Problem solving environment Rapid C++ technical computing environment,with built-in HDF Ceemple Special Purpose Commercial C++ support Software for working with CFD CGNS Special Purpose Open Source Analysis data Tools for working with partial Chombo Special Purpose Open Source differential equations Command-line tool to convert/plot a Perkin
    [Show full text]
  • Getting Data Into Visit
    LLNL-SM-446033 Getting Data Into VisIt July 2010 Version 2.0.0 Brad Whitlock wrence La Livermore National Laboratory ii DISCLAIMER This document was prepared as an account of work sponsored by an agency of the United States government. Neither the United States government nor Lawrence Livermore National Security, LLC, nor any of their employees makes any warranty, expressed or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trade- mark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States government or Lawrence Livermore National Security, LLC. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States government or Lawrence Liver- more National Security, LLC, and shall not be used for advertising or product endorsement purposes. This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory in part under Contract W-7405-Eng-48 and in part under Contract DE-AC52-07NA27344. iii iv Table of Contents Introduction Manual chapters . 2 Manual conventions . 2 Strategies . 2 Picking a strategy . 3 Definition of terms . 4 Creating compatible files Creating a conversion utility or extending a simulation . 7 Survey of database reader plug-ins. 9 BOV file format . 9 X-Y Curve file format. 12 Plain text ASCII files .
    [Show full text]
  • AIAA Paper 2012-1264
    AIAA-2012-1264 50th AIAA Aerospace Sciences Meeting, January 9 – 12, 2012, Nashville, TN Recent Updates to the CFD General Notation System (CGNS) Christopher L. Rumsey∗ NASA Langley Research Center, Hampton, VA 23681 Bruce Wedan† Computational Engineering Solutions, San Ramon, CA 94582 Thomas Hauser‡ University of Colorado Boulder, Boulder, CO 80309 Marc Poinot§ ONERA - The French Aerospace Lab, F-92322 Chatillon, FRANCE The CFD General Notation System (CGNS)—a general, portable, and extensible standard for the storage and retrieval of computational fluid dynamics (CFD) analysis data—has been in existence for more than a decade (Version 1.0 was released in May 1998). Both structured and unstructured CFD data are covered by the standard, and CGNS can be easily extended to cover any sort of data imaginable, while retaining backward compatibility with existing CGNS data files and software. Although originally designed for CFD, it is readily extendable to any field of computational analysis. In early 2011, CGNS Version 3.1 was released, which added significant capabilities. This paper describes these recent enhancements and highlights the continued usefulness of the CGNS methodology. Glossary of Terms ADF CGNS Advanced Data Format, www.grc.nasa.gov/www/cgns/CGNS docs current/adf API Application Programming Interface CGIO CGNS low-level library, www.grc.nasa.gov/www/cgns/CGNS docs current/cgio CHLone CGNS special purpose C library (for HDF5 files only), chlone.sourceforge.net CGNS CFD General Notation System, cgns.org CGNSTalk CGNS user forum,
    [Show full text]
  • BASEMENT Reference Manual
    BASEMENT System Manuals VAW - ETH Zurich v2.8.1 BASEMENT System Manuals VAW - ETH Zurich v2.8.1 Contents Preamble 5 Credits . 5 License ......................................... 7 1 Pre-Processing in QGIS with BASEmesh 11 1.1 Introduction . 11 1.2 Tutorial 1: Mesh Generation based on Pointwise Elevation Data . 12 1.2.1 Project Settings . 13 1.2.2 Coordinate Reference System Configuration . 14 1.2.3 Loading Input Data for Elevation Model . 14 1.2.4 Saving Layer as Shape File . 15 1.2.5 Loading the Model Boundary . 15 1.2.6 Editing the Model Boundary . 17 1.2.7 Loading Breakline Data . 19 1.2.8 Creation of the Elevation Model as TIN . 20 1.2.9 Adaption of the Breaklines for Quality Meshing . 21 1.2.10 Creation of Region Marker Points . 22 1.2.11 Creation of Quality Mesh . 24 1.2.12 Interpolating elevation data from elevation mesh (TIN) . 25 1.2.13 3D view of the mesh . 26 1.2.14 Export of mesh to 2dm . 27 1.3 Tutorial 2: Import/Modify an existing Mesh and use Raster Data as Elevation Model . 28 1.3.1 Importing a .2dm mesh file . 29 1.3.2 Modifying the material indices of elements . 29 1.3.3 Manual editing of mesh elements . 33 1.3.4 Renumbering the mesh . 34 1.3.5 Interpolating elevations from raster data . 35 1.4 Tutorial 3: Using dividing constraints along boundary cross sections and setting up a BASEMENT simulation with a mesh from BASEmesh . 35 1.4.1 Using dividing constraints for Quality meshing .
    [Show full text]
  • About Pycgns 3 1.1 CGNS Standard
    pyCGNS.intro/Manual Release 4.2.0 Marc Poinot August 27, 2007 CONTENTS 1 About pyCGNS 3 1.1 CGNS Standard...........................................3 1.2 Package contents...........................................4 1.3 Quick start..............................................5 2 Build and Install 7 2.1 Required libraries..........................................7 2.2 Optional libraries...........................................7 2.3 Installation process..........................................8 2.4 Single module installation......................................8 2.5 Configuration file contents......................................8 2.6 NAV depends.............................................9 2.7 MAP depends............................................9 2.8 WRA depends............................................9 3 Glossary 11 4 PDF Docs 13 i ii pyCGNS.intro/Manual, Release 4.2.0 Release 4.2 • CGNS.MAP is now a wrapper to CHLone python module • CGNS.WRA reborn, re-coding using Cython • CGNS.VAL reborn • completely new CGNS.NAV using Qt, VTK and Cython • YouTube demos here The package gathers various tools and libraries for CGNS end-users and Python application developpers. The main object of pyCGNS is to provide the application developpers with a Python interface to CGNS/SIDS, the data model. The MAP and PAT modules are dedicated to this goal: map the CGNS/SIDS data model the CGNS/Python implementation. The WRA module contains wrapper on CGNS/MLL and a MLL-like set of functions that uses the CGNS/Python mapping as implementation. The NAV module supports the CGNS.NAV graphical browser, with nice features about tree exploration, copy/paste and even global node changes. Then, the VAL module is a parser engine for CGNS/Python tree compliance checking. The CGNS.VAL tool can analyze your CGNS/HDF5 file and returns you a large panel of diagnostics.
    [Show full text]
  • MASS2, Modular Aquatic Simulation System in Two Dimensions, User
    PNNL-14820-2 MASS2 Modular Aquatic Simulation System in Two Dimensions User Guide and Reference W. A. Perkins M. C. Richmond September 2004 Prepared for the U.S. Department of Energy under Contract DE-AC06-76RL01830 DISCLAIMER United States Government. Neither the United States Government nor any agency thereof, nor Battelle Memorial Institute, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof, or Battelle Memorial Institute. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof. PACIFIC NORTHWEST NATIONAL LABORATORY operated by BATTELLE for the UNITED STATES DEPARTMENT OF ENERGY under Contract DE-AC06-76RLO1830 Printed in the United States of America Available to DOE and DOE contractors from the Office of Scientific and Technical Information, P.O. Box 62, Oak Ridge, TN 37831-0062; ph: (865) 576-8401 fax: (865) 576-5728 email: [email protected] Available to the public from the National Technical Information Service, U.S. Department of Commerce, 5285 Port Royal Rd., Springfield, VA 22161 ph: (800) 553-6847 fax: (703) 605-6900 email: [email protected] online ordering: http://www.ntis.gov/ordering.htm This document was printed on recycled paper.
    [Show full text]
  • Data Formats for Visualization and I T Bilit Interoperability
    Data Formats for Visualization and Int eroperabilit y Steve Lantz Senior Research Associate 10/27/2008 www.cac.cornell.edu 1 How will you store your data? • Raw binary is compact but not portable – “Unformatted,” machine-specific representation – Byte-order issues: big endian (IBM) vs. little endian (Intel) • Formatted text is portable but not compact – Need to know all the details of formatting just to read the data – 1 byyygg()te of ASCII text stores only a single decimal digit (~3 bits) – Most of the fat can be knocked out by compression (gzip, bzip, etc.) – However, compression is impractical and slow for large files • Need to consider how data will ultimately be used – Are you trying to ensure future portability? – Will your favored analysis tools be able to read the data? – What storage constraints are there? 10/27/2008 www.cac.cornell.edu 2 Issues beyond the scope of this talk… • Provenance – The record of the origin or source of data – The history of the ownership or location of data – Purpose: to confirm the time and place of, and perhaps the person responsible for, the creation, production or discovery of the data • Curation – Collecting, cataloging, organizing, and preserving data • Ontology – Assigning t ypes and pro perties to data ob jects – Determining relationships among data objects – Associating concepts and meanings with data (semantics) •Portab le data fo rmats can an d do addr ess som e of th ese i ssues… 10/27/2008 www.cac.cornell.edu 3 Portable data formats: the HDF5 technology suite • Versatile data model that
    [Show full text]
  • Tutorial Session Agenda
    CFD General Notation System (CGNS) Tutorial Session Agenda • 7:00-7:30 Introduction, overview, and basic usage C. Rumsey (NASA Langley) • 7:30-7:50 Usage for structured grids B. Wedan (ANSYS – ICEM) • 7:50-8:10 Usage for unstructured grids E. van der Weide (Stanford University) • 8:20-8:40 HDF5 usage and parallel implementation T. Hauser (Utah State University) • 8:40-9:00 Python and in-memory CGNS trees M. Poinot (ONERA) • 9:00-9:30 Discussion and question/answer period 2 CFD General Notation System (CGNS) Introduction, overview, and basic usage Christopher L. Rumsey NASA Langley Research Center Chair, CGNS Steering Committee Outline • Introduction • Overview of CGNS – What it is – History – How it works, and how it can help – The future • Basic usage – Getting it and making it work for you – Simple example – Aspects for data longevity 4 Introduction • CGNS provides a general, portable, and extensible standard for the storage and retrieval of CFD analysis data • Principal target is data normally associated with computed solutions of the Navier-Stokes equations & its derivatives • But applicable to computational field physics in general (with augmentation of data definitions and storage conventions) 5 What is CGNS? • Standard for defining & storing CFD data – Self-descriptive – Machine-independent – Very general and extendable – Administered by international steering committee • AIAA recommended practice (AIAA R-101-2002) • In process of becoming part of international ISO standard • Free and open software • Well-documented • Discussion
    [Show full text]