The Earth System Grid: A Visualisation Solution

Gary Strand Introduction Acknowledgments

•PI’s •ESG Development Team –Veronika Nefedova (ANL) –Ian Foster (ANL) –Ann Chervenak (ISI/USC) –Don Middleton (NCAR) –Carl Kesselman (ISI/USC) –Dean Williams (LLNL) –David Bernholdt (ORNL) –Kasidit Chanchio (ORNL) –Line Pouchard (ORNL) –Alex Sim (LBNL) –Arie Shoshani (LBNL) –Bob Drach (LLNL) –Dave Brown (NCAR) –Gary Strand (NCAR) –Jose Garcia (NCAR) –Luca Cinquini (NCAR) –Peter Fox (NCAR) Current Practices oScientist (or others) wants a visualisation oVisualisation person gets appropriate data after verifying with data manager as to the name, location, total size, etc. oData moved to “local” machine that has visualisation tools oVisualization created on “local” machine oHopefully, someone remembers to archive the visualisation Simple Vis Example Problems in the process oWhat if the data cannot be found (e.g. we have 1.2 million files, 73 TB of data), or the data manager is unavailable? oWhat if there isn’t enough disk space or sufficient other resources? oWhat if a better visualisation tool is located elsewhere? oWhat if the visualisation should be shared? oWhat if the visualisation is lost? o ESG is part of the answers to these questions What is ESG?

ANL: Computational grids, LBNL: Climate storage & grid-based applications facility

LLNL: Model diagnostics & inter-comparison

USC/ISI: Computational grids, & grid-based applications

NCAR: Climate change LANL: Next generation ORNL: Climate storage & predication and scenarios coupled models & computing computational resources ESG Architecture

LBNL

HPSS! High Performance! disk! Storage System! ANL

CAS! Community Authorization Services! NCAR SRM! gridFTP! gridFTP! openDAPg! Storage Resource ! Striped! server! server! Management! server!

MyProxy! Tomcat servlet engine ! server!

MCS client! MyProxy client! disk! LLNL CAS client! RLS client!

SRM! Storage Resource ! gridFTP! GRAM! Management! server! gatekeeper! gridFTP! server! gridFTP! ORNL

SRM! ISI gridFTP! gridFTP! Storage Resource ! SRM! server! Management! SOAP Storage Resource ! MCS! Management! Cataloguing Services!

RLS! RMI HPSS! MSS! disk! High Performance! Replica Location Services! disk! Mass Storage System! Storage System! Solutions oWhat happens when data cannot be found, or the data manager is unavailable? •Metadata catalogue service (MCS) •Replica location service (RLS) MCS and RLS and Metadata Services

ESG CLIENTS API ! PUBLISHING! ANALYSIS & VISUALIZATION! & USER INTERFACES! SEARCH & DISCOVERY! ADMINISTRATION! BROWSING & DISPLAY!

HIGH LEVEL METADATA SERVICES! METADATA! METADATA! METADATA & DATA ! METADATA! METADATA! EXTRACTION! ANNOTATION! REGISTRATION! BROWSING! QUERY!

METADATA! METADATA! METADATA! METADATA! AGGREGATION! VALIDATION! DISPLAY! DISCOVERY!

CORE METADATA SERVICES! METADATA ACCESS! SERVICE TRANSLATION! (update, insert, delete, query)! LIBRARY!

METADATA HOLDINGS!

Data &! mirror! ! Dublin Core! COARDS! COMMENTS! Metadata! ! Database! XML Files! Catalog! XML Files! Solutions (contd.) oWhat if there isn’t enough disk space or sufficient other resources? •Hierarchical Resource Manager (HRM) HRM Solutions (contd.) oWhat What if a better visualization tool is located elsewhere? •Distributed visualization CDAT Example of an ESG Script Access  The next-generation language, Python, is used to access the Earth System Grid (ESG) at LLNL

Import cdms, vcs db = cdms.open(“ldap://localhost:389/database=demo,ou=PCMDI,o=LLNL,c=US”) f = db.open( “ncep_reanalysis_mo”) ds = f(‘ts’) x=vcs.init( ) x.plot(ds) CDAT: Example of an ESG GUI Client Access Solutions (contd.) oWhat if the visualization should be shared?

• Access Grid plus Visualisation Tool Collaborative Environments

Science Portals + AccessGrid: University of Michigan (Knoop, Hardin) Vegetation & Ecosystem Mapping Program (VEMAP) NCAR/SCD VETS/KEG Argonne National Labs Conclusions Visualisation can require as many services and resources as the initial computation Many sites do not offer sufficient resources for the visualisations earth sciences require ESG provides, and will provide, the tools that enable visualisation on a grander scale Conclusions (contd.) ESG tools enable better data access, better data knowledge, and the processes of collaboration for the needs of investigating, visualising, and learning