Japanese Virtual Observatory (JVO) and Naregi (Japanese Grid Middleware Initiative)
EURO-VO DCA Workshop Japanese Virtual Observatory (JVO) and NaReGi (Japanese Grid middleware initiative)
Masatoshi Ohishi / NAOJ, Sokendai & NII 大石雅寿 / 国立天文台，総合研究大学院大学， 国立情報学研究所
[email protected] agenda • Status of the Japanese Virtual Observatory • On-demand image mosaicing service for the SuprimeCAM data • Quick Multiple Catalog access service • Prototype for a Workflow builder • NaReGi – National Research Grid Initiative in Japan • A testbed to federate NAOJ & KEK through NaReGi
2 Supported by • JSPS “Core to Core Program” (2004~2009)
• MEXT Grant-in-Aid “Information Explosion” (2001~ )
• National Institute for Informatics “CSI Program” (2007~ )
• NAOJ
2008 April 10 EURO-VO DCA Workshop 3 JVO collaborators
Scientists System Engineers NAOJ Fujitsu Ltd. SEC Ltd. • Ohishi • Kawarai • Morita • Mizumoto • Ishihara • Nakamoto • Oe • Machida • Kobayashi • Shirasaki • Tsutsumi • Sakamoto • Tanaka • Hiyama • Tsuyuki • (Honda) ICRR • Yasuda Supporter Aoyama Gakuin U. NII • Miura • Masunaga
4 Data Resources in NAOJ
• Subaru 8.2m Optical-Infrared Telescope • Kiso 105cm Schmidt Camera Nobeyama 45m • Okayama 188cm Optical Telescope • Nobeyama 45m Radio Telescope • Nobeyama Millimeter Array • Nobeyama Radioheliograph Subaru • VSOP • VERA • ALMA Data Resources in JAXA/ISAS
• ASCA X-ray astronomy satellite • YOHKO solar physics satellite • Ginga X-ray astronomy satellite • HALCA VLBI satellite • Geotail geomagnetosphere satellite ASCA • Akebono aurora observation satellite • AKARI Infrared satellite • SUZAKU X-ray satellite • HINODE YOHKO Solar sat.
HALCA Japan – Its Uniqueness
• Two major astronomical research institutes cover entire spectrum – NAOJ: ground-based instruments (radio ~ optical) and a supercomputer – ISAS: space-based instruments
• There has been close collaboration between the two institutes – building satellites, dedicated line, personnel exchanges, etc.
7 Overview of the JVO Portal Service
LDAP 解析 解析 解析Analysis Grid Servers Auth. System JVO SDSS QSO
Invoke SkyNode Subaru MDS Portal 1.0 … 2MASS User Web Service Storage Data Service HTTP Get FTP S. Reg. LDAP Other VOs SIAP/SSAP P. Reg. SkyNode Registry P. Reg. 2008 April 10 EURO-VO DCA Workshop 0.7.4 8 Log(F_IR)
MAG_B
2008 April 10 EURO-VO DCA Workshop 9 Spectrum Data Viewer
• Display • Baseline estimation • Line detection & parameter read • ・・・
2008 April 10 EURO-VO DCA Workshop 10 User Portal
http://jvo.nao.ac.jp/portal/
11 Simple Interface to Multiple Catalog Query
12 Access to the SuprimeCAM Data
• On-Demand Mosaicing of images and calibrations
• Pre-processed mosaic images are also accessible
13 Management of Multiple Servers • Data Analysis services under JVO ‒ SExtractor (Image → Catalog) ‒ HyperZ (Catalog → photometric redshift) • In operation on multiple servers → Monitor and Discovery Service (MDS) server
JVO Portal MDS Server Analysis Servers # of Jobs Service Request CPU load Job status Execute Available Server Info Confirm Job Completion Analysis Server Job Completion Info Management Data
Job submission Client Results Parallel Executions on Multiple Servers are available 14 – Reduction 10s (Bias+Flat +Distortion+ Astrometory) – Transfer 30s
Data Reduction: ~24 hours ~1 hour 15 Pop-up Menu for various functions
16 Quick Image viewer, SED plotter, etc
17 Link to external resources
18 Interconnected VO Data services in the World
Canada VO
China VO
Data Service Operations have just been started March, 2008. 1300+ data resources are accessible.
19 Portal
VO summer school SuprimeCAM mosaic
20 Quick Multiple Catalog Search
Toward higher access efficiency to “Frequently Used DBs” -- ~ 109 records
21 Table Design
Extract the minimum set of meta data from each DB:
Position, Wavelength & Intensity
Other info is shown as LINKs to reference
22 Acceleration by means of Table Partitioning
Table design: Search Result # of HTM Elaspted time (sec) • Partitioning with HTM radius objects conditions (Hierarchical Postgre Our Postgre Our Triangular Mesh) arcmin # ratio SQL method SQL method • Upper level : level 6 6 : 8×4 = 32768 1 2 6.460 0.042 154 32 32 segments 10 165 3.807 0.030 127 16 16 • Tables are partitioned by Upper HTM level : 60 6697 6.468 0.107 60 32 32 32768 tables 100 26720 2.016 0.307 6.6 4 16
180 57246 9.044 0.709 12.8 48 72 (by Tanaka-san) 23 Improvement of Work Flow system
Items in FY2007 • Improvement of WFDL – Shorten the volume of DL – Array definition – Awk-like description (loop WF Editor description in a single line) – and so on • Built-in functions – Available to register and use built-in functions Execution monitor • Status and Log system • Need a GUI builder – Investigating reference systems
2008 April 10 EURO-VO DCA Workshop 24 WF builder prototype by means of Taverna • Good graphical interface together with corresponding script
• Builder prototype in 2007 • Easy construction of WFs w/ WFDL • Visual display of execution status
25 National Research Grid Initiative (NAREGI) Project : Overview
- Originally started as an R&D project funded by MEXT (FY2003-FY2007) extended until FY2010 2 B Yen(~17M$) budget per year
- One of Japanese Government’s Grid Computing Projects Visualization Grid, Grid Technology Research Center, BioGrid, ITBL (IT-Based Laboratory), etc.
- Collaboration of National Labs. Universities and Industry in the R&D activities (IT and nano-science apps.)
- NAREGI Testbed Computer Resources (since FY2003) MEXT:Ministry of Education, Culture, Sports,Science and Technology 26 (1) To develop a Grid Software System (R&D in Grid Middleware and Upper Layer) as the prototype of future Grid Infrastructure in scientific research in Japan (2)To provide a testbed to prove that the High-end Grid Computing Environment (100+Tflop/s) can be practically utilized in the nano-science applications over the Super SINET. (3) To Participate in International Collaboration /Interoperability (U.S., Europe, Asian Pacific) GIN (4) To Contribute to Standardization Activities, e.g., OGF NAREGI Software Stack
Grid-Enabled Nano-Applications
Grid Grid Visualization Grid PSE Programming Data Packaging -Grid RPC Grid Workflow Grid -Grid MPI Super Scheduler Distributed Information Service
Grid VM
High-Performance & Secure Grid Networking
SINET3
Research NII etc IMS Organizations Computing Resources Japanese Academic（Science） Information Network（ SINET/ SuperSINET SINET3） 29 Circuit Speed SINET (44 nodes) 1 Gbps
Super SINET (35 nodes) 10 Gbps Kitami Institute of Technology International Line U.S.A： 10 Gbps×1
2.4 Gbps×1 Hokkaido Singapore： 622 Mbps×1 University Hong Kong： 622 Mbps×1
Collaboration with other global network Hirosaki University
(1) “lifeline” for educaional and research activities in Japan (2) Indispensable for advanced research Niigata University Tohoku University cooperation Gunma University Japan Advanced Institute of High-energy Accelerator Research Saitama University (3) Continuous development towards Science and Technology Toyama Univ. Organization Kanazawa Univ. Univ. of Electro-Communications the establishment of the full-fledged Shinshu Univ. Univ. of Ttskuba National Astronomical Observatory of Japan Fukui Univ. National Institute Yamanashifor Tokyo Univ. Waseda Univ. CSI Tottori Univ. Fusion Science Univ. Agriculture Kyoto Univ. And Technology Univ. of Tokyo Hiroshima Univ. NII National Institute of Informatics Shizuoka Univ. Chiba University Nagoya Univ. Tokyo Institute of Technology Okayama Univ. Kobe Univ. Osaka Univ. Kyushu Institute National Institute Institute of Statistical Mathematics Kansai Univ.Doshisha Univ. Yokohama National Univ. of Technology Yamaguchi Univ. Kagawa Univ. Of Natural Science Keio Univ. Japan Aerospace ExplorationJapan Agency for Kyushu Univ. Univ. of Tokushima Agency Institute of Space Marine-Earth Science
Ehime Univ. Nagasaki Univ. and Astronautical Science Technology Oita Univ. Kumamoto Univ. Kagoshima Univ. Participation SINET Institutions Univ. of the Ryukyu National Public Private Junior Technical Inter-univ. Others Total Univ. Univ. Univ. College College institute
82 48 266 78 42 14 184 713 Collaboration in Data Grid Area
• High Energy Physics（GIN)
- KEK (High Energy Accelerator Research Organization) - under collaboration with EGEE • Astronomy - National Astronomical Observatory of Japan (Virtual Observatory) • Bio-informatics - BioGrid Project
30 Large Hadron Collider at CERN
Detector for LHCb experiment
Detector for ALICE experiment 31 ~1850 ATLAS Detector physicists from 33 countries
32 Visit to CERN 33 Multi-TMulti-Layeredier Regional Regional Center Analysis 34 Scheme Center for the LHC 1 TIPS = 25,000 SpecInt95 ~PBytes/sec Online System ~100 MBytes/sec PC (1999) = ~15 SpecInt95
Offline Farm ~20 TIPS Bunch crossing per 25 nsecs. ~100 MBytes/sec 100 triggers per second CERN Computer Event is ~1 MByte in size ~622 Mbits/sec Center >20 TIPS or Air Freight
France Regional Germany Regional Italy Regional US Regional Center Center Center Center ~4 TIPS ~2.4 Gbits/sec
Tier2 Center Tier2 CenterTier2 CenterTier2 CenterTier2 Center ~1 TIPS ~1 TIPS ~1 TIPS ~1 TIPS ~1 TIPS ~622 Mbits/sec Physicists work on analysis “channels”. InstituteInstitute Institute Institute ~0.25TIPS Each institute has ~10 physicists working Physics data cache 100 - 1000 on one or more channels Mbits/sec Data for these channels should be cached by the institute server Workstations Subaru SuprimeCam Single Shot Raw Image
• 10~20 minutes exposure 160 MB / shot • Postprocesses (Bias+Flat+Distortion+Astrometory, mosaicing to fill the gaps, coadding ) are needed
2008 April 10 EURO-VO DCA Workshop 35 Testbed for NAOJ-KEK federation User Browsers March 2008 User Browsers
NAOJ Site KEK Site Portal Portal Portal
SS SS SuperSched Private network IS-NAS IS-NAS Infosys-NAS 192.168.2.101~ IS-CDAS IS-CDAS GridVM Eng GridVM Eng Infosys-CDAS
GridVM Ser GVM Ser GridVM Serv MDS MDS MetaData Serv Compute nodes AMS FireWall AccessManage AMS. AMS
Gfarm Gfarm Gfarm Storage DataGrid part
NAT/DNS NAT/DNS NAT/DNS
UMS/VOMS VOMS/KEK UMS/VOMS UMS/VOMS Compute nodes Test VO: naokek Federation between KEK and NAOJ
Image processing job was submitted to servers in NAOJ & KEK via NaReGi middleware
Job status can be monitored on a GUI
2008 April 10 EURO-VO DCA Workshop 37 Federation Test in NAOJ-KEK
• Setup astro libraries at KEK site SUBARU Telescope in Hawaii • Job submission to KEK with Work Flow Tool(WFT) at the NAOJ Portal Input Data:(2.7 GB) • Input data are transferred from NAOJ 10 CCD mosaic images and Output data are staged-out to 160MB x 17 NAOJ portal
NAOJ job sub KEK
NAREGI NAREGI Process: 10 Hours Servers Servers retrieve flat, bias correct distortion astrometry Astro lib mosaicing (1GB) coadd 17 frames Data 2.7GB Visualization • Output data was processed with vis. software as shown in the right picture.
50k+ objects identified in this frame. SuprimeCam Data
•SuprimeCam Data ~ 10TB. It takes 154 days to analyse all the data. parallel processing is crucial • De tailed analysis of all the SuprimeCam data could lead to: • detection of the Cosmic Strings by means of the gravitational lens effect • detection of outliers •e tc
2008 April 10 EURO-VO DCA Workshop 39 Dissemination of the VO Technology
• Data centers for astronomy in Japan (and East Asia) – Visiting demonstrations – Regular training courses
• Related science fields – Planetary Science (c.f. IPDA) – Solar Terrestrial Environment – Informatics ~ DB researchers
40 Supernovae Hunting: 41 Informatics + Astronomy Summary
• JVO has started its operations (data services only) since March 2008. http://jvo.nao.ac.jp/portal/ More than 1TB of data was already downloaded, and an averaged access exceeds 10k pages per month.
• Workflow system is under construction with a GUI.
• A testbed for federation of computing resources was constructed between NAOJ and KEK by means of the NaReGi middleware. Further tests would be needed. 42