1st Global Research Platform Workshop
Key Global Application Drivers in Asia and Korea Enabling Science Discovery - New particle, “Higgs” in CERN LHC - Gravitational Wave in LIGO High Energy Physics & Fusion Energy Science - Blackhole Event Horizon in Astronomy Supermassive black hole in the heart of galaxy M87
Bio/Genome Research Astronomy & Climate & Medical Science Changes
Advanced Scientific Education & e-Culture Computing Research
Future Internet & Construction High Energy Physics LHC Optical Private Network ES-PIC CA-TRIUMF US-T1-BNL AS43115 AS36391 AS43 █ █ █ █ █
TW-ASGC US-FNAL-CMS AS24167 AS3152 █ █ █
KR-KISTI RRC-KI-T1 AS17579 AS59624 █ █ █ █ CH-CERN AS 513 █ █ █ █ UK-T1-RAL RRC-JINR-T1 AS43475 AS2875 █ █ █ █ █
NDGF AS39590 FR-CCIN2P3 AS789 █ █ █ █ █ █
NL-T1 DE-KIT IT-INFN-CNAF AS1162, 1104 AS58069 AS137 █ █ █ █ █ █ █ █ █ █ █
T0-T1 and T1-T1 traffic 10Gbps 20Gbps T1-T1 traffic only Ref: https://twiki.cern.ch/twiki/bin/view/LHCOPN/OverallNetworkMaps 30Gbps █ = Alice █ = Atlas █ = CMS █ = LHCb 40Gbps 100Gbps [email protected] 20190516
Private network connecting Tier0 and Tier1s Layer3 (routed) Virtual Private Network - 20 R&E networks - 14 Tier1s and ~70 Tier2s in 5 continents - ~250 perfSONAR instances - Collaboration with Bell II experiment, DUNE experiment Asia North America
HEPCAS, LHCONE L3VPN: A global infrastructure for High Energy Physics data analysis (LHC, Belle II, Pierre Auger Observatory, NOvA, XENON) Cyfronet IHEP Beijing PIONIER CSTNET PSNC To: CERN CNGI-CERNET CMS ATLAS CANARIE/CANET Poland China China JANET Nikhef AGH Kraków, POZMAN UVic UToronto Canada U.Warsaw China Next MOXY NL-T1 To: SINET, SimFraU UK SURFsara ICM-PUB Generation ESnet, Internet2 TRIUMF-T1 Montreal (Montreal) NORDUnet Internet, SINET IC Netherlands KIAE-TRANSIT BCNet RAL Nordic Internet SINET Peers: McGill Exchange UTNET Japan Internet2, RU-VRF, UBC NDGF-T1, (RU-VRF) Tokyo NORDUnet, To RU-VRF, Helsinki BINP IHEP To GÉANT To ICEPP Hirosh., ESnet Waterloo GÉANT, NDGF-T1b Russia Protvino U Tokyo Tsukuba, MAN LAN, NDGF-T1c Moscow JINR-T1/T2 Tokyo SarFTI INR HEPNET-J Tokyo RRC-KI T1/T2 (Troitsk) ASGC to
GÉANT,CERNET Japan NetherLight GridPNPI KEK T1 DFN
ANA Germany SINETto Amsterdam KREONet2 To ASGC, - ESnet To Taiwan 400 RWTH DE-KIT-T1 Korea TEIN, via CANARIE JGN URAN SANET KISTI –T1 PNU Global Research Platform Network (GRPnet) TO CSTNET DESY GSI Slovakia CERN Via TEIN and Ukraine FMPh-UNIBA xx to KNU, KCMS CNGI-CERNET
Kharkov-KIPT Tokyo to SINET IEPSAS-Kosice
PacWave / CANARIE To SINET, PNWG KREONet2 JGN ESnet via (Seattle) Starlight SINET Seattle Peer: ESnet (same VRF as Japan ESnet, KREONet2 (Chicago) To Tokyo USA ESnet Europe) To ASGC, KREONet2 CESNET KRLight KREONet2, Peer: ASGC, ESnet, GÉANT Internet2, CANARIE, Hong BNL-T1 ANL CANARIE NetherLight GÉANT, Czechia to Daejeon FNAL-T1 KREONet2; Kong BNL-T1 ------(Amsterdam) Europe prague_cesnet_lcg2,
GÉANT, GARR, CSTNET NORDUnet PNNL, SLAC, ORNL CANARIE To FZU/praguelcg2 (London via Orient+) via (London Peers: ASGC, MANLAN TEIN CERNLight, Hamburg ASGC NORDUnet2, Poznan GÉANT, RU- Beijing/ To: MAN LAN VRF/KIAE, SINET, Dublin Amsterdam KERONet, / NL-T1; ASGC Singapore GÉANT to Beijing
TEIN North ASGC to ASGC, (New York) ------Frankfurt TEIN SURFsara, ESnet, Prague TransPAC via Orient+ PSNC, Paris Hong Kong US/NSF/IU Internet2, CERN, ESnet, NORDUnet, Vienna PNWG Hong SINET CANARIE CERN to London Kong PacificWave To LA Singapore to PacificWave Budapest JGN UWisc Geneva KREONet HKIX exchange) (distributed Madison Peers: Brussels Hong Kong ESnet, GÉANT to TEIN
TANet2 AGLT2 Internet2 Harvard Singapore ThaiREN Taiwan UM Lisbon Bucharest Marseille Thailand NCU, NTU PacWave MIT NET2 / BU ASGC AGLT2 (North East Tier 2) UNINET, (Sunnyvale) BTAA MREN Milan (Chicago) Madrid ERX-CHULANET MSU OmniPoP (Chicago)
, CERN , Taiwan Peers: ESnet, THAISARN, ASGC-T1 UCSD,UCSD PUBNET-TH, UT ASGC to NSTDA-TH NKN, India ASGC2 Arlington Singapore
CENIC/ GÉANT, CalREN UNINET-TH, TIFR SUT-TH UIC CANARIE, GARR Amsterdam NORDUnet, To ESnet To UNL MyREN UCSB UChi SINET (MWT2) CERNLight Malaysia ESnet PERN SingAREN SINET to Purdue Indiana GÉANT TOKYO Korea GÉANT GÉANT Pakistan Singapore UCSD USA Open NCP-LCG2 UIUC IU London CERN Peers: Caltech (MWT2) ND PK-CIIT ASGC, SINET, JGN, TEIN; Geneva
ASGC toASGC ESnet To: Internet2 IUPUI ESnet, CalREN, U Fl CERN-T0 (MWT2) London ASGC to PacWave WIX Geneva Amsterdam CERN-T1 (Los Angeles) (Washington) Amsterdam To ASGC London via Singapore ASGC Taiwan GÉANT SINET,
London Peers: SINET, To MANLAN to ESnet, Internet2, Internet2 GOREX TransPAC
USA SOX (distributedexchange) ERX-ERNET NKN (U of Guam)
Peers: UOK, UFL, CANET, (Atlanta) AtlanticWave India Guam IU, PURDUE, IUPUI, India CEA LHC1-RENATER Geneva , ESnet, CALTECH, MIT, TIFR VECC UOC, NORDUNET, Saclay France, AS2091
CERN Mumbai Kolkata GÉANT, AARNet, Belnet LPNHE ARNES AMPath/ Belgium CC-IN2P3-T1 GÉANT GÉANT LPC, APC, BEgrid-ULB-VUB, Slovenia to AARNet AMLight CPPM, LAL Begrid-UCL, IIHE SiGNET xx RoEduNet NAP of Americas IPHC, IPNHO Australia DNIC CUDI (Miami) SUBATECH LLR, Romania CEA- UMel Mexico UAIC xx, ISS, Perth ESnet, Internet2 via WIX, IRFU NIHAM, UNAM AmLight RedCALARA ITIM, NIPNEx3 SouthREUNA America SAMPA CBPF (U. São Paulo) (Rio de Janeiro) Chile LHCONE Map Ver. 5.1, May 30, 2019 – WEJohnston, ESnet, [email protected] PTTA FCCN Oceania NAP do Brasil International infrastructure by provider/collaboration UTFSM (São Paulo) RNP/IPĔ / ANSP Portugal RedIRIS GARR GRNET GARR LHCONE VRF domain/aggregator CANARIE NREN/site router at exchange point Ioannina - A provider network. various NORDUnet Brazil HEPGrid Spain Italy Communication links: CNAF-T1 Greece GÉANT KIAE, Russia (UERJ) ANSP Connector network – provides, PIC-T1 e.g., an L2 path between VRFs. 1/10, 20/30/40, and 100Gb/s or N x 100G SPRACE FAPESP INFN SINET, Japan KREONet2, Korea Bari, Catania, JGN – PNU LHC ALICE or LHCb site Amparo CIEMAT-LCG2, INFN Underlined link information ASGC, Taiwan UNL Frascati, Legnaro, London Provider network PoP router SingAREN/NSCC UAM-LCG2 Pisa, Napoli indicates link provider, not use ESnet transatlantic, USA CNAF-T1 LHC Tier 1 ATLAS and CMS Sites that are Milano, Roma1, Not currently connected to standalone VRFs NOTES CUDI JGN, SingAREN (+ TIENCC, SurfNet, NORDUnet, UChi LHC Tier 2/3 ATLAS and CMS 1) ONLY links involved in LHCONE are shown Torino LHCONE NDGF-T1 Dotted outline indicates distributed site GÉANT on the London link) KEK Belle II Tier 1/2 2) LHCOPN links are not shown on this diagram ANA-400 - Various links provided by CANARIE, ESnet, 3) For map explanation see “Rationale Behind the LHCONE Map” at Europe Exchange point RAL Dashed outline indicating a WLCG federation site not currently on LHCONE GÉANT, Internet2, NORDUnet, SURFnet, SINET,IU/NSF Yellow outline indicates LHC+Belle II site https://www.dropbox.com/sh/padxfo58j0j1raz/AADsB5K8fISH9FfhCjA4eCtea?dl=0 Internet2 ESnet CANARIE US AS11537 US AS293 CA AS6509
AARnet AU AS7575
MyREN MY SINET ThaiREN JP AS2907 TH AS24475 JGN CERnet JP AS17934 CN AS23911 ASGCNet CSTnet TW AS24167 CN AS7497 TEIN INHERENT HK: AS24490 CERNlight SG: AS24490 KREONet2 CH AS20641 ID AS18007 KR AS17579
PERN PK Asia-Pacific
NKN Not Asia-Pacifc IN AS55824 GEANT LHCONE connection LHCONE mutual transit TIFR EU AS20965 Connection in place, but no LHCONE IN AS2697 Asia-Pacific VRFs – Current Status Missing connection Localize bulk data in Cloud Service
Daniele Cesini , Patrick Fuhrmann, “The Data Lake model for the scientific computing of the next decade“, 2018 - Data Organization, Management, Access (DOMA) - Storage consolidation - Cashing and data access - Data Transfer among Storage in distributed Regional Storage (DRS) or among DRSs - QoS
Europe Asia America SUT, TH Site B Gbps Site A GSDC, KR Tbps Tbps Site A GSDC, KR
Gbps Open Storage Network Site C (OSG) Aisa-Lake EU-Lake *Asia Tier Center Forum Fusion Energy : KSTAR & ITER 1) Engineering Validation and Engineering Design Activities for the International Fusion Materials Irradiation Facility (IFMIF/EVEDA:2007-2017)
2) International Fusion Energy Research Centre (IFERC: 2007-2017), project leader is N.Nakajima a) DEMO Design and R&D coordination Centre b) Computational Simulation Centre c) ITER Remote Experimentation Centre - Estimation for the ITER data amount in 2016
ITER Phase Total DAN* archive rate Plasma duration Time Data Volume per Shot
Initial 2 GB/s (16Gbps) 400 – 500 sec (6.7 – 8.3 min) 800 GB – 1 TB
Final 50 GB/s (400Gbps) 1000 s (16.7 min) 50 TB
H. Nakanishi et al, “High-Performance Data Transfer for Full Data Replication between ITER and the Remote Experimentation Centre”, 26th IAEA Fusion Energy Conference, 2016
-Total archive capacity: 90 - 220 TB/day 3) Satellite Tokamak Programme (2007-2019) Participation to upgrade of JT-60 Tokamak to JT-60SA and its exploitation. Korean Participation Center
Russian Participation Center
The ITER remote experimentation centre will be made to participate the experim ent remotely, where the experimental proposal and data collection and analyses will be done through the broadband network.
ITER Organization (Saint-Paul-les-Durance)
ITER Remote Experimentation Centre (Rokkasho, Japan)
Validation of safety of the facility and Broad band internet Operation //
Data collection and analyses Monitor of the operation, Transfer results of experiment Data storage and Analyses
Setting discharge p Proposal and settin arameter Send discharge parameter g discharge param eter by remote user JW Farthing, “Progress on ITER remote experimentation Center”, 2017 11 Astronomy
Written by colaboration with Dr. Taehyun Jung, Korea Astronomy & Space Science Institute Global VLBI Communities
North America Europe
East-Asian
Oceania • 25m x 10 radio telescopes operated by NRAO (National Radio Astronomical Observatory), US • Observing wavelength: 90cm – 3mm • Correlator: NRAO in Socorro, USA • ~ 24 radio telescopes in Europe, Asia, South Africa and 10 radio telescopes in Puerto Rico • Observing wavelength: 90cm – 7mm • Correlator: The joint Institute for VLBI ERIC (JIVE) in Dwingeloo, Netherland • ~ 20 radio telescopes in China, Japan and S. Korea • Observing wavelength: 5cm – 7mm • Correlator: KASI in Daejeon, South Korea • ~ 11 radio telescopes in Australia operated by CSIRO • Observing wavelength: 20cm – 3mm • Correlator: Pawsey Supercomputing Centre in Perth, Australia Global VLBI Communities
North America Global VLBI Alliance InitiativeEurope • “Earth VLBI alliance” : VLBA + EVN + EAVN + LBA + … • Provide simpler the global VLBI access for astronomers in order to pursue the breakthrough science by maximizing the power of VLBI in worldwide
East-Asian
Oceania Gravitational Wave Fast Radio Burst (FRB)
• VLBI imaging can help § to clarify the astrophysics § precise localizations of the origin § to Identify potential persistent radio counterparts • e.g. Transients, Neutron star mergers, Gravitational Wave, Fast Radio Burst etc. • e-VLBI is greatly important for rapid response sciences and multi- messenger astronomy • Data Intensive Science § High performance networking and computing, Big data § Large-scale instruments : telescope § Data Center and HPC involved • Event-driven (or scheduled) Rapid Response Science § Fast response to request for observation § Real-time, more robust against failure § Immediate feedback on the state of variable and transient phenomena § Noticed immediately and addressed instantaneously § Dynamically addressable to telescope, network and correlator § Real-time correlation (saving the storage cost) • Collaborative Science § Globally Multi-partners and multi-sites of telescopes § Sometime a telescope is located at an out-of-the-way place § large-scale global collaboration Data-Intensive Science • End-to-end high performance network § Large bandwidth network with 100Gbps § Science DMZ + DTN + PerfSONAR + Science Engagement § (Static) allocation of lightpaths § Virtual Private Network
Rapid Response Science Collaborative Science • Dynamically provisioning and optimizing network § Locally and internationally (inter-continent) connected § Dynamic allocation of lightpaths § Bandwidth on demand § Automated GOLE § Programmable capability • Automated operation enabled by networkers and operated by astronomer
Why not? Automated Dynamic Virtual Private Network • Global VLBI Alliance (GVA) demonstration with the most common frequency (22GHz/13mm) § Number of telescopes - VLBA (10), EVN (14), EAVN (15), LBA (5) - Total 44 radio telescopes (RT) § Data rate - 1Gbps / 2Gbps / 4Gbps / 8Gbps (Currently) - 16Gbps / 32Gbps / 64Gbps … (Future) § Realtime e-VLBI capabilities - EVN/LBA: real-time e-VLBI correlation - VLBA/EAVN: not supported yet EVN correlation center (Dwingeloo, Netherland) EAVN correlation center (Daejeon, South Korea)
LBA correlation center (Perth, Australia)
VLBA correlation center (Socorro, USA) London Amsterdam Seattle Montreal CERN Chicago Paris Daejeon New York (Korea) Washington Tokyo Hong LA Kong Miami Taipei Hawaii
Singapore
Brazil RNP Australia Realtime observation and correlation using New Zealand telescopes available all over the world
KREONet2, KOREN, Internet2/SingAREN, TransPAC, SXTransport, SINET, AMLight Exp, AGGCnet, APR(SingAREN/JGN), NEAAR, ANA(Internet2, NORDUnet, SURFnet, CANARIE, GEANT), Esnet, CAE-1, GRP Observatory 1. Observatory 2. Observatory 3.
DTN ps DTN Science DMZ Science DTN ps DMZ Science Data Center A DMZ Supercomputer (Correlator) NREN A
DTN ps N*100Gbps Science DTN ps DMZ N*100Gbps Automated GOLEs Science Data Center B DMZ (Correlator) N*100Gbps NREN C NREN B
Observatory 5. DTN ps Observatory 4. DTN ps Science DMZ DTN ps Cloud Provider Science Science DMZ DMZ § e-KVN: Korean e-VLBI(electronic Very Long Baseline Interferometry) Network
=> Toward Real-time e-VLBI observation with real-time correlation among observatories
Korea-Japan Correlation Center (KJCC) @Daejeon
100G @East Asia @Europe Yonsei Astronomy Observatory 40G (+1G) @Seoul Daejeon 100G
KREONet2
Seoul
10G(+1G) Jeju 10G(+1G) Ulsan Sejong 10G @Australia
Tamna Astronomy Observatory @ Jeju Ulsan Astronomy Observatory @Ulsan
VLBI antenna @ Sejong Space Geodetic Observation Center
Biomedical Science and Genomics ICGC-PCAWG (ICGC-TCGA Whole Genome Pan-Cancer Analysis Working Group ) : 795.14 TB Cancer Cell Line Encylopedia : 30TB
Cancer Proteogenomics Project
1000 Genome Project: 200TB/ European Genome-Phenome Archive (EGA)Biomedical Science Community, KR TCGA (Cancer Genome Altas) :1.05PB/ Sequence Read Archive CPTAC KREONET/KREONet2
KOREA RESEARCH ENVIRONMENT OPEN NETWORK
10G Seattle Chuncheon 20G Chicago 20G Gangneung 100G Seoul Incheon 20G 20G 100G Suwon 300G United States of America
Cheonan 100G 20G Sejong Ochang 10G 30G
40G LHCOPN 10G Internet eXchange – up to 10G IX 40G Daejeon
20G 100G Pohang 100G Amsterdam Daegu 30G Jeonju CERN 100G 30G Ulsan Hong Kong 100G Changwon Europe Busan Gwangju China
40G - National-wide 100Gbps National Research Network 10G KREONET POP - 17 Domestic GigaPoPs, 4 International GigaPoPs 100G KRLIGHT POP - 100G upgrade for international network: Seattle, Hong Kong - New GigaPoP in Korea: Chuncheon - User: about 200 national research institutes, universities etc. Jeju (500,000 users) Europe
▷ (Swiss) CERN LHC ▷ (France) ITER Grid ▷ JIVE: European VLBI Network ▷ (Norway) Satellie Data Receiver (MoU) SURFnet ▷ (UK) SKA (Square Kilometre