Hsin-Yen CHEN ASGC GDB @ Taipei 3 Apr. 2019
Total Page:16
File Type:pdf, Size:1020Kb
Development of the LHCONE in Asia Hsin-Yen CHEN ASGC GDB @ Taipei 3 Apr. 2019 After WWW, CERN decided in 2000 to develop and deploy distributed computing to support a new generation of big data driven research. Middleware had been developed for geographically distributed cloud centers to share resources with applications to particle physics research over the past 15 years. ASGC was established to join the international collaboration centered at CERN to build a platform for distributed computing to support e-Science. By collaborating with CERN, we stay in the forefront of distributed computing. WWW was Invented at CERN 5 SEPTEMBER 2008 VOL 321 SCIENCE Building Advanced Distributed Computing Platform Strategy: Collaborations with Research Groups - Applications Drive & Technology Push R&D: DiCOS, System Efficiency & Machine Learning Applications Application (Science) • Deeper Understanding Natural Disaster • ATLAS, CMS (High Energy Physics) • Soundscape Monitoring Network • Alpha Magnetic Spectrometer (Particle & Astronomy) • Earth Science • KAGRA, VIRGO (Gravitation) • Cryo-EM (2017) • TEXONO (Neutrino) • Computational Biology (2017) • World Wide Grid Computing (CERN) • Advanced Networking (iCAIR) • Bioinformatics (U. Chicago) • Proton Therapy (NCU, CGU/CGMH) DiCOS: Distributed Cloud Operating System • Leveraging WLCG technologies and moving from petabyte-scale towards Exabyte scale: • ~100PB in 2018 and expected to have ~1000PB in 2025; • Mobilizing 10PB/month • 2M+ jobs/day • Extending beyond high energy physics • Growing our own R&D Capability 3 ASGC Global Network Sep. 2013 ~ Sep. 2018 10G Amsterdam Chicago 10G Geneva Tokyo Palo Alto New York 1G 10G Hong LA Kong 10G TAIWAN 10G ASGCNet ASGC Global Network 1 Sep. 2018 Amsterdam 10G Geneva 10G 2x10G Chicago 10G TAIWAN 10G 10G 10G Singapore TAIWAN Global R&E Network Amsterdam Chicago 2x10G Geneva Tokyo 1G 2.5G 10G Palo Alto 2.5G 10G New York 2.5G 1G Hong 10G *2 10G LA Kong 2*2.5G TAIWAN 2x10G Singapore 2x10G ASGCnet (AS 24167) ASNet (AS 9264) TWAREN (AS 7539) TANet (AS 1659) ASGC e-Science Global Network TANet Internet2 ESNet LHCONE CANARIE LHCONE FNAL 100G NASA LHCONE TWAREN SINET ASNet LHCONE STARLight LHCONE iCAIR U. Chi 10G 10G 20G JGN/NICT 10G APAN-JP 10G LHCONE TW US NORDUNET LHCONE AARNet/AU 10G CERNET/CN 10G 20G 20G CERNLight CSTNET/CN 10G LHCONE ERNET/IN 10G RU KIAE NKN/IN 20G LHCONE HARENET/HK SG NL 20G INHEREN/ID CERN KREONET/KR 10G KOREN/KR LEARN/LX 10G 1G 10G GEANT MYREN/MY LHCONE NREN/NP PERN/PK TEIN AMSIX UNINet/TH SARA VinaRen/VN LHCONE SingaREN/SG SURFNet LHCOPN Private network connecting Tier0 and Tier1s - Dedicated to LHC data transfers and analysis - Secured: only declared IP prefixes can exchange traffic - Advanced routing: communities for traffic engineering, load balancing. T1 T1 T1 T0 T1 T1 8 Numbers - 14 Tier1s + 1 Tier0 - 12 countries in 3 continents - Dual stack IPv4- IPv6 - 660 Gbps to the Tier0 - Moved ~160 PB in the last year https://twiki.cern.ch/twiki/bin/view/LHCOPN/OverallNetworkMaps 9 HL-LHC plan Here today 10 LHCONE VRF: A global high performance Science DMZ network infrastructure for High Energy Physics data analysis (LHC, Belle II, Pierre Auger Observatory, NOvA, XENON, DUNE) Bold black lines in the following plot are 100Gb/s backbone of LHCONE to support the ExaBytes science data transfer KIAE PIONIER CSTNet China CANARIE Poland PSNC CERNet2 China JANET SURFsara Cyfronet AGH Kraków, Europe China Next IHEP, Beijing Asia America Canada MOXY Nikhef U.Warsaw ICM Generation CMS ATLAS SINET+ SimFraU TRIUMF-T1 Montreal UK ESnet (Montreal) NORDUnet Internet, IC Netherlands Internet Uvic, Utor, UBC, McGill NL-T1 Nordic KIAE/JINR Exchange, KREONet2 GÉANT, NDGF-T1, MAN LAN, Helsinki Moscow INR Beijing NDGF-T1b Russia CNGI-6IX NDGF-T1c (Troitsk) ICEPP RRC-KI T1 HEPNET U Tokyo SINET JINR T1 JINR, IHEP Protvino, GridPNPI, Japan Sarov KISTI RCC-KI T2, ITEP Hirosh., Tsukuba, KEK T1 Tokyo SANET KREONet2 Tokyo DFN DESY Slovakia Korea Germany URAN FMPh- KISTI –T1 PNU Tokyo Global Research Platform Network (GRPnet) KIT DE-KIT-T1 UNIBA xx CERN SeatleCANARIE KNU, KCMS Osaka Ukraine IEPSAS- to 2 RWTH, Wupp.U, Kharkov-KIPT Tokyo PNWG, Kosice KREONet Starlight CANARIE GSI KREONet2, PacWave / SINET, (Chicago) ESnet, CERN, MREN, CANARIE, PNWG 2 ESnet Internet2ASGC , GÉANT, ESnet (same VRF as CSTNet Osaka (Seattle) GARR, CANARIE, ANL, KREONet KRLight ESnet Europe) CESNET GÉANT to ESnet USA FNAL, KREONet2, CERN, ht Orient+) Daejeon via PacificWave, GRPnet, NORDUnet, GÉANT, Czechia to BNL-T1 BNL-T1, FNAL-T1 Chicago (London via (London Seattle, CANARIE ASGC, KIAE/RU, CSTNET, GÉANT, Caltech, prague_cesnet_lcg2, CANARIE ESNET NORDUn NetherLig (Amsterdam) Europe SINET PNNL, SLAC, ANL, ORNL CUDI NL-T1, U. Mich, FZU/praguelcg2 et , , LA AmsterdamSURFsara, ESnet, Hamburg PSNC, KREONet2, Poznan (New (New York) ESnet, ESnet, KIAE/JINR Internet2, MAN MAN KREONet2 LAN CANARIE via via CANARIE Internet2, CERN, PacWave Amsterdam NORDUnet MANLAN JGN MIT, UChi, UFla, to CERN ASGC Frankfurt UOak, Vand, Prague Orient+ via GÉANT to to GÉANT Japan GÉANT (all of Paris Beijing Beijing JGNHong NRENS and sites) Vienna PNWG GEANT Kong and Brazilian sites), London exchange) (distributed (distributed HK PacificWave ESnet, NORDUnet, New York Budapest PacificWave/ KIAE/RU (HEPNET, GÉANT Hong Kong UWisc Geneva London HKIX GRIDPNPI, IHEP), to Brussels ESnet CANARIE Internet2 - Singapore Hong Kong AGLT2 Harvard to GÉANT TEIN MIT TEIN U Chicago, DePaul U, SINET Bucharest TEINMumbai PacWave UM Lisbon Illinois Century Network (ICN), NET2 (Sunnyvale) P AGLT2 Illinois State U, UIUC, Loyola U, BU Madrid Milan BTAA NCSA, Northwestern, (Chicago) Internet2, SINET, MSU OmniPo ASGC Southern Ill U, U Minn., Singapore RedIRI Caltech, UCSD, U Illinois Chicago (UIC) S Taiwan AARNet, UCSD UT WiscNet, U Michigan (UMich), India NKN, to Arlington JGN ASGC-T1 Michigan State U (MSU) GÉANT, TIFR ASGC CANARIE, ASGC2, NCU, UNL UIC NORDUnet, GARR PERN NTU UCSB UChi ThaiREN (MWT2) SINET GÉANT CERNLight ESnet Indiana Korea Pakistan Thailand Open NCP-LCG2 SingAREN MyREN UCSD UIUC PacWave (MWT2) USA London CERN PK-CIIT Singapore Malaysia IU(MWT2), ND, Los Angeles Purdue ESnet Geneva London CERN-T0 SINET, Caltech WIX Geneva Internet2 CERN-T1 CANARIE, Internet2 (Washington) Amsterdam SG ESnet ESnet Internet2 AARNet NKN NKN USA CERN India India Australia Vanderbilt, UFlorida, SOX AtlanticWave (distributed to Los Angeles Internet 2 exchange) CERN UMel NE, SoW, Harvard VECC, TIFR RENATER Belnet France CC-IN2P3-T1 LPNHE ARNES AMPATH Belgium LPC, APC, NAP of Americas BEgrid-ULB-VUB, Slovenia CPPM, LAL SiGNET xx (Miami) BegridNumbers-UCL, IIHE RoEduNet Ver. 4.31, Oct. 25, 2018 – WEJohnston, ESnet, wej@es.net IPHC, IPNHO LLR, Romania GARR CANARIE NREN/site router at exchange point NOTES Subatech LHCONE VRF domain/aggregator CEA- UAIC xx, ISS, Communication links: 1) LHCOPN paths are not shown on this diagram - 20 R&E networks IRFU NIHAM, ANSP Network provider 1/10, 20/30/40, and 100Gb/s 2) The “LHCONE peerings” at the exchange points CUDI FCCN ITIM, NIPNEx3 indicate who has a presence there and not that GARR London PoP router Connection internal to a domain, Mexico Portugal RedIRIS all peer with each other (see PTTA - 14 Tier1s and ~70 Tier2s in 5 and of unspecified bandwidth UNAM Spain CNAF-T1 Italy Collaborating sites not yet https://twiki.cern.ch/twiki/bin/view/LHCONE/Lhc CUDI JGN – NAP do Brasil Underlined link information OneVRF ) RNP/IPĔ continents PIC-T1 INFN GRNET connected to LHCONE SingAREN/NSCC (São Paulo) indicates link provider, not use 3) See http://lhcone.net for more detail. Brazil CIEMAT-LCG2, INFN Bari, Catania, Greece Exchange point/regional R&E communication nexus Frascati, Legnaro, UAM-LCG2 Pisa, Napoli HEPLAB xx Redecomep - ~250 perfSONAR instances Milano, Roma1, w/ switch providing VLAN connections Ioannina ANSP Torino PNU LHC ALICE or LHCb site Sites that are REUNA SAMPA CBPF HEPGrid UNL standalone VRFs SPRACE CNAF-T1 LHC Tier 1 ATLAS and CMS Chile (USP) (UERJ) - 6 collaborations - yellow outline indicates LHC+Belle II site UChi LHC Tier 2/3 ATLAS and CMS LPNHE KEK Belle II Tier 1/2 } - Dashed outline indicates distributed site CEA- LHCONE: A global infrastructure for the LHC Tier1 data center and Tier 2/3 analysis center connectivity LHCONE workshop @ APAN 38 (2014) SimFraU NDGF-T1 SCINET UVic UAlb CANARIE (UTor) NDGF-T1b NDGF-T1c NL-T1 NORDUnet GLORIAD TRIUMF-T1 Canada McGill (global) SARA Nordic KurchatovT1 Netherlands ICEPP Korea U Tokyo CERN-T1 SINET CERN MREN KISTI Japan (Chicago) Chicago Amsterdam (CERNLight) Korea (StarLight) (NetherLight) Geneva TIFR India Geneva CIC Indiana KNU AGLT2 (3) OmniPoP GigaPoP DESY KERONET2 UIUC (Chicago) RWTH Wup.U DE-KIT-T1 Korea MWT2 SLAC GSI DFN ESnet FNAL-T1 New York Germany CERN USA (MAN LAN) Seattle BNL-T1 India (PNWG) ASGC-T1+T2 CENIC GÉANT ASGC2 USA Caltech ASGC Europe Taiwan AGLT2 IU MSU MIT Los UNL AGLT2 AGLT2 UIUC Angeles UM WSU MWT2 NCU NTU CC-IN2P3-T1 UCSD IN2P3 CEA TWAREN Washington UWisc NE (10 sites) RENATER (IRFU) Taiwan UFlorida (WIX) PurU GLakes SoW France Vandebilt UNeb Internet2 Harvard USA PIC-T1 INFN CNAF-T1 RedIRIS (9 sites) GARR Spain Italy UNAM CUDI End sites – LHC Tier 2/3 LHCONE VRF domain NTU unless indicated as Tier 1 Mexico LHCONE VRF aggregrator networks UNL Sites that are standalone VRFs Chicago Regional R&E communication nexus Communication links, 10, 20, 30, and 100Gb/s 12 August 2014 See http://lhcone.net for details. 12 Progresses made in Asia since 2015 Asia Asia-Pacific VRFs 2017 Internet2 ESnet CANARIE US AS11537 US AS293 CA AARnet