Grid Computing in

OXANA SMIRNOVA, RECFA MEETING, LUND, 20 MAY 2016 Outline

• Infrastructure • Software • People • Funding Background

• Grid research for LHC in Sweden started in 2000 – Involved originally all Particle Physics groups (Lund, Stockholm and Uppsala) and all major computing centres in Sweden – Supported until 2003 by NorduNet2 through a common Nordic project: NorduGrid • Primary goal: establish a Nordic Tier-1 center (NGDF-T1) of the Worldwide LHC Computing Grid (WLCG) – Pledge support to ALICE and ATLAS • To support a distributed Tier-1, we developed a special Grid software: ARC middleware NDGF-T1: a distributed center

Scientists

Authorised users directory (VOMS)

ARC Indexing Cache Cache Cache Cache servers Copenhagen Linköping Umeå

SGAS Cache Cache Cache accounting Helsinki Bergen Lund server ARC computing services Operators Site-BDII servers

CERN

Data Data Data Data dCache Data server Data Data FTS server Data dCache storage cloud Data Data NDGF-T1 logistics

• In production since 2006 – WLCG MoU signed by 4 • Operated by the Nordic e-Infrastructure Collaboration (NeIC) – Funding from the Nordic Council of Ministers via NordForsk • Staff: ~5 FTE (funded by NeIC), external experts: ~7 FTE (funded by national research programmes) • In Sweden, contributions are coordinated via SNIC: Swedish National Infrastructure for Computing – Second largest national research infrastructure after MAX-IV – Joins 6 computing centres – Was proposed in 2001, strongly influenced our Grid decisions and approaches (ARC in particular) – Also provides a WLCG Tier-2 (in Lund) Computing and storage needs of ALICE and ATLAS

ALICE and ATLAS needs in 2016 (total • NDGF-T1 targets: Tier0+Tier1+Tier2) and Nordic – ALICE: 9% of all contribution Tier-1 needs 180,00 – ATLAS: 6% of all 160,00 Tier-1 needs 140,00 • Internally the targets are 120,00 split proportionally to the authors count 100,00 – Sweden’s total 80,00 share in NDGF-T1: ~40% 60,00 • Tier-2: no formal targets, 40,00 fixed pledge 20,00

0,00 KCPU-cores Disk, PB Tape, PB (approx.) ALICE ATLAS ALICE Nordic ATLAS Nordic NDGF-T1 performance

• We meet the targets, overall • Performance is similar to other Tier- 1s • Difficult to offer extra capacity to ATLAS, since it already uses a lot – Unlike other Tier-1s, all our resources are

shared

, using WLCG accounting data , WLCG using accounting

Flix

Josep Source: Source: ARC middleware

• Development driven by NDGF-T1 needs – Common interface to shared supercomputer-class facilities • First release: April 2004 • Current release: 15.03u7 (yesterday) – 42 releases so far • 100 000+ lines of code – Free, Open Source Apache v2.0 license • 44 past and present contributors – 11 in Sweden (4 active) • Coordinated by the NorduGrid Collaboration – Not a project, hence no budget and no end-date ARC usage: a growing trend

• ARC usage in WLCG: – In 2010: 10 sites, almost all in Nordic countries – In 2016: 134 sites around the world (~20% of WLCG) ARC worldwide (as of May 2016) Grid experts in Sweden

Researchers: Computing experts: • Lund University: • HPC2N (Umeå): – Balázs Kónya – ARC Technical – Roger Oskarsson – systems expert Coordinator – Mikael Rännar – NeIC’s LHC – Florido Paganelli – ARC developer Committee member – Oxana Smirnova – NeIC – CERN – Mattias Wadenstein – NDGF-T1 Liaison and ARC developer coordinator • • LUNARC (Lund): – Richard Brenner – NeIC’s LHC – Robert Grabowski – systems expert Committee chairman – Jonas Lindemann – LUNARC – Mattias Ellert – ARC developer director • NSC (Linköping): – Jens Larsson – systems expert What do we do

• ARC middleware development and support: – overall technical coordination, strategic planning, technology surveys – ARC information system, related client and server components – ARC build and distribution – issue tracking, documentation – policies development, standardization, e-infrastructures integration – outreach, requirement gathering, user support, training • NDGF-T1 operations – Hardware and service deployment and operations – Liaison between CERN and NDGF-T1 – Membership in the NeIC’s LHC advisory committee • SNIC Tier-2 operations • Non-Grid operations (small-scale allocations at SNIC) Funding sources

• International projects: – EU FP5: EDG (unpaid); EU FP6: KnowARC and EGEE; EU FP7: EMI and EGI » Technical Coordinator in KnowARC and EMI – Nordic: NorduGrid (2001-2003), NGN (2004-2006), NGiN (2006-2010), NDGF/NeIC (2003-ongoing) » Technical Coordinator in NorduGrid, CERN Liaison in NDGF/NeIC • National projects: – Government: eSSENCE; SNIC projects; several Research Council grants in Lund and Uppsala – Faculty (in Lund): COMPUTE; Computing for Subatomic Physics Funding peculiarities

• With very few exceptions, no project supports even a single full FTE – Most experts are co-funded through several projects • In Sweden, infrastructure funding is separate from research funding – Funds can not be re-allocated to e.g. PhD students or vice versa • Old habits die hard: many believe that software costs nothing and computers are cheap – Understanding that our facilities are a part of CERN in Sweden is still not fully there – Full appreciation of the industrial scale of the LHC computing is still missing (NDGF-T1 alone costs ~3 MEUR/year) – Understanding of LHC and WLCG software maintenance costs is missing – WLCG software development is seen as neither physics nor computer science or engineering – no suitable funding calls Vicious circle of scientific computing

Coding and operations • Familiar situation by physicists in all sciences and countries Neither No physics, nor computing computer • Plan A: give experts science academic credits for software and data • Plan B: fund No No funding computing experts publications irrespectively of credits No academic credits Summary

• Sweden is an important contributor to the WLCG – Successful operation of the NDGF-T1 – Development and support of the increasingly popular ARC middleware – Swedish Grid experts are highly respected internationally • We enjoy close and fruitful cooperation between researchers and computing resource providers in Sweden • We still struggle to obtain adequate funding – Early misconceptions of “free” software and “cheap” commodity computing did not do service to LHC