NorduGridNorduGrid andand relatedrelated Projects:Projects: OverviewOverview forfor IHEPCCCIHEPCCC
Farid Ould-Saada University of Oslo NorduGrid and ATLAS A research collaboration that develops and supports Advanced Resource Connector middelware and coordinates the various inputs to the ARC code Provides middleware to research groups, national Grid projects: ARC on 50 sites in 14 countries NorduGrid: yesterday, today, tomorrow
From ...... To
EDG >ARC 4Nordic >14 countries Tesbed >50 sites 20cpu’s >5000 cpu’s ... fromHEP a research project to > + Bio,Chem.,... 2001 >2003 ... from a Grid testbed to May 01 Oct 02 Apr 03 06/06 ... It is NOT an infrastructure, NDGF Nordic–does Council not of operate ministers or Naturalcontrola Science research resources Collaborationcollaboration of Nordic NorduGrid (Nordunet 2 program) Researcha major Councils middlewHEP institutes+ others (NOS-N) NGN network Nordunet3 ... Now several related projects KnowARC 05/10/2006 www.nordugrid.orgare provider 3
Æ TheThe bigbig picturepicture aroundaround NorduGridNorduGrid
Researh Projects N e Group tw o rk s N3 N G N Partner ARC middleware Institute Home Coder
KnowARC NDGF
05/10/2006 www.nordugrid.org 4 WhyWhy furtherfurther developdevelop ARC?ARC?
• Lightweight standalone client package, easy to install and use • Reliable resource for scientific applications in many research fields • Available on a wide range of Linux platforms
• Non-centralised architecture
ARC middleware architecture • Needs no centralized operations infrastructure • Non-intrusive, coexists with other softwares and configurations
See Top 10 Reasons to Use ARC for more details
Æ Suits needs of heterogeneous distributed shared resources Æ Is in many aspects interoperable with other middlewares
05/10/2006 www.nordugrid.org 5 HEP vs Non HEP: Example of applications run simultaneously on NorduGrid Although HEP initiated, Non HEP applications dominate
• Applications "gridified" with NorduGrid include various non HEP examples • biomedical sciences (e.g. genome research), • geophysical science (climate research), • material science (solid state and quantum chemistry research) • space science (interferometric studies) , • VCR (VideoRecorder) , … http://www.nordugrid.org/applications/appdb.html
• Runtime Environment Registry http://www.csc.fi/grid/rer/
• Non HEP applications dominate even during CERN ATLAS data challenges • See the NorduGrid online monitor for more details http://www.nordugrid.org/monitor
Æ Medical Imaging Library (St-Petersburg-NorduGrid) Æ Possible participation in ENLIGHT++ Hadron therapy program Æ within the Nordic Grid Neighbourhood
05/10/2006 www.nordugrid.org 7 Fourth Nordic Grid Neighbourhood Workshop
St-Pettersburg, May 20, 2005 F. Ould-Saada, Oslo University
● Info, duties, goals
➢Partly devoted to
➢Medical Imaging Library
05/10/2006 www.nordugrid.org 8 Medical Imaging Library
Unified MedicalPDS PDS Center 2 Dicom Acess Unified Interface Dicom Local Acess HDD Interface Local HDD Metadata Digital image Catalog Processing
UDAIUDAIUDAI PDS Computer Aided Fast LAN Diagnostic
UDAIUDAIUDAI PDS Public DICOM Storage First Nordugrid experience with Medical Library
Lund Sweden St. Petersburg Metadata Russia Catalog Public Public DICOM DICOM Storage Storage
05/10/2006 www.nordugrid.org 10 Nordic Grid Neighbourhood NGN Year 2, 3? DENMARK: University of Copenhagen (NBI, DIKU) RUSSIA: Southern Danish University, Dep. of Computer Science St. Petersburg State University, Dep. of Physics Aalborg University, Dep. of Computer Science Petrodvorets Telecommunication Center ESTONIA: Dubna? National Institute of Chemical Physics and Biophysics (NICPB) SWEDEN: Estonian Educational and Research Network (EENet) Lund University, Department of Physics Tartu University, Institute of Technology Linköping University, National Supercomputer Center FINLAND: Umeå University, Department of Computing Science and University of Helsinki, Helsinki Institute of Physics HPC2N LITHUANIA: Uppsala University, Department of Radiation Sciences Vilnius University, Institute of Materials Science and Applied Research Royal Institute of Technology, NADA/PDC, KTH NORWAY: Stockholm University and KTH, Department of Physics University of Oslo, Physics institute LATVIA University of Bergen, Department of Physics ICELAND University of Tromsø, Department of Computer Science NTNU, Trondheim, Department of Computer and Information Science ... 9 countries, 24 institutions NTNU, Trondheim, IT division (ITEA) Objectives of Nordic Grid Neighbourhood- NGN
➢ Support and strengthen contacts and communication between Nordic and Baltic States and Northwest Russia in field of Grid technologies.
➢ Network activities cover education, reciprocal knowledge transfer and research and development.
➢ Aim at optimising the exchange between a broad spectrum of computer scientists, researchers and students by organizing conferences, seminars, tutorials and up-to- date courses in Grid-computing.
➢ Support the continuation of development of NorduGrid ARC middleware, with inputs from new inputs
➢New functionalities, Scalability testing
➢New fields of Application (medicine, biology, ...)
➢Interoperability (ARC vs gLite, LCG, …) 05/10/2006 www.nordugrid.org 13 CHEP 2006 Article CHEP 2006 Poster Some +contributions of the NGN network
➢Various Grid Applications (not only HEP!)
➢St-Petersburg and Medical Imaging Library, ...
➢Plans for participation in ENLIGHT++ Hadron therapy program:
➢St-Pettersburg and possible NorduGrid involvement
➢Welcome people involved in development and deployment of various Grid middlewares
➢Invited speakers from gLiTe, LCG, Condor, ...
➢ LCG and gLiTe users
➢ ARC-LCG interoperability task force
➢This network does not compete with any EU (or other) initiatives!
➢Baltic Grid EU application written during the NGN workshop in Tallin!
➢KnowARC EU application finalised at the NGN conference in Oslo! 05/10/2006 www.nordugrid.org 14 GoalsGoals ofof KnowARCKnowARC
The mission of KnowARC is to create a novel, powerful Next Generation Grid middleware – extend and re-design ARC
Industry, academia – ARC in major Linux distributions Existing ARC middleware
Experienced – New platforms: Windows, Solaris, Mac OS-X middleware application developers developers to promote Grid standardization and interoperability – interoperability with other Grid solutions (EGEE and OGF compliant grids) – possibilities to include (or to be included in) other infrastructures. to prove usage in Health care, Industry and Science – Bioinformatics: Autoimmune Diseases – Medical Informatics: Lung Diseases – Automotive industry
05/10/2006 www.nordugrid.org 15 OfficialOfficial ProjectProject informationinformation
Title of Contract: Grid-enabled Know-how Sharing Technology Based on ARC Services and Open Standards Acronym: KnowARC Contract Nber: 032691 Instrument: STP - Specific Targeted Research Projects (aka STREP) Thematic priority/domain: Information Society Technologies (IST) Call title: IST Call 5 Call identifier: FP6-2005-IST-5 Activity: IST-2005-2.5.4 - Advanced Grid Technologies, Systems and Services Program: FP6 Duration: 36 month Start date: June 1, 2006 Community Contribution: EUR 2 899 494:- Resources: 17.5 FTEs in total, 12.5 financed Coordinator: University of Oslo Contact: Prof. F. [email protected] , www.knowarc.eu
05/10/2006 www.nordugrid.org 16 1010 partnerspartners fromfrom 77 countriescountries
NorduGrid members and research teams in medicine, bioinformatics, physics, engineering, automotive industry apps, IT
Oslo University Norway Lund University Sweden Copenhagen University Denmark Uppsala University Sweden NIIF Hungary Geneva Uni. Hospitals Switzerland Kosice University Slovakia Lübeck University Germany science+computing ag Germany SUN Microsystems Hungary
05/10/2006 www.nordugrid.org 17 how sharing services” FromFrom ARCARC toto KnowARCKnowARC servicesservices
WP1 “Grid Foundations” creates the new service oriented structure from the current NorduGrid/ARC middleware and WP2 “know-how sharing services” adds higher level “know- to the core services. WP3 “Interoperability and highway to standards and policies” enables integration with other middlewares such as gLite of EGEEassurance” and other OGF compliant Grids WP4 “KnowARC Solutions for Society and Businesses” utilises the know-how services and adds application domain solutions. The core services are tested, certified and packaged into easily installed distributions in WP5 “Software distribution, Quality
WP6 ensures “dissemination, exploitation and outreach” activities. During the entire project WP7 takes care of the project “management”.
05/10/2006 www.nordugrid.org 18 Workflow in KnowARC and graphical presentation of Work Packages
05/10/2006 www.nordugrid.org 19 KnowARCKnowARC developmentdevelopment planplan
Core Services – Next generation Grid Standards & Interoperability middleware architecture survey –OGSA and design – gLite gateway – Web Service interfaces over ARC services Applications – Back-ends – Automotive industry – Sandboxing & virtualization – Medical image processing – Security framework (delegation) – Statistical genomics – Engineering portal Higher level services (construction industry) – Self-healing flexible storage and user-friendly storage Quality assurance, software interface distribution – Self healing grid jobs: job – Build system, testing, support migration & job manager – Profiling, performance analysis, – P2P-like information backbone, usability studies novel brokering approaches – Pilot Grid system – Scalable accounting service –ARC in major Linux – Dynamic application framework distributions management – New platforms: Windows, – ARC-enabled Taverna and Solaris, Mac OS-X flowGuide (workflow engines) 05/10/2006 www.nordugrid.org 20 NordicNordic DataGridDataGrid FacilityFacility
Goal: create a Nordic Grid infrastructure, • primarily for LHC Grid computing (Tier1) NOS-N DK SF N S – 2003-2005: pilot project – NorduGrid/ARC middleware chosen as the basis Nordic Data Grid Facility June 1st 2006: NDGF 2006-2010 is launched – Funded (2 MEUR/year) by Nordic National Research Councils (NOS-N) – Nordic production Grid with Common security framework – Builds on a history and competence of Nordic Grid collaboration – Co-ordinates & hosts major Grid projects (e.g. the Nordic LHC Tier-1) – Operates a Nordic storage facility for major scientific projects – Develops Grid middleware (ARC contributor with 6- ) – Single Point of Entry for international collaborations (LCG, ...) – Represents the Nordic Grid community internationally NDGF coordinates activities - does not own resources7 middleware or middleware positions – Tier1 computing and storage resources provided by Nordic HEP community – Middleware provided NorduGrid • Interoperability with gLiTe – NDGF Partnerships • NORDUnet, National Grid Projects, National computing resources, NorduGrid
05/10/2006 www.nordugrid.org 21 NordicNordic DataGridDataGrid FacilityFacility
Distributed Nordic Tier1
05/10/2006 www.nordugrid.org 22 Nordunet3 Internet Research Programme
•Innovative services and tools for NorduGrid. PI: Farid Ould-Saada, University of Oslo.
• 4-year Grant •Oslo, Lund, Uppsala, Copenhagen, Helsinki, Reykjavik •Mainly: • Data Management: storage, indexing and movement • Education and expert training in Grid computing
Æ 5 PhD grants starting October 2006 http://www.nordunet3.org/ Æ Grid school
Æ call for collaboration with Baltic and Russia
05/10/2006 www.nordugrid.org 23 Nordic Grid activities strengthen the WLCG CERN Courier 46/7 Nordic countries have participated in Grid developments at CERN since the beginning, although at the early stages they already opted for developing their own middleware. Most Nordic Grid projects now use the Advanced Resource Connector (ARC), developed by NorduGrid, while some also use the gLite middleware deployed by the Enabling Grids for E-sciencE (EGEE) project. Several sites are involved both in NorduGrid and in projects such as EGEE, and therefore deploy both ARC and gLite. KnowARCKnowARC sessionsession atat EGEEEGEE’’0606
Tuesday, 26 September 2006 – This session provides an introduction to the KnowARC project and discuss issues of interest to KnowARC and EGEE, including interoperability and diverse applications. The KnowARC Project Status of the ARC middleware Investigating interoperability in production grid middleware solutions Status of ARC - gLiTe interoperability KnowARC applications in medicine, bioinformatics and industry
05/10/2006 www.nordugrid.org 25 KnowARCKnowARC sessionsession atat EGEEEGEE’’0606
To conclude I hope I clarified somewhat the situation around NorduGrid and related projects – There have been 3 new projects that started this summer. • Funding from both EU and Nordic states – Collaboration with WLCG, EGEE, Globus, and several other project are becoming much clearer • There is still lot to do from both sides – Further developments of ARC include several key issues: • Common standards and Interoperability with other middlewares / interoperation with other infrastructures • Applications in HEP, other sciences and industry • We are in OGF, GIN, ... All these activities are meant to be complementary to other Grid activities. Thank you for your attention!
05/10/2006 www.nordugrid.org 26