CernVM - a virtual software appliance for LHC applications C. Aguado-Sanchez 1) , P. Buncic 1) , L. Franco 1) , A. Harutyunyan2), P. Mato 1), Y. Yao 3) 1) CERN, Geneva, 2) Yerevan Physics Institute, Yerevan, 3) LBNL, Berkeley Predrag Buncic (CERN/PH-SFT) CernVM Project • Talk Outline CernVM Project • Building blocks • User Interface and API • Scalability and performance Fu ture plans & directions Release status Conclusions Links CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/2009 - 2 CernVM Project • Portable Analysis Environment using Virtualization Technology (WP9) Approved in 2007 (2+2 years) as R&D activity in CERN/PH Department Start ed J anuary 2008 Sister project to Multicore R&D • Project goals: Provide a complete, portable and easy to configure user environment for developing and running LHC data analysis locally and on the Grid independent of physical software and hardware platform (Linux, Windows, MacOS) Decouple application lifecycle from evolution of system infrastructure Reduce effort to install, maintain and keep up to date the experiment software Lower the cost of software development by reducing the number of compiler-platform combinations CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/2009 - 3 • A complete Data Analysis environment available for each experiment Code check-out, edition, compilation, local small test, debugging, … Grid submission, data access… Event displays, interactive data analysis, … • No software installation required • Su spend/resume capabilit y [email protected]/9/09 4 CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/2009 - 4 Key Building Blocks • rBulder from rPath (www.rpath.org) A tool to build VM images for various virtualization platforms • rPath Linux 1 Slim Linux OS binary compatible with RH/SLC4 • rAA -rPthLiPath Linux App liance Agen t Build types Web user interface XMLRPC API Installable CD/DVD Stub Image • Can be f ull y customiz ed an d ex ten ded by Raw Filesystem Image means of plugins (#401) Netboot Image Compressed Tar File Demo CD/DVD (Live CD/DVD) Raw Hard Disk Image • CVMFS - CernVM file system Vmware ® Virtual Appliance Read only file system optimized for software Vmware ® ESX Server Virtual Appliance distribution Microsoft ® VHD Virtual Apliance • Aggressive caching Xen Enterprise Virtual Appliance Virtual Iron Virtual Appliance Operational in offline mode Parallels Virtual Appliance • For as long as you stay within the cache Amazon Machine Image Update CD/DVD Appliance Installable ISO • CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/2009 - 5 Conary Package Manager class Root(CPackageRecipe): name='root' version='5.19.02' buildRequires = [ 'libpng:devel ', 'libpng:devellib','krb5:devel', 'libstdc++:devel’,'libxml2:devel', 'openssl:devel','python:devel', 'xorg-x11:devel', 'zlib:devel', 'perl:devel', 'perl:runtime'] def setup(r): r.addArchive('ftp://root.cern.ch/root/%(name)s v%(version)s.source.tar.gz’) r.Environment((,())'ROOTSYS',%(builddir)s') r.ManualConfigure('--prefix=/opt/root ') r.Make() r.MakeInstall() CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/2009 - 6 CernVM Variations group-<experiment>-desktop (lightweight X environment) group-<experiment> (grou ps and ex tra packages group-cernvm-desktop required by experiment) X11 group-cernvm-devel group-slc4 (SLC4 compatibility libs) (development tools) (SLC4 compatibility libs) compat-db4 compat-openssl compat-libstdc++slc3 compat-libxml2 compat-readline compat-tcl group-cernvm compat-tk 100 MB (core packages) CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/2009 - 7 As easy as 1,2,3 imppport xmlrpclib import os url=‘http://user:password@host:8004/rAA/xmlrpc’ server = xmlrpclib.ServerProxy(url) r = server.cernvm.Config.configGridUIVersion(”3.1.22-0") 1. Login to Web interface 2. Create user account 3. Select experiment, appliance flavor and preferences CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/2009 - 8 CVMFS • CernVM File System (CVMFS) is derived from Parrot (http://www.cctools .org )and) and its GROW-FS plugin code base and adapted to run as a FUSE kernel module adding extra features like: possibility to use multiple file catalogues on the server side ttfilidiithhldtransparent file compression under given size threshold dynamical expansion of environment variables embedded in symbolic links CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/2009 - 9 Scalable Infrastructure CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/2009 - 10 Publishing Releases 1. Each experiment is given a VM to install and test their software using own 2. Publishing is an atomic installation tools operation Current deployment model Primary (master) software repository Load Secondary software Secondary software Reverse Proxy Balancing repository repository Reverse Proxy Load Balancing Load RPReverse Proxy Balancing Regional Reverse Proxy Reverse Proxy CernVM (()thin client) Reverse Proxy Site Reverse Proxy The aim is to reduce latency which is the most important issue for distributed network file systems [email protected]/9/09 12 CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/2009 - 12 Benchmark setup ROOT (version 5.22, 100Mb) • The benchmark runs in Xen VM and includes compilation and execution of ROOT stressHepix suite The system has to find and fetch binaries, libraries and headers from the network file system but, once loaded, the libraries remain in a local cache • We compare the performance of CVMFS to AFS by artificially introducing network latency and limiting bandwith (using tc tool) CVMFS vs AFS (Δ ) • These plots show the time penalty ttt=tAFS,CVMFS - tlocal resulting from having the application binaries, search paths, and include paths reside on a network file system, Additional performance loss due to virtualization (not shown here) is consistently <5% • CVMFS shows consistently better performance than AFS in case of ‘cold cache’ ‘Worm cache’ performance is in case of AFS and CVMFS equal to local file access Removing single point of failure (1) HTTP server Slave servers could be deployed on Proxy strategic locations to reduce latency Server and provide redundancy HTTP server CernVM HTTP server Proxy Proxy Server CernVM Server Proxy Server HTTP server HTTP CernVM server Proxy Server HTTP server Removing single point of failure (2) CernVM HTTP server CernVM Content Proxy Distributio Server n Network HTTP server CernVM LAN WAN Use P2P like mechanism for discovery Use existing Content Delivery Networks to of nearby CernVMs and cache sharing remove single point of failure between them. No need to manually Amazon CloudFront (http://aws.amazon.com/cloudfront/ setup proxy servers (but they could still Coral CDN (http://www.coralcdn.org) be used where exist) Bridging Grids & Clouds • BOINC Open-source software for volunteer computing and grid computing http://boinc.berkeley.edu/ • CernVM CoPilot development Base d on BOINC, LHC@HOME exper ience an d Cern VM image Image size is of outmost importance to motivate volonteers Can be easily adapted to Pilot Job frameworks (AliEn,Dirac, Panda) • … or Condor Worker,,p or proofd.. Aims to demonstrate running of ATLAS simulation using BOINC infrastructure and PanDa BOINC LHC@HOM E PanDA Pilot CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/2009 - 17 CernVM CoPilot AliEn/DIRAC/PanDA Adapter 0. Send host JDL 1. Append framework (free dis k space, free memory, specific information and available packages) request a job 3. Send input files and 2. Send user job JDL from Task commands for execution Queue (packages are already there) 4. When the job is done 5. Register output files send back the output files (and the result of validation) CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/2009 - 18 Simplifying Grid deployment • Nimbus (former Globus Workspace Service) Nimbus is a set of open source tools that together provide an "In frastructure-as-a-Si"(IS)ldService" (IaaS) cloud comput ing so lilution • http://workspace.globus.org/ Google Summer School (hosted at ANL) project to deploy a one-click, auto-configuring virtual Grid overlay for Alice/AliEn • Successfully created virtual AliEn site for ALICE with one command • See CHEP’09 contribution by Artem Harutyunyan (poster #214) CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/2009 - 19 1.2.0 Release now available • Available now for download from http://rbuilder.cern.ch/project/cernvm/releases • CbCan be run on Linux (KVM, Xen,VMware Player, VirtualBox) Windows(WMware Player, VirtualBox) Mac (Fusion, Parallels, VirtualBox) • Release Notes http://cernvm.web.cern.ch/cernvm/index.cgi?page=ReleaseNotes • Appliance can be configured and used with ALICE, LHCb, ATLAS (and CMS) software frameworks ALICE (AliEn 2.15,2.16, AliRoot v4-14-Rev-04, v4-15-Rev-03, v4-16-Rev-01) ATLAS (14.2.20, 14.2.23, 14.2.24, 14.4.0, 14.5.0, 14.5.1) CMS (CMSSW_2_2_5, CMSSW_2_2_6 ) LHCB (GAUDI_v19r8, GAUDI_v19r9, GAUDI_v20r0, GAUDI_v20r2, GAUDI_v20r3, GAUDI_v10r11, DAVINCI_v20r3, DAVINCI_v19r12, DAVINCI_v20r3,DAVINCI_v22r1) CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/2009 - 20 Summary • If you work for one of the LHC experiments and you have one or a few problems listed below then CernVM might be what you are looking for: Your expppypgyeriment software is not compatible with your hardware