Desktop Grids for Escience

Produced by the IDGF-SP project for the International Desktop Grid Federation

A Road Map Desktop Grids for eScience

Technical part – December 2013 IDGF/IDGF-SP International Desktop Grid federation http://desktopgridfederation.org

Edited by Ad Emmen Leslie Versweyveld

Contributions Robert Lovas Bernhard Schott Erika Swiderski Peter Hannape

Graphics are produced by the projects. version 4.2 2013-12-27

IDGF-SP is supported by the FP7 Capacities Programme under contract nr RI-312297.

Copyright (c) 2013. Members of IDGF-SP consortium, see http://degisco.eu/partners for details on the copyright holders. You are permitted to copy and distribute verbatim copies of this document containing this copyright notice but modifying this document is not allowed. You are permitted to copy this document in whole or in part into other documents if you attach the following reference to the copied elements: ‘Copyright (c) 2013. Members of IDGF-SP consortium - http://idgf-sp.eu’. The commercial use of any information contained in this document may require a license from the proprietor of that information. The IDGF-SP consortium members do not warrant that the information contained in the deliverable is capable of use, or that use of the information is free from risk, and accept no liability for loss or damage suffered by any person and organisation using this information.

– 2 – Preface

This document is produced by the IDGF-SP project for the International Desktop Grid Fe- deration.

Please note that there are some links in this document pointing to the Desktop Grid Federation portal, that requires you to be signed in ﬁrst.

This is the companion document to a management document that serves as an introduction.

– 3 – Table of Contents

Preface------3

Introduction------6

1Roadmaps 7 1.1How to start a Desktop Grid?------7 1.2Road map for University (or research institute) to set up a local Desktop grid------8 1.3Road map for Volunteer Desktop Grids (Crowd Computing)------12 1.4Country politics: as part of a (Green) eScience strategy------12 1.5Companies to set up and participate as part of a "Corporate Social Responsibility or Sustainable entrepreneurship------13

2 Technical Documentation------17 2.1 Desktop Grid Technologies------18 2.1.1 Overview------19 2.1.2 BOINC ------19 2.1.3 XtremWeb and XWHEP------20 2.1.4 OurGrid------21 2.1.5 HTCondor------21 2.1.6 Commercial Technologies------22 2.2 Application Development and Porting------24 2.2.1 Application Development------24 2.2.2 Porting Applications to Desktop Grids------25 2.2.3 Application Portals------26 2.3 Integration of Desktop Grids and Other Computing Infrastructures--28 2.3.1 EDGeS 3G Bridge------28 2.3.2 Paths into Clouds ------28

3 Communication and marketing------29 3.1Available communication channels------29 3.1.1- BOINC channels ------29 3.1.2- IDGF channel------29 3.1.3- Country and local channels ------30 3.2 Available marketing and communication tools------31 3.2.1 Portal------32 3.2.2 Message boards------32 3.2.3 Flyers------32 3.2.4 Press------32 3.2.5 Participation in events------32 3.2.6 Social Media------32 3.2.7 Certiﬁed services------33

– 4 – 3.3 Experiences in marketing desktop grids------33 3.3.1 CityGrid example: AlmereGrid------33 3.3.2 Public Grid: EDGeS@Home------3351 3.3.3 A Grid for charity: Charity engine------37

4 Legal and organisational information ------38 4.1 Privacy issues------39 4.2 Licensing------39 4.3 Misuse------40 4.4 Warranties------40 4.5 Security based on certiﬁcates------40 4.6 IPR------40 4.7 Interoperation with Service Grids------40

5Green Desktop Grids------43 5.1 Relevance of Green Desktop Grids------43 5.1.1 Responsibility for real costs: Wise or waste use of energy-43 5.1.2 Selection of Green IT arguments: CO2 footprint and the energy mix------43 5.1.3 € - a metric for Green IT success------43 5.1.4 The eleventh commandment: You shall not waste energy--44 5.2 Green Desktop Grid Methodologies------44 5.2.1 The balance between complexity and Green features------44 5.2.2 Powerful organisational methodologies------45 5.3 Measurements and comparisons with standard data centres------45

6Glossary------47

– 5 – This detailed document is aimed at Introduction administrators and technical operators in the ﬁeld.

If you happened to be in search of a guide into This part of the road map is divided into five the world of Desktop Grid Computing, volunteer sections. & crowd computing, then you do not need to look any further for you are about to discover The first section describes how to proceed to the ultimate "Desktop Grids for eScience" set up a Desktop Grid. In fact, it provides companion. several road maps for different stakeholders, i.e. the academic world, volunteer Desktop Grids, Desktop Grids for eScience - a Road Map aims policy stakeholders at country level, and to provide insight in the power of Desktop Grid company stakeholders to build out a corporate computing, what it can do for eScience, how it social responsibility structure and a Desktop can support scientists in the academic world, Grid infrastructure that supports a sustainable researchers and engineers in industry, and how entrepreneurship. citizens can actively be involved in this process. As such, Desktop Grids offer a holistic approach The second section is a technical description. It of providing huge amounts of computing power offers an overview of the different Desktop Grid for science. It is presented as a White Paper: technologies including a technology comparison we try to present you correct, objective matrix. A Desktop Grid infrastructure also information. requires applications running on it. Therefore, this chapter also elaborates on application The Road Map consists of two parts: a development and application porting with a management document, aimed at stakeholders short description of application portals. The in government, academia and industry and a second section concludes by explaining how technical document, aimed at administrators Desktop Grids can be integrated with other and technical operators in the field. computing infrastructures.

The management document provides a high The third section addresses communication and level and general overview and introduction of marketing issues. Here, you can find an Desktop Grid Computing for eScience. It offers overview of the communication channels and a catchy glimpse of what Desktop Grids have in the communication and marketing tools. This store for eScience by raising the appetite with a section also provides three case studies of few compelling success stories. A survey Desktop Grid marketing and promotion conducted to gauge the interest among the experiences. public to donate idle computing time for eScience offers some remarkable results. The The fourth section deals with the legal aspects green aspects of Desktop Grid computing are of Desktop Grid computing. It offers useful being highlighted as well as the promising future information on organisational problems developments in Desktop Grids. And there is including privacy, licensing, misuse, warranties, also a short focus on how to bring Desktop Grid certificates, IPR, and interoperation with Service computing to the general public with smart and Grids. dedicated communication and marketing tools. The fifth section provides answers on how to make and keep the Desktop Grid infrastructure green. What options do we have to choose from? Which are the challenges related to these options?

– 6 – invertmenst are needed, put of course the 1. Roadmaps operational costs will be higher

Operational and user policies: local policies Desktop Grids can be used in different types of vary by organisation, please contact the local IT organisations: in academia, in industry and in department first groups of volunteer users or even at country level. In case of a smaller volunteer project initiated by a university with about 3,000 active hosts, the majority of costs are for a 50%-time system Each wants to achieve a slightly different goal administrator/programmer. The total start-up and the route to get there is different for each cost was about € 3000 for a server with a RAID type of organisation. storage server. The total monthly cost was In this section, you can find information about about € 5000 per month. the boundary conditions and policies to set up a In case of the largest BOINC based project; Desktop Grid. SETI@HOME, the start-up cost is about € The specific requirements for Desktop Grid 40000 USD and € 12000 as monthly cost computing in the above mentioned (including electricity, power, personnel and other environments will be explained step by step. costs). 1.1 How to start a Desktop Grid? Please note that these costs do not cover the operational and others expenditures related to computers offered by volunteers or the other First of all, the boundary conditions and policies shared computers in the campus/resource must be clarified. institute or in the company. Deployment and operation of Desktop Grids Links requires less efforts than other e-Infrastructures, such as HPC clusters or supercomputers; the Derrick Kondo, Bahman Javadi, Paul Malecot, Franck barriers to entry of such systems are Cappello and David Anderson. Cost-Benefit Analysis of significantly higher. Cloud Computing versus Desktop Grids. 18th International Heterogeneity in Computing Workshop, May Human resources: concerning the needed 25 2009, Rome. skills, usually an system administration in part http://mescal.imag.fr/membres/derrick.kondo/pubs/ time with average skills can operate a Desktop kondo_hcw09.pdf Grid infrastructure. The labour cost depends Distributed Computing in the Cloud - BOINC and Amazons on the country and type of the organisation. Elastic Compute Cloud EC2 IT infrastructure: usually one server is the http:/www.rechenkraft.net/wiki/index.php? title=Datei:Distributed_Computing_in_the_Cloud_- expected hardware requirement, software is _BOINC_and_Amazons_Elastic_Compute_Cloud_EC2_E usually for free available as an open-source NGLISCH.pdf software stack. The operational cost is not negligible but much lower than the start-up cost of hardware. There is experience with operating the server in the Cloud, In that case no

Steps/ Select Study legal Make Select Secure Select Select Deploy/ Port Implement Implemen sections of goal requireme a plan user ﬁnancing type technology application organisation t the roadmap nts group communi cation and support General 4 2.2 2.1 2.2 2 3 Universities/ 1.2 1.2 Phase I 1.2 1.2 1.2 3, 1.2 1.2 Phase 1.2 Phase III 1.2 Phase III, 1.2 Phase Research Phase I (general) Phase Phase II Phase II Phase V III 1.2 Phase V IV organisations 5.1.2 Phase I (scale-up) II (local) Companies 1.5 1.5 Phase I 1.5 1.5 1.5 Phase - 1.5 Phase 1.5 Phase IV 1.5 Phase IV 1.5 Phase Phase II (local) Phase Phase III III IV V 1.5 Phase II II (general)

– 7 – 1.2 Road map for University (or research Studying the related best practices might be institute) to set up a local Desktop grid very useful as well. For the most widespread technologies, Virtual Images and demo accounts are also provided for efficient self- Phase I Gather Desktop Grid related training. information from IDGF sources All of these steps can help figure out and define the preliminary goals and some plans. Gather information and contact the nearest IDGF representative Links

Tutorials on International Desktop Grid Federation portal

The most convenient way of starting and http://desktopgridfederation.org/tutorials gathering information is reading this roadmap document and follow the links and references Technical WIKI on International Desktop Grid Federation for more detailed explanations. The nearest Homepage IDGF representatives and experts can be easily found in the Members section of the IDGF http://desktopgridfederation.org/technical-wiki website.

Links Phase II Gather Desktop Grid related How-to-start section of International Desktop Grid information from local sources Federation portal Collect local needs and local http://desktopgridfederation.org/how-to-start information on opportunities Experts Section of International Desktop Grid Federation Homepage The primary motivation for having an own Desktop Grid in a university or research institute http://desktopgridfederation.org/experts is to serve the local users with an easy-to- maintain, green, and cost-efﬁcient IT infrastructure. It is very important to collect needs and application ideas from the local Join the IDGF community. Studying typical applications The beneﬁts of IDGF membership can be taken scenarios and success stories can help in this from the very beginning in the roadmap, that is process. Also, it is important to check the local why, we recommend the interested ones to join conditions. IDGF as early as possible. Links Links Applications page on International Desktop Grid Federation portal Membership application page on International Desktop Grid Federation portal http://desktopgridfederation.org/applications http://desktopgridfederation.org/membership-application

Be self-trained or attend a half-day IDGF Check local policies tutorial Several different policies regulate the Numerous materials are available on-line for introduction and operation of new IT solutions self-training. However, we recommend to and services inside universities and research attending at least a half-day tutorial given by institutes. The most widespread and usual IDGF experts at various events or on-demand. requirements are the following ones (among For example, a 2-hour hands-on training is others): enough to perform all steps of deployment of a typical Desktop Grid with applications from The operation of the Desktop Grid should not scratch. interfere with the teaching and research activities in the laboratories, and should have

– 8 – negligible influence on the student experience Links or the productivity of researchers. Special configurations and policies in local desktop grid The Desktop Grid system should work in-line environments on the International Desktop Grid Federation Homepage with the energy saving policy introduced by the university or the research institute; e.g. in http://desktopgridfederation.org/technical-wiki/-/wiki/Main/ completely idle mode the unattended PCs must Special+configurations+and+policies+in+local+desktop +grid+environments be shut down automatically or put on very low power consumption sleep mode so they can Introduce Desktop Grid policies awake when new work arrives. It can happen that some modifications are Concerning safety regulations, the Desktop Grid needed in the local policies. On the other hand, servers and clients must be handled similarly to some typical policy documents, such as AUP traditional servers if they are unattended (i.e. and SLA are provided that can be tailored fire alarm). according to the local circumstances.

Introducing of the Desktop Grid must meet the Links security requirements (ﬁrewall, protection of user's data, etc.) as well, in some institutes the Acceptable use policy' on the International Desktop Grid peer-to-peer software solutions are handled Federation Homepage specially (CERN, MTA SZTAKI). http://desktopgridfederation.org/technical-wiki/-/wiki/Main/ Links Acceptable+use+policy

Special configurations and policies in local desktop grid Deploy existing applications from the environments on the International Desktop Grid repositories Federation Homepage (part of D2.1.1 deliverable) http://desktopgridfederation.org/technical-wiki/-/wiki/Main/ Various tested applications are available from Special+configurations+and+policies+in+local+desktop the repositories in order to serve the local user +grid+environments communities. If the required application cannot be found in the repository; the application Build team and allocate (start finding) support experts of IDGF are welcome to resources address the support requests through the local teams or centrally. 1. The team requires usually at least one expert in system administration.

Optional: deploy or use portals to access Phase III Implementation resources

The easy access of computational resources and applications are crucial from the exploitation point of view. Scientiﬁc gateways Based on the requirements and capabilities, and portals can contribute to overcome such after choosing the technology the infrastructure difﬁculties. P-GRADE grid portal, WS-PGRADE must be designed together with allocation of portal, and OurGrid portal are available for necessary hardware components. Since the Desktop Grids. Desktop Grid systems try to harness and utilise Links spare IT capacities, the extra hardware requirements are minimal. P-GRADE website http://portal.p-grade.hu/ Usually there are two steps during the GUSE website deployment of local Desktop Grids: http://www.guse.hu/ - installation of server components - installation of clients OurGrid public portal http://portal.ourgrid.org/

– 9 – Optional: use IADM for application porting monitoring

There are a growing number of applications maintenance, and available in the repositories but there is an option to port applications to Desktop Grids. local support. IDGF Application Development Methodology (formerly callled EDGeS Application Detailed guidelines are provided for these Development Methodology) has been designed purposes. to guide users in the process of porting their Links applications to Desktop Grids for this purpose and several application developers have Report on best practices in infrastructure support (D2.3.1) followed its steps to get reliable and efﬁcient applications. http://idgf-sp.eu/documents/208980/571253/IDGF- SP_D2.3.1.pdf During the application porting process the development, test, and validation infrastructure Introduce Desktop Grid topics in curricula can be used, provided by the IDGF core (in case of universities) members or deployed locally using LiveCD or Virtual Images. Desktop Grids have been successfully integrated in the curricula of some technical Links higher education institutes, summer schools, and they are good candidates even for Report on best practices in application support (D3.3.1) secondary schools because of the relatively simple and easy-to-understand approach http://degisco.eu/documents/10522/38259/ DEGISCO_D331_v19.pdf towards the complex IT systems and solutions, and eScience itself. The curricula can cover the two most important topics:

applications Phase IV Operation, training, communication how to build Desktop Grids

Links

Summer School on Grid Application Support , Clouds and Train the users work flows The available User's Guides for the selected http://www.egi.eu/news-and-media/newsfeed/ news_2013_0024.html technologies can be followed but the IDGF experts are also available to give the necessary trainings. A very frequent problem is that when Publish and disseminate results the users (e.g. biologists, chemists, or The publication and communication of the meteorologists) speak their own technical or achieved scientific and other results can be scientific jargon and the communication might provided via various channels locally or by be difficult. The best trainers can speak the IDGF. 'languages' of both communities. Links Links

5.3.1 Available communication channels Programming the Grid Please check the section later in this document. http://desktopgridfederation.org/technical-wiki

Phase V Scale-up the infrastructure Operate the infrastructure

The operation of infrastructure is lightweight (similarly to the deployment) but the sysadmins must care about some usual IT tasks, such as There are three options to involve more resources into the Desktop Grid.

– 10 – Get worker or client nodes from other Another option is to join another VO with the campus(es) or department(s) Desktop Grid. Usually it requires more negotiation and a MoU between the VO and the Colleagues from other campuses and Desktop Grid operator, some examples are departments can be easily involved in the available. In this case a 3G Bridge instance project. The incentive mechanisms present in must be deployed and operated locally. peer-to-peer Grid computing middleware can be exploited to facilitate the creation of large communities formed by independent desktop Grids. See for instance the example of OurGrid. Links

Start a volunteer project (crowd computing) 3G Bridge http://desktopgridfederation.org/technical-wiki/-/wiki/Main/ This option is recommended after having an 3G+Bridge operational and stable local desktop grid. The volunteers must trust in the grid service operator, i.e. the university or research institute, and 'going public' without solid background and experiences (at least in the local Desktop Grid operations) is hazardous, and the volunteers' trust might be lost.

That is why, it is highly recommended to validate the new applications (to be deployed in the volunteer project) against the criteria deﬁned in the IADM methodologies.

On the other hand, according to the surveys the application must be attractive from the volunteer's point of view. They like supporting e.g. medical applications but business applications are usually not welcome.

The advertisement and public relations of volunteer based Desktop Grids is also critical, some technologies provide communication channels towards the possible volunteers (BOINC).

Links

Technical Documentation BOINC http://boinc.berkeley.edu/trac/wiki/ProjectMain OurGrid community http://status.ourgrid.org

Integrate the infrastructure with service grids

There are two options in order to integrate the created Desktop Grid infrastructure with EGI (European GrIndInfrastructure) conencted Grids (Service Grids).

The easiest way is to use the existing bridge deployed by another partner and share jobs between the gLite based Desktop Grid VO, and the Desktop Grid itself.

– 11 – 1.3 Road map for 1.4 Country politics: as Volunteer Desktop Grids part of a (Green) (Crowd computing) eScience strategy According to our experiences, it is recommended to get practiced ﬁrst by local Desktop Grid technologies before launching a The acceptance and penetration of using public volunteer project. The previous section volunteer and local desktop Grids as a daily describes the roadmap for this, and only after routine in the HPC and especially in the HTC the successful 'in-house' experiments it is related scientiﬁc communities are relatively low. recommended to make it available for the wider Since the user base of volunteer and local audience. Desktop Grids relies particularly on these communities, the responsibility of funding Links agencies and higher level decision makers is to two-fold: Volunteer Computing: The Ultimate Cloud By David P. Anderson, ACM Crossroads, Spring 2010/Vol. 16, No. 3 catalyse the information exchange among the http://portal.acm.org/citation.cfm? key stakeholders, and id=1734164&dl=ACM&coll=DL&CFID=7911886&CFTOKE N=41923379 incorporate the volunteer and local desktop Grids in the Green and eScience roadmaps/ strategies (and in their implementation) at country-level more explicitly.

The rest of the section discusses these topics based on related Roadmaps/Agendas.

In order to achieve the ﬁrst goal, the invitation of IDGF representatives for consultancies and public information days organised by the funding agencies and R&D related departments of ministries may improve the communication. Later the involvement of more representatives in the related pan-European and worldwide support organisation can be the next phase. These steps are in-line with the e-IRG roadmap, descriptions in chapter 4,Reorganise for Change.

The Digital Agenda, Section 2.7.1. 'ICT for environment' clearly states that 'The ICT sector should lead the way by reporting its own environmental performance by adopting a common measurement framework as a basis for setting targets to reduce energy use and greenhouse gas emissions of all processes involved in production, distribution, use and disposal of ICT products and delivery of ICT services.' The IDGF members are ready to cooperate in the related Green computing actions and projects nationally and internationally as well.

The Digital Agenda, Section 2.3 'Trust and security' describes topics which are crucial for Desktop Grids as well, the joint efforts in such areas are also the common interests of the key

– 12 – stakeholders in order to provide better security mechanism for end-users and resource owners 1.5 Companies to set up (i.e. volunteers). and participate as part From a technical point of view, the efficient exploitation of Desktop Grids depends on the of a "Corporate Social available internet bandwidth in a lot of cases. Responsibility or The Digital Agenda, Section 2.4 'Fast and ultra fast internet access' will let the volunteer Sustainable Desktop Grid operators allow to provide a more efficient computational platform for the entrepreneurship scientists. Joint actions with network service providers may help attract more volunteers. Phase I Check company policies The e-Infrastructures Reflection Group (e-IRG) is a policy body on using infrastructures for science (e-infrstructures) in Europe has published a Road Map for e-Infrastructures in 2012. The Road Map gives a vision for 2020 Company polices restriction inline with the European Commission;s vision for 2020 (Horizon 2020). The Road Map says: One of the first things that should be done “e-IRG’s vision for 2020 is that Europe needs a before starting a Desktop Grid inside a company single “e-Infrastructure Commons” for is to contact the IT department and confirm that knowledge, innovation and science, as a living the company doesn't have policies that prohibit ecosystem, which is open and accessible and the use of Desktop Grids internally or any other continuously adapts to the changing restriction about the Desktop Grid software requirements of research .” license. Some companies may have some restrictions about using open source software or software not registered internally.

The EIROForum, in which a number of large e- Software license restriction infrastructures (large European science centres.)are organised, published a plan for It is necessary to know the software that will be European e-Science infrastructures. They executed/installed inside the Desktop Grid and propose to install Research Accelerator Hubs confirm with the IT department that the (ReAcH) for major scientific research areas. company has the sufficient or adequate Volunteer computing should be integrated in the software license to use it. e-Infrastructure commons proposed by e-IRG,. Links

Software license for running inside a Grid Links http://www.elasticlm.de/ﬁleadmin/SmartLM/Download/ Digital Agenda for Europe The_Business_Side_of_Software_Licensing.pdf http://ec.europa.eu/information_society/digital-agenda/ index_en.htm Network connection e-IRG Roadmap 2012 Some companies, for security reasons, http://www.e-irg.eu/publications/roadmap.html recommend that the network connection Distributed Computing Infrastructure (DCI) Collaborative between the Desktop Grid machines should be Roadmap a different one from the main network used http://edgi-project.eu/documents/10515/50865/ internally. Contact the IT department to know EDGID22v2.pdf the security requirements of the company A Vision for European e-Infrastructure for the 21st Century (ﬁrewall, protection of user's data, etc.). hhttp://e-irg.eu/images/stories/einfra-bobjones-eirg- nov2013.pdf Phase II Gather Grid related information from IDGF sources

– 13 – Gather information and contact the nearest Phase III Gather Grid related information IDGF representative from local sources The most convenient way of starting and gathering information is reading this roadmap document and follow the links and references for more detailed explanations. The nearest Collect local needs and local information on IDGF representatives and experts can be easily opportunities found in the Members section of the IDGF website. The primary motivation for having an own Desktop Grid in a company is to gain in Links computing power, speed, and productivity. This leads to an easy-to-maintain, green, and cost- How-to-start section of International Desktop Grid efficient IT infrastructure. Federation portal http://desktopgridfederation.org/ Some advantages for companies are: Experts Section of International Desktop Grid Federation Homepage No need to buy large and expensive servers for running time consuming applications. http://desktopgridfederation.org/experts Efficient on using idle resources. Many Join the IDGF resources sit idle especially during off business hours or are not using 100% of the computer The benefits of IDGF membership can be taken power consumption. from the very beginning in the roadmap, that is why we recommend the interested ones to join Modular environment: Don't have single IDGF as soon as earliest. points of failure. If one of the desktops within the Desktop Grid fails there are other resources Links able to pick the load.

Membership application page on International Desktop Scalability: Need more compute resources? Grid Federation portal Just plug them in by installing Desktop Grid http://desktopgridfederation.org/membership-application client on additional desktops. They can also be removed easily.

Be self-trained or attend a half-day IDGF Upgrading: Can be done on the ﬂy without tutorial scheduling downtime. Since there are so many resources some can be taken ofﬂine while Numerous materials are available on-line for leaving enough for work to continue. self-training. However, we recommend to attend at least a half-day tutorial given by IDGF experts It is very important to collect needs and at various events or on-demand. For example, a application ideas from the local community. 2-hour hands-on training is enough to perform Studying typical applications scenarios and all steps of deployment in a typical Desktop success stories can help in this process. Also, it Grids with applications from scratch. is important to check the local conditions.

Studying the related best practices might be Links very useful as well. For the most widespread technologies, Virtual Images and demo Grid and Cloud Computing: A Business Perspective on accounts are also provided for efﬁcient self- Technology and Applications training. http://www.springer.com/business+%26+management/ business+information+systems/book/978-3-642-05192-0 Links (chapter 3)

Tutorials on International Desktop Grid Federation Build team and allocate (start ﬁnding) Homepage resources

http://desktopgridfederation.org/tutorials The team requires usually at least one expert in Technical WIKI on International Desktop Grid Federation system administration. Homepage http://desktopgridfederation.org/technical-wiki

– 14 – Optional: deploy or use portals to access Phase IV Implementation resources

The easy access of computational resources and applications are crucial from the Based on the requirements and capabilities, exploitation point of view. Scientiﬁc gateways after choosing the technology the infrastructure and portals can contribute to overcome such must be designed together with allocation of difﬁculties. P-GRADE grid portal, WS-PGRADE necessary hardware components. Since the portal, and OurGrid portal are available for Desktop Grid systems try to harness and utilise Desktop Grids. spare IT capacities, the extra hardware Links requirements are minimal.

Usually there are two steps during the P-GRADE website deployment of local Desktop Grids: http://portal.p-grade.hu/ GUSE website installation of server components http://www.guse.hu installation of clients OurGrid public portal http://portal.ourgrid.org/ Links Optional: use IADM for application porting Special configurations and policies in local desktop grid environments' on the International Desktop Grid Federation Homepage There are a growing number of applications available of the repositories but there is an http://desktopgridfederation.org/technical-wiki/-/wiki/Main/ option to port applications to Desktop Grids. Special+configurations+and+policies+in+local+desktop +grid+environments IDGF Application Development Methodology has been designed for this purpose and several Introduce Desktop Grid policies application developers have followed its steps to get reliable and efficient applications. It can happen that some modifications are needed in the local policies. On the other hand, During the application porting process the some typical policy documents, such as AUP development, test, and validation infrastructure and SLA are provided that can be tailored can be used, provided by the IDGF core according to the local circumstances. members or deployed locally using Virtual Images. Links Links

Acceptable use policy on the International Desktop Grid Federation Homepage Report on best practices in application support (D3.3.1) http://desktopgridfederation.org/technical-wiki/-/wiki/Main/ http://degisco.eu/documents/10522/38259/ Acceptable+use+policy DEGISCO_D331_v19.pdf

Deploy existing applications from the Phase V Operation, training, communication repositories

Various tested applications are available from the repositories in order to serve the local user communities. If the required application cannot Train the users be found in the repository; the application support experts of IDGF welcome the requests The available User's Guides for the selected through the local teams or centrally. technologies must be followed but the IDGF experts are also available to give the necessary Links trainings. A very frequent problem is that when the users (e.g. biologists, chemists, or Membership application page on International Desktop meteorologists) speak their own technical or Grid Federation Homepage scientiﬁc jargon and the communication might http://desktopgridfederation.org/membership-application

– 15 – be difﬁcult. The best trainers can speak the operations and in their interaction with their “languages” of both communities. stakeholders on a voluntary basis."

Links Corporate Social Responsibility is part of the Europe 2020 strategy for smart, sustainable and Programming the Grid inclusive growth. It can help to shape the kind of http://desktopgridfederation.org/technical-wiki competitiveness model that Europe wants. The key issues of corporate social responsibility Operate the infrastructure vary from company to company. For example, The operation of infrastructure is lightweight enterprises in the retail sector might have to (similarly to the deployment) but the sysadmins deal with the risk of poor labour standards in must care about some usual IT tasks, such as their supply chain, while a mining company is more concerned by the need to avoid infringing monitoring the human rights of people living near its operations. maintenance, and Small and medium-sized enterprises (SMEs) local support. are the predominant form of enterprise in the European Union. If Europe and its enterprises Detailed guidelines are provided for these are to reap the full beneﬁts of CSR, it is vital to purposes. make sure that SMEs are fully engaged and that what they do is fully recognised.

Small businesses are typically not less Links responsible than large enterprises. They may not know and use the term "CSR", but their Report on best practices in infrastructure support (D2.3.1) close relations with employees, the local http://idgf-sp.eu/documents/208980/571253/IDGF- community and business partners often mean SP_D2.3.1.pdf they have a naturally responsible approach to business. Publish and disseminate results CSR is more relevant than ever in the context The publication and communication of the economic crisis. It can help to build (and rebuild) achieved scientiﬁc and other results can be via trust in business, which is vital for the health of various channels provided locally or by IDGF. Europe's social market economy. It can also point the way to new forms of value of creation based on addressing societal challenges, which may represent a way out of the crisis. Links Links Available communication channels Please check this section of this document. Sustainable and responsible business http://ec.europa.eu/enterprise/policies/sustainable- business/corporate-social-responsibility/index_en.htm Phase VI Participate as part of a European business network for corporate social Corporate Social Responsibility responsibility

or Sustainable entrepreneurship http://www.csreurope.org/

Desktop Grid in the CSR Corporate Social Responsibility (CSR) can make a signiﬁcant contribution towards Basically at least two opportunities can be sustainability and competitiveness, both in distinguished which can improve the Corporate Europe and globally. The European Social Responsibility (CSR) of companies. Commission deﬁnes the CSR as: Some applications running on a Desktop Grid "A concept whereby companies integrate social have obviously direct social impact. Among and environmental concerns in their business others, these targeted applications can be listed in this category:

– 16 – City Population Dynamics and Sustainable Company success stories Growth (CPDynSG) Two success stories have been selected for Drug discovery with Autodock illustration purposes.

Social Simulations The first one is a global company, IBM, which has been awarded by the Asia's Foremost CSR BNB-GRID for Protein structure prediction Awards Programme. The award was given to IBM Philippines (an ICPC country) in the On the other hand, using Desktop Grid category 'Poverty Alleviation' because of the technology has positive environment impact as related efforts and achievements of BOINC- well; the company can significantly contribute to based IBM World Community Grid. reduce the ecological footprint by operating a green Desktop Grid instead of other The other example represents a local initiative; technologies. Rabobank from the Netherlands shows the AlmereGrid, the first citygrid in the world in its Links demo room. Rabobank was a technology provider for AlmereGrid in the BeINGRID World Community Grid project. http://www.worldcommunitygrid.org Links

Asia's Foremost CSR Awards Programme Awards Winners http://www.chinaretailnews.com/2009/11/24/3168-asias- foremost-csr-awards-programme-awards-winners- announced/

– 17 – Technology comparison matrix

BOINC XtremWeb OurGrid Condor Architecture Client-Server Client-Server Peer-to-peer Central Broker Application Management Centralized Centralized Decentralized Decentralized Resource Providers can No Yes Yes Yes act as Resource Consumers Task Distribution Pull Pull Push Push Deployment/ Medium / Low on Low Low Medium Administration Complexity client side Application Development/ High / Medium (with Low Low Medium Porting Complexity wrapper) Support for Volunteer Yes Yes Yes No Desktop Grids Security Features Code signing, Result Sandbox Sandbox Authentication, validation (Virtual Authorization, Machine) Encryption Web Interface Yes (Monitoring) Yes Yes, via No (Monitoring) OurGrid Portal (Monitoring, Job Submission) Distinguishing Features Volunteer experience Flexible Peer-to-peer Support for access rights model, parallel jobs model, Network-of- (MPI, PVM), Support for X. Favors workflows (DAG) 509 mechanism. certificates Support for X. 509 certificates Number of Deployments ~100 ~10 A few ~100 (~1M CPUs in big projects) Programming Language C/C++ Java Java C/C++ Documentation and Good Good Good Very Good quickstart materials

– 18 – computing, making it easier and cheaper for 2. Technical scientists to use. The ﬁrst BOINC-based projects launched in 2004, and today there are Documentation about 60 such projects, in a wide range of scientiﬁc areas.

The chapter with a technology overview and The BOINC software consists of two parts: references to technical documentation starts server software that is used to create volunteer with a description of the different existing computing projects and client software. Anyone Desktop Grid technologies. They are compared can create a project. Each project operates its to each other in a technology comparison own server and provides its own web site. matrix. The next subpart addresses the Volunteers install and run client software on applications running on Desktop Grids: how to their computers. The client software is available develop applications and how to port them to for all major platforms, including Windows, the Desktop Grid infrastructure. Here, you can Linux, and Mac OS X. also find information on application portals. The chapter ends by describing how you can A BOINC project is identified by a single master integrate Desktop Grids with other types of URL, which is the home page of its web site and computing infrastructure, i.e. with the EDGeS also serves as a directory of scheduling servers. 3G Bridge, and Clouds technologies. A project can involve one or more applications, and the set of applications can change over time. BOINC accommodates a wide range of applications; it provides flexible and scalable mechanisms for distributing data, and its scheduling algorithms intelligently match 2.1 Desktop Grid requirements with resources. Existing applications in common languages (C, C++, Technologies FORTRAN) can run as BOINC applications with little or no modification.

2.1.1 Overview Having installed the client programme, volunteers can then attach it to any set of In case of local desktop Grids where the projects, and for each project can assign a provided PCs are not from volunteers, the resource share that determines how the approach of University of Westminster Local computer’s resources are divided among the Desktop Grid (WLDG) can be followed as best projects. Attaching to a project allows it to run practice that connects laboratory PCs of the arbitrary executables on one’s computer, and University of Westminster (London, UK) into a BOINC provides only limited (account-based) BOINC based infrastructure. sandboxing. So the volunteer must assess the project’s authenticity, its technical competence, In case of local desktop Grids where the and its scientific merit. resource providers are particularly volunteers, the OurGrid (peer-to-peer) or XtremWebHEP-E BOINC does accounting of credit, a numerical examples can be followed. measure of a volunteer’s contribution to a project. It also provides a mechanism that In case of global desktop Grids, the approach of exports credit-related data for processing by SZTAKI Desktop Grid (for few applications) and credit statistics sites operated by third parties. Ibercivis (for larger networks) can be followed As part of the accounting system, BOINC as best practice because of their success at provides a cross-project user identification large scale as well. mechanism based on the volunteer’s email address. This mechanism allows leaderboard 2.1.2 BOINC sites to display user credit statistics summed BOINC (Berkeley Open Infrastructure for over multiple projects. Network Computing) is an open source platform A level of indirection can be placed between for Desktop Grid computing. It is being client and projects. Instead of being attached developed at U.C. Berkeley Spaces Sciences directly to projects, the client can be attached to Laboratory by the group that developed and a web service called an account manager. The continues to operate the SETI@home project. client periodically communicates with the BOINC was established in 2002 to develop account manager, passing it account credentials general-purpose middleware for volunteer and receiving a list of projects to attach to. This – 19 – framework can be used for delegation of project to secure and confine distributed resources selection to some trusted committee, analogous usage. This mechanism is achieved by the to mutual funds. implementation of notions of user and access rights. These new features permit to extend Links user actions over the platform as well as to secure resource usage and confine application BOINC Project Homepage deployment. http://boinc.berkeley.edu/ The new version of the XtremWeb-HEP User Documentation middleware, XWHEP 8, that was introduced in http://boinc.berkeley.edu/wiki/Main_Page February 2012, designated as “FlyingGrid2”, Technical Documentation adds the new “volunteer sharings” paradigm. http://boinc.berkeley.edu/trac/wiki/ProjectMain This new version keeps all previous functionalities and introduces new paradigms, Quick start: distributed computing in one hour or less (Slides) services and protocols to implement the “volunteer sharing” paradigm. The architecture http://boinc.berkeley.edu/slides/boinc_intro.pdf remains the same. Desktop grids intend to aggregate distributed volunteer computing Installing a BOINC Grid resources (known as “worker” in XWHEP) and http://www.edges-grid.eu:8080/web/11219/2/wikipage/ distribute tasks on demand. According to a local 11978/InstallingBoincGrid activation policy, the worker connects to the centralized server, downloads task (binary code 2.1.3 XtremWeb and XWHEP and their associated data) and executes the XtremWeb is an open source software to build a downloaded binary code. This mode is known lightweight Desktop Grid by gathering the as the “pool” mode. unused resources of Desktop Computers (CPU, By introducing the “volunteer sharing” paradigm, storage, network). Its primary features permit the worker still works following the pool mode multi-users, multi-applications and cross- mechanism but can also propose some objects domains deployments. XtremWeb was to share: data, applications and library. developed by INRIA/IN2P3 as a research project to explore scientific issues and The figure shows the architecture of XWHEP8, applications of Desktop Grid, Global Computing including volunteer sharing. This is still a three- and Peer to Peer distributed systems. XWHEP tier architecture with centralized services that (XtremWeb-HEP) is a successor of XtremWeb ensure the deployment coherency, distributed currently developed at LAL (Laboratoire de clients to use and manage the platform, and l’Accélérateur Linéaire) belonging to IN2P3- workers to aggregate volunteer resources. CNRS. It is written in Java and available under Compared to the original architecture we can GPLv2 license. see the new “volunteer sharing” paradigm on worker side. This represents any object (data, The XWHEP/ XtremWeb architecture is application, library) the worker wishes to composed of Servers, Workers and Clients. propose to the deployment. If the required One or a group of Servers is installed and conditions are met (mainly the security issues), maintained by system administrators to host this volunteer sharing may be proposed to centralized Services such as Scheduler and users. It is the volunteer resource owner Result Collector. Workers are installed by responsibility to propose or deny volunteer resource owners on their PCs to propose their sharing from its resource. computing resources to be aggregated within an XWHEP/XtremWeb infrastructure. Clients are The current version is XtremWeb 8.3.2, which installed by resource users (scientists, for has some minor corrections compared to 8.0. example) on their PCs to interact with the infrastructure. The Client permits users to manage the platform, install applications and use distributed resources (submit jobs, retrieve Features: results...). Jobs submitted by Client are registered on the Server and scheduled on Flexible access rights model; Support for X.509 Workers. Note that within this architecture any certificates; Worker can be a Client. Support for OpenId and OAuth authentication XWHEP enforces greatly the security of systems; XtremWeb by introducing mechanisms aiming

– 20 – Support for Virtual Machines; Peer to Peer OurGrid is based on a peer-to-peer architecture, communications facilities; where each grid site corresponds to a peer in the system. The main problem using a peer-to- We interface for monitoring, job submission, peer system is that the performance could be data management. compromised by freeriders: a peer that only consumes resources, never contributing back to Links the community. In order to incentive peers to cooperate, and consequently discourage the XtremWeb website free riders, OurGrid uses a Network of Favours http://www.xtremweb.net/ mechanism: a favour is the allocation of a XtremWeb-HEP website processor to a peer that requests it, and the value of that favor is the value of the work done http://www.xtremweb-hep.org/spip.php?lang=en for the requesting peer. Each peer keeps a local XWHEP User Guide record of the total value of the favours it has http://www.xtremweb-hep.org/spip.php?rubrique16 given to and received from any other peer. The XWHEP Admin Guide rationale is that each peer autonomously prioritize donations to the peer to whom he http://www.xtremweb-hep.org/spip.php?rubrique16 owes most favours, motivating cooperation. Architecture of XtremWeb-HEP The OurGrid team also runs the OurGrid Community, an open, free-to-join, cooperative grid in which sites donate their idle computational resources in exchange for accessing other sites' idle resources when needed. It is in production since December 2004. Any site can join the system by downloading and installing the OurGrid software. No human contacts or negotiation are needed for a new site to join the system.

Links

OurGrid website http://www.ourgrid.org/ User Documentation 2.1.4 OurGrid http://www.ourgrid.org The OurGrid middleware enables the creation of OurGrid Community peer-to-peer computational grids. In the peer- http://status.ourgrid.org to-peer grids enabled by OurGrid, computing and storage resources originated from spare resources (idle CPUs and free disk space), are provided by a whole community of grid participants, and are shared in such a way as to make those who have contributed the most to get the most out of the grid whenever they need it. The software is written in Java, allowing any resource capable of running a Java Virtual Machine to be tapped on to the grid. OurGrid is an open-source software distributed under GPL license.

OurGrid provides a scalable and secure platform for execution of Bag-of-Tasks (BoT) applications. BoT applications are those parallel 2.1.5 HTCondor applications whose tasks are independent. These include parameter sweep simulations, HTCondor is a high-throughput distributed batch rendering of images, optimization algorithms, computing system developed by the HTCondor and many others. Research Project at the University of Wisconsin- Madison, USA, previously known as Condor.

– 21 – While providing functionality similar to that of they are willing to run. These requirements and conventional batch systems, HTCondor's novel preferences can be described in powerful architecture allows to perform well in expressions, resulting in HTCondor's adaptation environments where other batch systems are to nearly any desired policy. weak: high-throughput computing and opportunistic computing. The goal of a high- HTCondor can be used to build Grid-style throughput computing environment is to provide computing environments that cross large amounts of fault tolerant computational administrative boundaries. HTCondor's power over prolonged periods of time by "flocking" technology allows multiple HTCondor effectively utilizing all resources available to the compute installations to work together. network. The goal of opportunistic computing is the ability to use resources whenever they are Links available, without requiring one hundred percent availability. HTCondor website http://research.cs.wisc.edu/htcondor/ HTCondor can be used to manage a cluster of User and Administrator Documentation dedicated compute nodes. In addition, unique mechanisms enable HTCondor to effectively http://research.cs.wisc.edu/htcondor/manual/ harness wasted CPU power from otherwise idle Publications on HTCondor desktop workstations. For instance, HTCondor http://research.cs.wisc.edu/htcondor/publications.html can be configured to only use desktop machines where the keyboard and mouse are idle. Should HTCondor detect that a machine is no longer available (such as a key press 2.1.6 Commercial Technologies detected), in many circumstances HTCondor is able to transparently produce a checkpoint and migrate a job to a different machine which would otherwise be idle. HTCondor does not require a Platform LSF Desktop shared file system across machines - if no LSF Desktop is a Desktop Grid technology shared file system is available, HTCondor can especially built for company internal use. It is transfer the job's data files on behalf of the user, conceptually integrated into the Platform LSF or HTCondor may be able to transparently product line and is professionally supported. redirect all the job's I/O requests back to the The LSF Desktop server is available for Linux submit machine. As a result, HTCondor can be and appears as an LSF server with so many used to seamlessly combine all of an CPUs (job slots) as there are desktops organisation's computational power into one connecting to the desktop server. The LSF resource. Desktop client is available for Microsoft A HTCondor pool is comprised of a single Windows only. It uses native Windows machine which serves as the central manager, mechanisms for data protection and job and an arbitrary number of other machines that sandboxing. This is welcomed by Windows have joined the pool. Conceptually, the pool is a system administration as they can control collection of resources (machines) and resource configuration and behaviour by policies in very requests (jobs). The role of HTCondor is to fine granularity. Most applications that run from match waiting requests with available the Windows command-line will work with LSF resources. Every part of HTCondor sends Desktop. The security mechanisms periodic updates to the central manager, the implemented are targeted to Enterprise use - centralized repository of information about the inside the secure company network only. state of the pool. Periodically, the central Inside the Enterprise, one could also use manager assesses the current state of the pool Platform LSF instead. LSF is available for all and tries to match pending requests with the operating systems – no restriction to Windows appropriate resources. or Linux – and runs perfectly on desktops. The ClassAd mechanism in HTCondor provides Different from LSF Desktop, the priority for the an extremely flexible and expressive framework workload can be chosen within the boundaries for matching resource requests (jobs) with of the selected OS. Depending on the resource offers (machines). Jobs can easily importance of the workload and the commitment state both job requirements and job of the desktop owner, higher priorities can be preferences. Likewise, machines can specify chosen compared to “screen-saver” – enabling requirements and preferences about the jobs

– 22 – quality of service for workload. MPI and facilities needed and the heat produced is workﬂows are natively supported. extensively reducing the carbon footprint.

Platform is part of IBM. Links

IBM Platform computing Nerdalize http://www-03.ibm.com/systems/technicalcomputing/ http://nerdalize.com platformcomputing/

DataSynapse / TIBCO GridServer Quarnot Desktop Grid computing provider DataSynapse was acquired by Tibco in 2009 for US$28 This French startup offers a similar service as million. Until then the company's installed base Nerdalize.Throughout their Q.ware distribution was about 100 worldwide organisations running platform tasks can be 4 times cheaper DataSynapse application management software performed compared to what other solutions on across approximately 1 million CPUs making it, the market can provide. Their Q-rad radiators probably, the most successful desktop grid turn the energy produced during the middleware after BOINC. computational activities into heating. Although the radiators do not run when heating is not Nowadays DataSynapse software is provided required, a low-consumption mode can still be as a TIBCO GridServer which is one of the maintained, with enough computing power market's leading grid computing infrastructure available. platforms. With more than one million deployed CPUs spread across 1,000 global installations, It is also an extremely environmentally friendly TIBCO GridServer is the backbone of enterprise approach “by spreading power and heat to right grids deployed by businesses operating in the places”. world's most demanding industries. Links TIBCO Grid server http://www.tibco.com/products/automation/application- Quarnot development/grid-computing/gridserver/default.jsp http://www.qarnot-computing.com/ Parabon Frontier http://www.parabon.com/ ProActive Parallel Suite http://www.activeeon.com/products Digipede Network http://www.digipede.net/ Red Hat MRG Grid http://www.redhat.com/mrg/grid/ Univa Grid MP http://www.univa.com/products/grid-mp.php

Nerdalize

Nerdalize provides an innovative, ecofriendly solution to energy waste caused by the use of computer servers. In their model, servers are used as heat radiators (Grid Heater) in homes. The company sells the aggregated computing while the energy produced is converted into heat. As the home owner is reimbursed for the energy used by the heater,the heating costs are reduced to nothing.

The advantage is multi-sided: the computation costs less as there are no expensive cooling – 23 – For example, BOINC provides both server-side 2.2. Application (work unit generation, result validation and assimilation) and client-side (application Development and execution control, checkpointing and resource Porting usage accounting) interfaces. Since there are many legacy applications, that have either no source code available or would require too 2.2.1 Application Development much effort to port, BOINC also provides a conﬁgurable wrapper which lets you run any Applications suitable for running on a desktop executable (or sequence of executables) with grid should meet the following requirements: no modiﬁcation.

Only master/worker or parameter sweep GenWrapper is an advanced generic solution parallelisation (bag-of-tasks applications) for wrapping and executing an arbitrary set of legacy applications for BOINC. It utilizes a script No shared data storage based programmable (POSIX like) environment to describe how the application is to be run and No MPI or internal communication between how the task should be processed. This is worker nodes realized by an advanced interpreter which provides the most common UNIX commands, a Nodes can use the results of other nodes, POSIX shell interpreter and BOINC functions but only through the server accessible from the shell in the same way on multiple platforms (Windows, Linux and Mac OS Typically long running jobs with small or X). medium-sized (max. 100 MB per slave) inputs and outputs Both XtremWeb/XWHEP and OurGrid support execution of arbitrary binary files without registration and porting which simplifies Applications running on desktop grids are application development. generally composed of two distributed parts: GBAC (“Generic BOINC Application Client”) is a server-side (master part) and client-side virtualization (VirtualBox) based wrapper. (worker part). Beyond its name it aims to be a generic The server-side part of application is framework providing virtualized environments responsible for generating tasks to be sent to for various distributed computing infrastructures desktop machines, validating results of task (DCIs). Its goal is to make possible the execution and processing these results into execution of arbitrary binary files that have some form presented to the application user. special requirements (OS version, library dependencies) without porting for different The client-side part runs on desktop machines DCIs. It was implemented using the DC-API and is responsible for executing tasks. It must Meta API and does not rely on any middleware be able to run as a background process without specific functionalities. The current version interactive input or GUI. The heterogeneity of supports BOINC, Condor and XtremWeb/ desktop grid nodes implies that the client-side XWHEP middleware beside standalone part of the application should be available for all execution. popular platforms. In case of XtremWeb/XWHEP, an application The desktop grid technology provides a developer should first register an application middleware layer between these two parts of executable by uploading it on XtremWeb Server. application which transparently distributes It is possible to register multiple application application code and tasks among grid binaries for different target platforms. Any user machines, monitors task execution, handles can register its own applications, but the failures, collects task results and passes them application access rights depend on the user to the application. credentials. Administrator rights are needed to register a public application accessible by all Desktop grid technologies provide generic users. After the application is registered a user Command-Line (CLI) or Application can submit new jobs by providing an application Programming Interfaces (API) for application identifier, command-line arguments and development, deployment and management. references to input data. Bag-of-tasks

– 24 – applications are supported by managing groups Desktop Grids based on a push model, such as of jobs. OurGrid and Condor normally do not require code modifications for applications that already OurGrid deals with BoT applications, i.e. run on a stand-alone machine. applications comprised of independent tasks. An application is described by a user in the form Links of job description file. A job consists of multiple tasks. A task is formed by initial, remote, and BOINC Application Development final subtasks, which are executed sequentially. Subtasks are external commands invoked by http://boinc.berkeley.edu/trac/wiki/ProjectMain OurGrid; consequently, any programme, written BOINC Application Development in any language, can be a subtask. The initial http://boinc.berkeley.edu/trac/wiki/AppIntro and final subtasks are executed locally, on the user machine. The initial subtask is meant to set GenWrapper up the task’s environment by, for instance, http://genwrapper.sourceforge.net/ transferring the input data to the grid machine GBAC – “Generic BOINC Application Client” selected to run the task. The final subtask is http://gbac.sourceforge.net/ typically used to collect the task’s results back to the user machine. The remote subtask runs XWHEP User Guide on a grid machine and performs the http://www.xtremweb-hep.org/lal/doc/xwhep-user-guide.pdf computation per se. In addition to its subtasks, Running a Job in OurGrid a task definition also includes the grid machine http://www.ourgrid.org/ requirements. OurGrid allow users to write subtasks without knowing details about the grid Road-map for Running Jobs in Condor machines on which they will be run, such as http://www.cs.wisc.edu/condor/manual/ how their file systems are organized. The v7.5/2_4Road_map_Running.html abstractions storage and playpen represent Application Development (SZTAKI Desktop Grid) storage spaces in the unknown grid machine. http://desktopgrid.hu/index.php?page=22 File transfer commands are available to send DCI-API and retrieve files in the storage spaces. http://www.desktopgrid.hu/index.php?page=34 Condor supports both execution of unmodified binary files as well as programmes relinked with Condor library which provides automatic checkpointing and remote system calls. 2.2.2 Porting Applications to Desktop Grids Controlling the details of a job submission is a submit description file. The file contains Although many applications have been ported information about the job such as what to the grid or developed specifically to be run on executable to run, the files to use for keyboard various distributed computing resources, these and screen data, the platform type required to efforts were in many cases ad-hoc and did not run the programme, and where to send e-mail follow any particular methodology. In most of the when the job completes. You can also tell Grid projects middleware development was at Condor how many times to run a programme; it the focus, and application development served is simple to run the same program multiple in many cases demonstrative purposes only. times with multiple data sets. Even in examples of production level application development efforts, these applications were The Distributed Computing API (DC-API) was very often poorly documented which made created by MTA SZTAKI to allow easy further improvement cumbersome. In order to implementation and deployment of distributed overcome these difficulties and to identify a applications on multiple grid environments. In process and a series of documentation order to accommodate the needs of very templates, the EDGeS project has developed different grid environments, the DC-API the IDGF Application Development Methodology supports only a restricted master-worker (IADM). This methodology was successfully programming model. The application developed validated and utilized in the EDGeS project using DC-API can be run in BOINC, Condor and when porting 22 target applications, including XtremWeb/XWHEP environments. The API popular ones such as AutoDock, to the EDGeS comes prepacked in a Debian package, Grid infrastructure. available freely from the official website. IADM identifies the following well defined stages that have a suggested logical order:

– 25 – 1. Analysis of current application grid platforms. P-GRADE Portal hides low-level grid access mechanisms by high-level graphical 2. Requirements analysis interfaces, making even non grid expert users capable of defining and executing distributed 3. Systems design applications on multi-institutional computing infrastructures. Workflows and workflow based 4. Detailed design parameter studies defined in the P-GRADE 5. Implementation Portal are portable between grid platforms without learning new systems or re-engineering 6. Testing programme code. Technology neutral interfaces and concepts of the P-GRADE Portal help users 7. Validation cope with the large variety of grid solutions. More than that, any P-GRADE Portal installation 8. Deployment can access multiple grids simultaneously, which enables the easy distribution of complex 9. User support, maintenance & feedback applications on several platforms.

The overall development process in most cases gUSE (grid User Support Environment) is an is non-linear allowing revisiting and revising the evolution of P-GRADE which is implemented as results of previous phases at any point. Each a set of Web services that bind together in stage is described by its objectives, participants, flexible ways on demand to deliver user outcome and set of questions to be answered. services in Grid and/or Web services environments. User interfaces for gUSE IADM also defines how the recommended services are provided by the WS-PGRADE Web software tools (DC-API, GenWrapper, GBAC, application. WS-PGRADE is a Web portal WS-PGRADE portal, EDGeS Application hosted in a standard portal framework. WS- Repository) can aid the process of application PGRADE uses the client APIs of gUSE services development and execution in Desktop Grids. to turn user requests into sequences of gUSE The most recent versions of BOINC allows to specific Web service calls. WS-PGRADE hides automatically also install Virtual Box. A BOINC the communication protocols and sequences application can then be a Virtual machine (VM) behind JSR168 compliant portlets. End users that packages the application inside. This can access WS-PGRADE via Web browsers. A makes application porting easier as you only graph editor component can be downloaded have to package an application into one Virtual from WS-PGRADE via the browser to the user Machine that can run on a BOINC client machine. The editor can be used to define the machine inside Virtual Box. Not only makes this static skeleton of workflows, while the HTML creating applications for a Desktop Grid easier, pages of WS-PGRADE provide interfaces to it also increases the security. add content to graphs, to generate complete Grid/Web service applications. Other Desktop Grid technologies have similar virtualisation features. OurGrid developers provide a public portal which enables users to run jobs in OurGrid Links Community opportunistic grid without installing and configuring OurGrid software on their List of applications ported to Desktop Grid infrastructures machines. The Portal also provides a data storage service where users can place http://desktopgridfederation.org/applications application and input files. Output files Application Porting Introduction generated by jobs can also be retrieved using http://desktopgridfederation.org/documents/10508/22303/ this user-friendly service. IDGF-Application-porting-introduction.pdf EDGeS Application Development Methodology This public portal is intended to demonstrate the technology. It can be tested with little effort. http://portal.desktopgridfederation.org/documents/ 10508/80409/EADM.ppt Potential users can evaluate the usability of an OurGrid system for their applications before 2.2.3 Application Portals engaging in installing their own local system.

The P-GRADE Grid Portal is a web based, MTA SZTAKI and the University of Westminster feature-rich environment for the development, announced the opening of a public Molecular execution and monitoring of workﬂows and Docking Science Gateway in the mid of 2012. workﬂow based parameter studies on various The gateway is freely accessible for bio-

– 26 – scientists and powered by the WS-PGRADE Grids to minimizse this. For insatncesending the portal and the SZTAKI Desktop Grid same task to seevral different donors, and technologies. Interested users can visit the veryﬁng the results. Private and local grids (eg. gateway website and sign up with Facebook, Condor-based) where such trust is high usually OpenID, or email account. The docking employ only standard security mechanisms experiments are executed on the public such as authentication, authorization and EDGeS@home desktop grid that offers the encryption. Centralized volunteer computing computational power of nearly 20,000 volunteer projects (eg. BOINC-based) assume that computers. The gateway currently supports volunteers trust project applications and three molecular docking scenarios for virtual therefore use digitally signed executables in screening and random blind docking. No order to prevent distribution of malicious code. speciﬁc computing expertise is required to execute the docking experiments as the Peer-to-peer Desktop Grids that allow any user execution is supported by an intuitive high level to run its own application (eg. based on OurGrid user interface. The gateway was developed or XtremWeb) have low amount of trust within the framework of the European projects between participants and therefore should use EDGI and SCI-BUS. more advanced security mechanisms. The most widely used of such mechanisms is sandboxing Links by running untrusted executable inside a restricted environment or a virtual machine. This P-GRADE website mechanism introduces additional management http://portal.p-grade.hu/ and performance overhead but ensures that the host system will not be damaged by executable GUSE website from Desktop Grid. Substantial efforts have http://www.guse.hu been devoted to lower the additional software OurGrid public portal costs associated with this technology, so today http://portal.ourgrid.org/ that is not an issue anymore. Molecular Docking Science Gateway Another security measure that can prevent https://autodock-portal.sztaki.hu/ untrusted applications from running on Desktop Grid is the use of Application Repositories that provide an authoritative source for trusted and validated applications. Security Issues in desktop grids

There are several security risks associated with Desktop Grids. The most important one is There is a lot of technology and good practices execution of malicious code on donor machines. available to help increase the security level of This issue is taken in account by all major Desktop grids. Desktop Grid technologies. The best practices depend on the amount of trust between donors and application developers/users. The Best Practice document prodcued by IDGF- At the core of security in volunteers Desktop SP provides an extensive explanation on how to grids (crowd computing) is the trust the donors secure Desktop grids. have in the organisation that operates the Desktop Grid and its services. The client/worker Links software should be downloaded from the Grid operator’s portal, the client/worker should be Security Issues connected to the Operator’s Grid server. Once http://boinc.berkeley.edu/trac/wiki/SecurityIssues this is done the very nature of the client/worker software and set-up guarantees it is almost impossible to execute code that is harmful for the donor’s machine.

A user that run programmes on a Desktop Grid the should also put trust in the Desktop Grid operator, but because the donors always have full control on their own machines executing the programmes, they could tamper with the results. There are several mechanisme in Desktop – 27 – used to be able to guarantee a reasonable SLA 2.3 Integration of level for users. Second GBAC provides virtualized environments on top of Desktop Desktop Grids and Other Grids. Its goal is to allow running any Computing application without validation or porting in a customizable environment and in a more secure Infrastructures (sandboxed) way. Another approach has been implemented by the 2.3.1 EDGeS 3G Bridge EDGI project. The idea is to implement small Clouds of always available clusters that can be The Generic Grid-Grid Bridge (3G Bridge) is a used to be able to guarantee a reasonable SLA software component used within the EDGeS level for users. project that provides the core component of the Service Grid - Desktop Grid interoperability Finally, Cloud bursting strategies have been solution. This technology enables one to exploited by peer-to-peer grid systems, in which combine these two types of grid systems in users can buy worker nodes on-demand from order to establish a seamless and vast grid public Cloud providers to momentarily increase resource pool. In particular, the software is used the performance of their applications at low to bridge EGI service grid with the BOINC and costs. XtremWeb desktop grids. Links Links Integrated ARC, Desktop Grid, and Eucalyptus bridge. Policies and setup operations for Grid operators http://edgi-project.eu/documents/10515/50865/EDGID61.pdf http://desktopgridfederation.org/technical-wiki/-/wiki/Main/ SiInvestigating Business-Driven Cloudburst Schedulers for Policies+and+setup+operations+for+Grid+operators E-Science Bag-of-Tasks Applications General information on the EDGeS Bridge http://www.computer.org/portal/web/csdl/doi/10.1109/ CloudCom.2010.67 http://edges-grid.eu/web/edges/57 Integrated ARC, Desktop Grid, and Eucalyptus bridge. http://edgi-project.eu/documents/10515/50865/ EDGID61.pdf 2.3.2 Paths into Clouds Siena initiative

Cloud computing is en vogue. Clouds are http://www.sienainitiative.eu/ important infrastructures or building blocks of Generic BOINC Application Client (GBAC) infrastructures. Hence it is important to http://gbac.sourceforge.net investigate the impact that cloud computing will SpeQuloS - have on Desktop Grid computing. http://graal.ens-lyon.fr/~sdelamar/spequlos/ The integration of and collaboration between Grids and Clouds is investigated in several European projects. The Siena initiative is working together with them to create a Road map on a future e-Infrastructure Road map that includes Grids and Clouds.

One step, explored by for instance LHC@Home and XtremWebHEP-E is to package applications in standardized Virtual machine (VM) and let the Desktop Grid infrastructure handle VM's instead of applications. This opens a whole range of questions on security, stability and validation.

Different approaches are being implemented by the EDGI project. First SpeQuloS provides Quality Of Service (QoS) to Desktop Grids by supplying resources from Infrastructure-as-a- Service Clouds. The idea is to implement small Clouds of always available clusters that can be

– 28 – 3. Communication and 3.1 Available marketing communication channels Communicating and marketing Desktop Grids involves addressing several audiences: 3.1.1 BOINC channels Potential volunteers BOINC is the only Desktop Grid middleware that Scientiﬁc users not only provides software, but an ecosystem of users at all levels around it. System administrators On the message boards of BOINC also Funding and other agencies suggestions for communicating and advertising Press projects can be found.

The EDGeS project conducted a survey about There is regularly a BOINC workshop what people expect from a Desktop Grid. More organised. recently the World Community Grid also If you set up a volunteer desktop grid, it can be conducted an in-depth member study. Findings useful to get it listed on one of the BOINC from these studies can be used as a basis for statistics websites. your communication plan. Links In this chapter we describe three aspects of communicating about Desktop Grids: BOINC developer website Available communication channels http://boinc.berkeley.edu/trac/wiki/WikiStart BOINC message boards This part describes the organisations that already do communication about Desktop Grids. http://ibercivis.eshttp://boinc.berkeley.edu/dev/ BOINC workshops Available marketing and communication tools http://boinc.berkeley.edu/trac/wiki/BoincEvents This part describes the tools, such as web BOINCstats portal, ﬂyers, etc. http://boincstats.com/

Experiences in marketing desktop grids

This part describes a number of examples on 3.1.2 IDGF channel how communication tools are used by some typical Desktop Grids. When you become member of the International Desktop Grid federation, you can benefit from the marketing and communication channels of that organisation. Links First, you apply for IDGF membership by filling Acceptance of Desktop Grid computing amongst SME's out an application form. Once accepted by the and the General Public; Ad Emmen, Leslie Versweyveld, IDGF community and having paid the Stichting AlmereGrid, Almere, The Netherlands membership fee, you can enjoy several services http://www.edges-grid.eu:8080/c/document_library/ that are provided at the IDGF web portal. get_file? p_l_id=11065&folderId=11075&name=DLFE-1027.pdf The International Desktop Grid federation brings http://www.worldcommunitygrid.org/about_us/ together experts from all areas related to viewNewsArticle.do?articleId=323 Desktop Grid computing. The community also includes people who are familiar with integrating desktops into scientific infrastructures. As a member, you are part of this network and can benefit from their expertise.

– 29 – The IDGF portal hosts a library with flyers, 3.1.3 Country and local channels presentations given at conferences and workshops, press material, and technical The IDGF chapters are regional or national documentation. There is also an up-to-date organisation that can be used to communicate news archive. to a specific regional auydience. In addition there are several other organisatiions active. In the tutorial section, you can download the educational material from past tutorials and subscribe to future tutorials. Material from past workshops is available in the events section. Country Desktop Grid support Here you can download presentations given at organisations previous workshops and conferences and Alliance Francophone is a community serving register for upcoming conferences and science through Desktop Grid computing using workshops. the BOINC platform. Alliance Francophone has In the infrastructures section you can find useful some 17.000 members world wide. (of which information on current up-and-running Desktop 13.000 in France). Grid infrastructures as well as applications that Rechenkraft.net e.V. supports the development run on these platforms. of the new Distributed Computing-project The IDGF portal hosts an interactive forum "Constellation" in the aerospace domain and section where you can discuss Desktop Grid asks interested programmers to participate. issues in message boards with the IDGF Participants can support science actively with members and experts from Desktop Grid e.g. their personal computer, mobile phones or projects such as IDGF-SP. Playstation 3. They can help in searching for drugs against cancer and other illnesses. For administrative issues, you can contact the Without prior knowledge, they can take part in IDGF administration. resolving highly complex scientific problems. Or just have a little bit of fun, and have discussions Links in the forum/chat and contribute to one of the many fun projects. International Desktop Grid Federation http://desktopgridfederation.org Ibercivis is a citizen computation platform that allows society to participate on scientific IDGF Membership application research in a direct way and in real time. It is an http://desktopgridfederation.org/membership-application initiative in Spain that intends to involve in IDGF Experts volunteer computation the maximum number of http://desktopgridfederation.org/experts citizens. It uses the calculation capacity of the computer in idle moments to carry out tasks IDGF Downloads derived from an investigation project. Ibercivis http://desktopgridfederation.org/downloads brings the citizens closer to leading IDGF Newsarchive investigations and makes them participant in the http://desktopgridfederation.org/news-archive scientific knowledge generation. At the same time, it provides the scientific community with a IDGF Technical documentation powerful calculation tool. The computer turns http://desktopgridfederation.org/technical-wiki into an open window to science, creating a IDGF Tutorials channel for the direct dialogue between http://desktopgridfederation.org/tutorials researchers and society. Conferences with IDGF presence and IDGF workshops http://desktopgridfederation.org/7 In several regions/countries there are local Infrastructure and application section chapters of IDGF. There you can also find http://desktopgridfederation.org/infrastructures support and communication opportunities for Interactive IDGF Forum section your regions. http://desktopgridfederation.org/forum IDGF administration contact Country Organisation http://desktopgridfederation.org/contact France Alliance Francophone

Germany Rechenkraft e.V.

– 30 – Spain Ibercivis that might otherwise not be completed due to the high cost of the computer infrastructure Hungary IDGF-Chapter required in the absence of a public Grid. As part of the commitment to advancing human welfare, Russia IDGF-chapter all results are in the public domain and made public to the global research community. the Netherlands IDGF-chapter Organisation Location South-East Asia IDGF-chapter

Citizen Cyber Science London Specialized organisations LHC@Home Geneva The Citizen Cyberscience Centre aims to develop citizen cyberscience applications World Community Grid USA targeting research on humanitarian and development issues as well as fundamental science, by co-ordinating collaborative projects between scientists from North and South. It organizes hands-on workshops for scientists in developing countries in order to spread know- Links how about citizen cyberscience and its benefits for their research. The organisation also Alliance Francophone provides on-line educational material about the http://www.boinc-af.org/ research challenges addressed by citizen Citizen Cyber Science cyberscience projects, encouraging greater public participation. The centre is based on an http://citizencyberscience.net/ international partnership. Ibercivis http://ibercivis.es LHC@home is a volunteer computing programme which enables the user to IDGF regional/country chapters contribute idle time on his computer to help http://desktopgridfederation.org/chapters physicists develop and exploit particle LHC@Home accelerators, such as CERN's Large Hadron http://lhcathome.cern.ch/ Collider. Volunteer computing makes sense for the LHC. In particular, volunteer computing is Rechenkraft e.V. good for tasks which need a lot of computing http://rechenkraft.de/ power but relatively little data transfer. In 2004, World Community Grid CERN's IT Department became interested in http://www.worldcommunitygrid.org/ evaluating the sort of technology that is used by volunteer computing projects like SETI@home. LHC@home became the overall title for these 3.2 Available marketing efforts, and a programme called SixTrack, which simulates particles travelling around the LHC to and communication study the stability of their orbits, became the first application to be tested. It was chosen because tools it can fit on a single PC and requires relatively little input or output, but a lot of processing power. To market Desktop Grids and communicate to different audiences, you can use a lot of tools. World Community Grid brings people together Remember to choose a tool that fits the from across the globe to create the largest non- audience. For instance if you want to profit computing Grid benefiting humanity. It communicate to citizens in the Netherlands on a does this by pooling surplus computer social media site, Hyves may be a better choice processing power. Innovation combined with than Facebook. visionary scientific research and large-scale volunteerism can help make the planet smarter. The tools here are listed with a Volunteer Success depends on like-minded individuals. Desktop Grid in mind. World Community Grid is making technology available only to public and not-for-profit Portal organisations to use in humanitarian research Message boards – 31 – Flyers good to have some flyers at hand for distribution. Press The flyer should mention the goal of the Participation in events Desktop Grid, how to participate, and a general url at a minimum. Social media 3.2.4 Press Certification services Try to build a good relationship with the press. Do not just send press releases: they will go in the waste bin if that is all you do - unless your 3.2.1 Portal Desktop Grid does contribute to major scientific A portal, a website, is the most important breakthrough. communication tool to keep in touch with 3.2.5 Participation in events volunteers. You can set up a portal using any of the popular content management systems. We From a promotion point of view it is very like to use Liferay, but Drupal, Joomla, and important to regularly organise tutorials and many others are just as good (almost). When workshops, preferably in joint collaboration with you are setting up a BOINC project it comes established Grid and Cloud computing with a web site out-of-the-box. conferences. You can also submit papers in order to give a presentation about your Desktop Always keep the portal up to date. Write Grid activities at this type of conferences. It will content targeted at your audience, preferably in heighten your visibility and extend your their own language. Provide information on audience. If the conference has an exhibition, what applications you are running and on who you can try to rent a booth. This will give you an you are (people need to trust you before they opportunity to talk with interested parties and to donate computing time). present your Desktop Grid computing project by The portal should also give access to the means of posters. volunteer's contribution in terms of computing 3.2.6 Social Media power to the Grid. Could be this needs to be password protected. Feedback is important: Many people use social media today. Also new people want to know what is done with their social media come and “old” social media computing time. disappear. Hence it is useful to choose the right ones. An important thing to remember is that 3.2.2 Message boards you do not really have much influence on what Message boards (Discussion Forums) can be is happening on social media. Not even with an important way to communicate with what happens with your information or your volunteers. All modern content management postings. And it is always good to remember systems have discussion forum modules. that even today the majority of people are not on any social medium. A message board in general has to be moderated. The least you need to do is to Video sites, like YouTube or Vimeo can be used answer questions from the volunteers. to upload video’s about your project, users, or applications. Vimeo is considered to be more There can also be disadvantages to using “professional” while YouTube has a bigger message boards: some people tend to reach. Do not forget to link back to your own dominate discussions. Savvy volunteers can be website. very useful to discuss with on a technical level, but when laymen see the discussion (in English Twitter is used typically for short news and not their native language) they can think messages or to “talk” to other people. There is “this is not for me”. Try to clearly separate the not a big volunteer or desktop Grid tweeting discussion chapters to minimize the risk. community yet. But you can follow, for instance, IDGF and the World Community Grid on Twitter 3.2.3 Flyers Profile sites, like FaceBook, LinkedIn, and Yes, the good old paper flyers. Of course you MySpace can be used to form and participate in put a PDF file in the portal too, but if you meet groups. So they can be useful to come in someone at an event or meeting, it is always contact with other people or new volunteers.

– 32 – You can also post photos or texts to it. LinkedIN is more aimed at professionals than the other 3.3 Experiences in sites. marketing desktop grids Wikipedia is a kind of encyclopedia, so it is In this section we show some examples how useful for older projects, or mature applications. different Desktop grids did use different According to Wikipedia policy you should communication channels in a communication always be able to refer to a multitude of mix to achieve the goal of getting the message scientific resources, newspapers, etc. And accross to specific audiences. We give a variety remember once your text is copied on of examples as a source of inspiration. Wikipedia, it is Wikipedia that has the copyright. Text for Wikipedia has to be “new”. And it was 3.3.1 CityGrid example: AlmereGrid good practice not to write too much about Background yourself, or your own company. AlmereGrid was the first CityGrid in the world. As with your own portal, websites or flyers, After half a decade, AlmereGrid is Social media have to be maintained. This takes internationally recognized for its pioneering time, and when your presence is getting more efforts, and has been and still is, a major partner prominent, it takes more and more time. in several European Research & Development projects.

Links AlmereGrid’s main mission is to enable citizens to support science. The AlmereGrid Volunteer IDGF on twitter Computing Grid is running scientific applications. People in the city of Almere can http://twitter.com//gridfederation donate their idle computing time. IDGF Vimeo video channel http://vimeo.com/idgf It also supports scientific experiments in Grid and Cloud computing that need a distributed Renderfam.fi introduction on YouTube computing test . http://www.youtube.com/watch?v=4VUWrZRCtI8 Rechnenkraft Wikipedia page AlmereGrid has two main Volunteer Desktop Grids that each come in two flavours. The most http://de.wikipedia.org/wiki/Rechenkraft.net important Grid is the BOINC based production Grid. BOINC is the technology used. BOINC is a 3.2.7 Certified services very solid and widely used Desktop Grid Security is always an issue in volunteer Desktop middleware software. Apart from the production Grids. Will the application not harm my Grid, AlmereGrid also operates a smaller Test machine? “Is it not misused?” are often heard BOINC based Desktop Grid. questions. Providing certified services is a way The other middleware flavour is XtremWeb. to handle these issues. IDGF offers certification AlmereGrid also operates a production & test services, which allows you to get a certificate for Grid based on XtremWeb. In total about 3500 instance for applications you do provide. computers from volunteers are connected to AlmereGrid.

– 33 – At the international level AlmereGrid has what AlmereGrid is doing to help science. This assisted in setting up an International Desktop also helps in attracting new scientiﬁc Grid Federation (IDGF) for international applications. collaboration between peers. Mentions in international press are important for the new and existing international collaborations and the international projects. Communication message

AlmereGrid was set up to advance science by the use of innovative technology. The core was, Collaboration with other projects and still is, setting up a scientific Grid based on volunteer computing. When over the years, new AlmereGrid participated in several European technologies come and go, AlmereGrid will Desktop Grid computing projects including incorporate the most useful ones, and BEinGRID, a community back-up service for contribute to the research in advanced small businesses; EDGeS, Enabling Desktop technologies by providing test beds. Grids for eScience; EDGI, European Desktop Grid Initiative, expanding into the Cloud; Communication tools used DEGISCO, Desktop Grids for International Scientific Collaboration; and Contrail, Open As an example we provide several Computing Infrastructure for Elastic Services. communication tools we used in the past or still The objective is to create a European-wide are using. . Desktop Grid operators community and to have an impact on the European politic level. Event The AlmereGrid Desktop Experience AlmereGrid organized the Aexact Initiative, Workshops aimed to exchange experience which was titled "From Abacus to Grid". This between Desktop Grid operators, partner initiative was aimed at citizens to involve them organisations, application developers and in the use of Desktop Grid computing to solve scientific users. scientific questions by organizing an exhibition with practical examples of Desktop Grid The scientific part of these workshops computing to solve scientific problems and by concentrated on Desktop Grid Applications for organizing a contest in which participants were eScience and eBusiness. Scientific proceedings asked to provide a scientific problem which have been issued from these workshops. could be solved by using Desktop Grid computing. Currently AlmereGrid is involved in the Dutch Health Hub and providing a testbed for the The exhibition in the museum “Kunstlokaal Cloud computing project Contrail. Station Muziekwijk” was a first step towards the vision of bringing more science to the city. The Desktop Grid Demonstrator Corner Station Muziekwijk is an old railway station old for Almere, the station dates from the mid - that AlmereGrid assembled a “Grid Demonstration was partially put out of service. Corner” consisting of a number of Raspberry Pi computers. It also includes a small affordable AlmereGrid also visited children in schools. Video Wall powered by Raspberry Pis. This is a Children had to provide difficult questions which an inexpensive, yet very good way to could be solved through Desktop Grid demonstrate how Grid or parallel computing in computing. general works. The demonstration set can also easily be moved to schools or other events for Press release and press contacts explanation. Furthermore it can be used in training sessions. Regional, national and international press briefings for written and audiovisual press media have been issued. Links Mentions at regional level are important to inform the local citizens about AlmereGrid and AlmereGrid the opportunity to donate computing time. http://almeregrid.nl Mentions at national level are of interest for the Aexact same reasons and to show on a national level

– 34 – http://almere2018.org 3.3.2 Public Grid: EDGeS@Home Aexact press release Background http://almere2018.org:8086/almere2018/vmp/repository/ n54.pdf EDGeS@Home was set up to support Dutch regional television coverage applications that also run on the large European http://www.omroepﬂevoland.nl/nieuws/recent? service Grid: EGEE (now EGI). NewsKey=5479D73C07E37DA0C12575A5004E84D5 The ﬁrst application that was used and Dutch Health Hub IPC promoted was a Nuclear Fusion application http://dhh-ipc.nl called ISDEP. EDGeS@Home runs on both BOINC and XtremWeb middleware. Volunteers can choose.

Communication message

Help solving the world's energy crisis is appealing to many citizens. Hence we decided to focus our initial communication around it.

Communication tools used

A complete communication mix was used:

Launch event

Press release and press contacts

Targeted portal

Flyer

Collaboration with scientists

The Launch event was the focal point of the communication. All other communication tools were ready at that time. It also offered an opportunity to talk to the press about the news. At the event we invited scientists to talk about the Fusion research, experts to talk about EDGeS@Home and the technology, and similar projects to show there is a choice of scientiﬁc projects one can choose from.

A press release was made available in many languages (EDGeS@Home is international). Several media were contacted directly in a follow up.

Links

EDGeS@Home http://edgesathome.org Flyer http://www.edges-grid.eu:8080/c/document_library/ get_ﬁle? p_l_id=29093&folderId=73880&name=DLFE-1618.pdf Launch event http://edges-grid.eu/web/userforum/4thtraining Press release http://www.edges-grid.eu/web/edges/56

– 35 – 3.3.3 A national Desktop Grid: Ibercivis Ibercivis brings the citizens closer to leading investigations and makes them participant in the Background generation of scientific knowledge. At the same time, it provides the scientific community with a Ibercivis was developed in Spain with the powerful calculation tool. The computer turns cooperation of the Institute of Biocomputation into an open window to science, creating a and Physics of Complex Systems at the channel for the direct dialogue between University of Zaragoza, CIEMAT, CETA- researchers and society. CIEMAT, Spanish National Research Council (CSIC) and RedIris. The project tasks were Communication tools used distributed in different scientific-technological centres in Spain with the aim of creating a new Ibercivis has launched a monthly publication on platform for science-based volunteer distributed research carried out in Ibercivis. The format of computing. The project is an European-made these publications is an interview with a development of the USA-based SETI@home principal researcher for each application. and Berkeley Open Infrastructure for Network Computing (BOINC) distributed computing The Ibercivis project has Teaching Units whose concepts. Ibercivis had the University of main purpose consists in bringing scientific Zaragoza-based distributed computing project research to secondary schools. called Zivis as its predecessor in 2007, and started to work in 2008 in Spain. Zivis was a The project organises workshops at regular pioneer local distributed computing project times. For example, researchers and funded by the ayuntamiento (city council) of the technicians from different schools as the city of Zaragoza. universities of Extremadura, Granada, Autónoma de Madrid and the CSIC, among The Ibercivis infrastructure can be used for others, attended on 8th and 9th September calculating applications ranging from fields like 2010 the I Ibercivis Workshop, held at the nuclear fusion and protein folding to materials Institute of Biocomputing and Physics System simulations. In July 2009, the Ibercivis platform Complex (BIFI). The purpose of this meeting was extended to Portugal assuming a was to bring positions between the three pillars transnational Iberian dimension, following the on which it is based Ibercivis: research, agreement to this effect signed by the development and communication, creating governments of both countries in the Luso- workshops around different development Spanish Summit January 2009 in Zamora, proposals and communication. Spain. Portuguese institutions affiliated with Ibercivis include the Ministry of Science, Technology and Higher Education, the Centre Links for Neuroscience and Cell Biology at the University of Coimbra, and the LIP experimental Ibercivis high energy physics laboratory. http://www.ibercivis.es Ibercivis is not a temporary project - unlike Didactical material many BOINC-based volunteer computing http://www.ibercivis.es/index.php? projects, so it will be possible to submit module=public§ion=channels&action=view&id_channe applications indefinitely. In addition, Ibercivis is l=4&id_subchannel=38 designed to run not just one but several Monthly publication applications belonging to different disciplines, in a manner similar to the IBM-funded World http://www.ibercivis.es/content/modules/webboard/channels/ files/00000064_df6809f06827594bbc84e7c0dda36c4f.pdf Community Grid. Ibercivis Workshop Communication message http://www.ibercivis.es/index.php? module=public§ion=channels&action=view&id_channe Ibercivis is a joint scientific collaboration of the l=4&id_subchannel=135 Portuguese and Spanish governments, but is open to citizens and scientists of other countries outside the Iberian Peninsula who wish to join the initiative. It uses the calculation capacity of the computer in idle moments to carry out tasks derived from an investigation project.

– 36 – 3.3.3 A Grid for charity: Charity engine really can be the greenest supercomputer on the planet - especially now that BOINC on Background Android has just been released. On mobile chips, the energy efficiency of a volunteer grid is Based on the proven idea of volunteer absolutely unbeatable. computing, Charity Engine harnesses the vast, wasted capacity of the world’s home and office PCs - a technically simple feat, but one which requires the permission of the world’s PC Communication tools used owners first. The Charity Engine app gains that permission by converting the donated Portal and through the charities. computing into real money for top international charities - and also, every so often, a substantial cash prize for one lucky volunteer. Links Profits are shared equally between Charity Engine, the users and the charities. (Companies Charity Engine can also choose to share the profits from their PCs instead of being in the prize draws.) The http://charityengine.com Charity Engine Desktop Grid currently has over 20,000 PCs in over 100 countries.

Communication message

Not only is Charity Engine the cheapest and potentially most powerful computing platform in the world, it is also the most environmentally friendly. For a start, it requires no new hardware. That’s half the energy costs of any new system. Nor does it require vast new billion-dollar facilities to house it, staffed and air- conditioned. In fact, it needs virtually no net cooling at all. Unlike datacentres and supercomputers, we don’t stress the components (our volunteer’s PCs). A typical laptop CPU can cruise at 60% activity for just another 4-10 Watts compared to idling, so that is our default setting. No excessive heat, no noisy fans, no surprise bills - and the grid effectively gains ~60% of a full-power CPU for less energy than charging a smartphone. This is the key to Charity Engine’s incredible efﬁciency: using machines which already exist and are already switched on.

GPUs do use more power to run Charity Engine (+30W, on average), but they are actually even more efﬁcient than CPUs. We can then reduce carbon costs even further by using a technique we’re calling the ‘Winternet’ - something that only a global grid can do. Supercomputers and data-centres have one problem they cannot escape: megawatts of heat. Fortunately for us, it’s always cold somewhere - and everywhere is cooler at night. Charity Engine will therefore keep shifting data to the globe’s coldest PCs, where the extra heat is actually a welcome bonus and not just wasted on air conditioning. It

– 37 – issues; the matrix will have only 3 rows and 4. Legal and columns; resource owners, Desktop Grid service provider and the merged data owners/ organisational application provided/data provider/end-user. information This is the case, when the IT department creates a public volunteer Desktop Grid for the research department in order to run their own The following matrix summarizes the most application with the own data of the research important topics among the key players of department. Desktop Grids in details. In the most simplest case when the target is to This matrix allows us to discover several crucial establish a local Desktop Grid, which is issues in complex cases when the policies must operated locally and used inside a research lab be harmonized e.g. with Service Grids. by itself using their own data and application However, please note that in academic or provided by the local developers from the local university environments several roles are research lab; the legal issues can be maximally merged into one key player quite frequently. For eliminated. example, the end-user provides the data and The matrix intends to cover the situation when the application with his/her own data. It the desktop grid or any other service is provided simpliﬁes the legal situation and organisation for free.

resource data Desktop application data end-user owners owners Grid service provider provider providers (storage) resource (through RO grants (through (through (through owners (RO) granting access to granting granting granting access to DG his/her own access to DG access to DG access to service resources by service service DG service provider) joining a DG provider) provider) provider) project data owners (through DO grants DO grants DO stores DO allows (DO) validation access to access to (anonymized) the process of (anonymized) (anonymized) data in exploitation EADM) data in data during storage of DG of data production development provider Desktop Grid No warranty No warranty DGSP DGSP grants DGSP service by the DGSP by the DGSP provides access during provides providers but validate but apply support and development access with (DGSP) applications best host the and in SLA or with with EADM, practices/ application production best effort and protect technologies base with resource and privacy owners recommend privacy anonymizatio statement n application (through AP accepts AP AP grants AP uses providers validation the data collaborates in access to Grid- (AP) process of owner policy testing and stored data friendly SW EADM) validation license (or proprietary software) and hold IPR data provider (through DP ensures DP ensures DP ensures DP grants (storage, DP) validation privacy and secure access secure access access to process of confidentiality from Grid from apps stored data EADM) end-users (through E-U accepts E-U accepts E-U accepts E-U accepts (E-U) accepted the data AUP from the SW licensing policy of data AUP) owner policy grid service policies (or provider provider proprietary software) – 38 – resource owners (RO): The owner of the Desktop Grids, Desktop Grid providers, data computing re-source (i.e. desktop computer) and resource owners of Desktop Grids. provided to the Desktop Grid data owners (DO): The owner of the input and 4.1 Privacy issues output data that will be processed by the In order to ensure data anonymization data application that uses the Desktop Grid. There protection best practices and standards must be might be different owners of the input and followed by several key players. It includes the output data; it is out of the scope of this European Directive 95/46/EC on the protection document. of individuals, e.g. resource providers for volunteer Desktop Grids, with regard to the Desktop Grid service providers (DGSP): The processing of their personal data, and entity that manages de Desktop Grid infrastruc- concerning the data privacy the various service ture to run an application. This application has providers and data owners must follow the been registered to the Desktop Grid after a national or local guidelines as well. certification and/or validation process using EADM methodology Three major cases can be distinguished: application provider(AP): The person/entity Between End-user / Desktop Grid service who created (programmed) the application that providers: Desktop Grid service providers are will run in a DG. It is also the person/entity who responsible to handle the private data (email manages the application updates, the certifica- addresses, names, IP addresses) of end users tion/validation process and provides updates to according to the above described policies. the DesGtop grid repository Between Resource owner / Desktop Grid data provider (storage, DP): The entity that service provider: Desktop Grid service providers stores the input data that will be processed by are responsible to handle the private data (e.g. the application that runs on a Desktop Grid, or email addresses, names, IP addresses) of that stores the intermediate of final results of resource owners in case of volunteer Desktop this processing Grids according to the general policies. end-user (E-U): The entity who benefits from Between Data owners / data provider the results of the application execution on the (storage) or any further players: As a thumb Desktop Grid. rule, data owners must be responsible for anonymization of the input data before Links delivering them to the data provider or any other players later. Grid and Cloud Computing: A Business Perspective on Technology and Applications http://www.springer.com/business+%26+management/ business+information+systems/book/978-3-642-05192-0 (chapter 7) 4.2 Licensing The Desktop Grid middleware solutions and Memorandum of Understanding between EGEE and EDGeS tools, which are supported and recommended by IDGF, are open-source and free software http://www.edges-grid.eu/c/document_library/get_file? packages mainly from the EDGI and DEGISCO folderId=36519&name=DLFE-1212.pdf project partners. Memorandum of Understanding on Collaboration (between EDGeS and SEE-GRID-SCI) The ported and available applications from the http://edges-grid.eu/web/edges/37 repositories. and data from the data providers Memorandum of Understanding between EGI-InSPIREand must have Grid-friendly licensing as well. EDGI However, the new applications can be either https://documents.egi.eu/public/RetrieveFile?docid=496&v proprietary or open-source ones. For local ersion=7&filename=EGI-MOU-EDGI.pdf usage the proprietary application code is not a major barrier but volunteer Grids mostly require open-source applications in order to build and keep the trust. Concerning the legal issues we focus on the key players with special duties (see blue cells) and rights (see yellow cells); end-user of

– 39 – have valid certificate issued by a Certificate 4.3 Misuse Authority: The candidate applications which will use a volunteer Desktop Grid system for their obviously, when submitting jobs from a execution, must be validated using a test Service Grid through the 3G Bridge by a Service infrastructure based on IADM. In case of local Grid end-user desktop Grids, this phase is optional. The validation process ensures that the if the end-user and the application provider infrastructure could not be used for illegal are the same, and he/she intends to execute computations. The introduction of Acceptable the jobs on a Service Grid through the 3G Use Policy helps ensure this requirement, too. bridge 4.6 IPR As a recommendation; the intellectual property 4.4 Warranties rights generated by a Party under a desktop The validation process in the IADM does not Grid collaboration shall be the property of that allow harmful jobs, which could destroy part or Party who shall be free to protect, transfer and the whole infrastructure. In a part of the use such Intellectual Property Rights as it validation process the application is tested in a deems fit. Notwithstanding the foregoing each limited test environment, not the whole Party shall grant the other a non exclusive infrastructure. Despite of the best efforts in the royalty free, perpetual license to use the IADM, due to the nature of the applied software Intellectual Property Rights generated by it stack (e.g. using BOINC distribution) which are under the DG collaboration for use within its provided without warranty and 'as is', the project or for the exploitation the results thereof. appropriate disclaimer must be placed on each Such license shall include the right to document, software component, and service sublicense the entities involved in the project. entry points.

An example in case of software: 4.7 Interoperation with

Disclaimer of Warranty Other Grids and Clouds Since the number of service grids in production THIS SOFTWARE AND THE ACCOMPANYING is rather limited and the service grids have FILES ARE DISTRIBUTED "AS IS" AND various policies; connecting the new Service WITHOUT WARRANTIES OF ANY KIND, AND Grid to a Desktop Grid requires negotiation THE IMPLIED WARRANTIES OF between the Grid operators. This can be written MERCHANTABILITY OR FITNESS FOR A down in a Memorandum of Understanding. PARTICULAR PURPOSE ARE HEREBY DISCLAIMED. ANY STATUTORY WARRANTY The major topics that must be covered are the OF NON-INFRINGEMENT IS ALSO following with some recommended resolutions: DISCLAIMED. responsibility: Each operator will maintain full responsibility for its own activities

IPR: proprietary rights relating to the results of activities undertaken by the collaborators under the MoU shall be jointly held by the 4.5 Security based on projects

Grid certificates warranty: no warranty Obtaining, using, and renewing a certificate by an average Grid user is very often a licenses and access rights: a project grants cumbersome process especially for scientific the other project a free, irrevocable and end-users without solid IT skills, and in some perpetual license to use such rights for joint IPR ICPC countries (where the CA does not functioning) as well. The solution of most Desktop Grids follows an approach which lowers the barrier to entry in using Desktop Collaboration with EGI to support European Grids and also allows the user to not posses a scientists certificate. Only in some cases the user must

– 40 – What EGI members can do with Desktop Grids? Icrease the number of heavily--used EGI Desktop Grids are already available for NGIs applications on the integrated Desktop Grid and VOs thanks to the long-term collaboration infra-structure with focus on widespread tools, between the EGI community, MTA SZTAKI from solutions and approach the Hungarian NGI, and other key members of the International Desktop Grid Federation. The Improve existing documentation with updated result was a bridging technology that provides roadmaps, training materials, and manuals. interoperability between the EGI core infrastructure and Desktop Grid solution at job Complete the ﬁnal remaining steps for full management, monitoring, and accounting integration, concerning for example support levels. tools

Why should EGI scientists consider Desktop Find joint EGI/IDGF champions Grid resources? Several reasons: The Virtual Team will support NGIs and VOs Integration - seamless integration between that wish to try running their current applications EGI services and Desktop Grids means that in this combined infrastructure, and if necessary same same jobs, inputs, parameters, and ﬁne-tune them taking into consideration the submission tools can be used by the nature of Desktop Grids. scientists thanks to virtualisation technologies The immediate goal of the Virtual Team is to Visibility - the scientiﬁc and technical goals increase the size of the community who speaks and achievements of your NGI/VO will reach not only Grid but also Desktop Grid, and thou-sands of citizens, with further support it with more interconnected VOs, more opportunities for stronger engagement with applications and better materials. the volunteers and the society in general In the long-term, we hope to increase the Sustainability - involving a worldwide number of new user communities who start community of individual resource providers using EGI thanks to the alternative approach of will contribute to your sustainability or Social Desktop Grids. Responsibility Plan

What will scientist get? The most important beneﬁt is extra computing resources. MTA SZTAKI operates two large-scale volunteer Desktop Grids for EGI users through production-level bridges:

SZTAKI Desktop Grid dedicated to NGI_HU users, comprises 100,000 registered computers with 600 million normalized CPU hours delivered since 2005.

EDGeS@home is allocated for EGI users and has 24,000 registered computers and has delivered about 300 million normalised CPU hours since 2009.

Scientists are encouraged to join the Virtual Team as a next step.

IDGF/EGI als want to encourage NGIs and VOs to join a new Virtual Team to promote Desktop Grid Solutions (DG VT), where they will:

Promote Desktop Grid-related technologies in the EGI community

Promote the use of available bridged Desktop Grid resources (e.g. EDGeS@home) by more VOs

– 41 – pay for energy – Desktop Grid computing is not 5. Green Desktop Grids free of costs.

Operators and users (=scientists) of Desktop Desktop Grids are the real Green Grids Grids have the responsibility to make wise use of this energy. Key advantage of Desktop Grids is the minimal power density compared to conventional data centres. Typically, PCs participating in Desktop Grids in Europe are not hosted in air- conditioned environments. Without the energy burden of air-conditions, Desktop Grid are 5.1.2 Selection of Green IT arguments: CO2 intrinsically “greener” than data centre based footprint and the energy mix clusters and thereof built Service-Grids. One of the originally formulated Green IT aims Also, since most Desktop Grids are based on was to reduce CO2 footprint of IT activities. A opportunistically harvesting idle cycles, they can broader formulation targets the reduction of be associated to low power consumption modes energy consumption in general and especially that minimise the energy consumed when the thermal emissions in metropolitan areas, both resources are idle and there is nothing to be impacting processed in the Desktop Grid. a) global climate The term “Green-Desktop Grids” has been introduced along with the ﬁrst hype on Green- b) local (micro) climate and Computing”. Here we want to discuss why wise c) human quality of life. use of energy is highly relevant for Desktop Grids and how energy saving methodologies Production of CO2 and accordingly the can be applied. reduction of CO2 footprint are difﬁcult to measure from the perspective of a concrete IT 5.1 Relevance of Green activity like computation. Even if energy consumption as such is accounted for, it Desktop Grids depends on the local energy mix how much CO2 this is equivalent to.

5.1.1 Responsibility for real costs: Wise or The electrical energy mix, the combination of Waste use of energy electrical energy sources, depends on national speciﬁcs, see table. E.g. Denmark produces Desktop Grids gather donor machines up into 25% of its electricity consumption from wind – the 100-thousands per project. sometimes up to 150% (when strong winds produce more electricity than Denmark needs) Let us assume the average modern PC uses causing negative energy prices at the spot about 300 Watts full power. That individual market. Neither the single PC owner who power consumption times those 100.000 contributes compute time to a scientiﬁc project machines result in tremendous 30MW of power nor the operator of the Desktop Grid server that consumption. A typical project may consist of offers the respective workload item is in control 200.000 jobs with an average runtime of 2 of the energy source used to produce electricity hours. Also for example chose a typical end consumed by the computational effort. customer price of 25 Eurocts/kWh for electricity: 5.1.3 € - a metric for Green IT success 200.000 Jobs x 2 hours x 300 Watts = 120MWh for one campaign In order to measure the effectiveness of energy saving policies and methods, we need to 120MWh x 25 Eurocts/kWh = 30.000 Euro introduce a metric that can be “metered”. The contributed by donors for one campaign or 15 obvious advantage of “kWh” as the base metric Eurocts/job. for Green IT is the simplicity of measurement: electricity is metered everywhere. Different from Now, asking contributors for their machines and computer centres and conventional Service- compute time actually means asking them to Grids, policies and methods are applied and

– 42 – executed in Desktop Grids only by the volunteer CPU speed steps effort of the resource contributor. Switch off or put to sleep unused Example: a donor located in France will make machines use of energy produced by renewals + nuclear = ~90%. Only 10% of his or her energy All those recommended methodologies are consumption is connected to CO2 production. based on the fundamental principles of Desktop CO2 footprint for computation in France is mostly Grids: the volunteer, the donor stays in control of irrelevant while we agree that saving energy is her/his resources. It is up to the donor to accept nevertheless of high importance. advice on e.g. energy-aware conﬁgurations.

As success metric for Green IT, the translation We do not even discuss potential remote-control into cost, into money, is helpful to connect to of donor machines and do not intend to provide business considerations and propel motivation. technology means to centrally enforce certain With € (for kWh) as metric, contributors can settings or behaviour. relate their choice of workload and policy- compliance to the personal electricity bill: Green The implementation of the Green Desktop Grid Desktop Grids help the planet and your budget! methodologies takes place by information presented to the donor with the invitation to 5.1.4 The eleventh commandment: You shall chose her/his preferred options to optimize not waste energy energy-efﬁciency.

The Green-Methodologies describe a number of The only exception are Desktop Grids that are measures to make best possible use of energy. centrally managed, like e.g. “The University of Westminster Local Desktop Grid” described The energy used for computation does not below. Here the owner, the university, is the always result in waste-heat but sometimes in well “volunteer” and implements centralized received dual use when heating is based on management means for all their resources, also electricity anyway and actual ambient capable to adjust Desktop Grid-Client temperature makes additional heating necessary. configurations. Companies or institutions considering to contribute groups of resources will In most situations, consumed energy equals use their Windows-clients management facilities waste-heat and good consideration should be to achieve this level of control. taken to assure responsible use of it. In consequence, we do not see the need (yet - unless you educate us differently) to provide 5.2 Green Desktop Grid remote-control of donor machines as native Methodologies feature of Desktop Grid technology. The short-list of 7 methodologies is a collection of 5.2.1 The balance between complexity and best practices, techniques and policies. Some of Green features these methodologies are of technical nature, some of organisational. Some of them are readily Managing a Desktop Grid in a “greener” way implemented, some still object of research. adds complexity to the already not trivial base functionality. Adding complexity is acceptable Organisational methodologies: only below a certain threshold: The aim to save energy shall not cause technological instabilities Exploitation of natural ambient or excess operational difficulties. To which extend conditions these difficulties occur, depends primarily on the Cool strategy: avoid air-condition use used Desktop Grid technology. Middleware like BOINC does not allow to steer and control the Technological methodologies donor's desktop machine. Consequently, switching off PCs must make use of extensions Energy profiling of applications or external techniques which results in additional complexity. Other Desktop Grid middleware like Ambient metrics based Green OurGrid has this feature built-in and can even optimization select between suspend ad hibernate modes.

Time-of-day and weather dependent When planning a new Desktop Grid, the energy tariffs appropriate balance between preferred features

– 43 – and resulting complexity will have to be the heat pump can utilize to get rid of the heat. considered. The prime advice to configure Desktop Grids: avoid air-condition use! Selection criteria for the “maximum temperature” as described above could be that temperature which would just not 5.2.2 Powerful organisational methodologies yet trigger the start of the local air-condition. Applications could be chosen by the contributor Organisational methodologies can be which are indicated to produce a certain heat/ implemented mostly independent of the chosen time only (compare “Energy profiling of technology, still they provide excellent results. applications”), by keeping the ambient Both organisational methodologies make use of temperature in acceptable range. the fact, that the Desktop Grid donor frequently Energy profiling of applications is sufficiently educated and interested to follow instructions. To understand the potential of energy profiling, we need to consider that energy consumption Exploitation of natural ambient conditions and by this, heat production of computers is When advertising a new scientific challenge, massively influenced by the workload. a new project, one may add recommendations As listed in great detail for thousands of to donors from climate-wise suitable regions. computer products by Energy-Star, typically the energy consumption varies for a factor of about Different locations yield different climates 2 to 4 between idle and fully loaded. In other Kazakhstan, Amaty words: when a PC is idling with some 50 Watts, http://worldweather.wmo.int/070/c00152.htm Desktop Grid activity may induce beyond 200 Watts of energy consumption and heat Russia, Moscow generation, making use of all cores in the box. http://worldweather.wmo.int/107/c00206.htm Hungary, Budapest A pure CPU bound application would possibly http://worldweather.wmo.int/017/c00060.htm be able to run this CPU (and possibly even all 4 or 6 or 12 cores) to 100%, pushing the total Denmark, Copenhagen power consumption to the maximum. http://worldweather.wmo.int/173/c00190.htm Spain, Zaragoza Another application will work with some I/O to RAM or HDD, effectively keeping the power and http://worldweather.wmo.int/083/c01240.htm temperature of the PC lower.

Different applications and codes consume more or less CPU at any given time, resulting in By this targeted recruiting, still the donor different energy consumption per time interval; decides what to subscribe to. By adding they behave differently in raising machine and information on assumed power consumption per ambient temperature. Accordingly applications compute task and resulting heat production, can be classiﬁed with a heat index as +, ++ and donors can take informed decision on whether +++ for example. The +++ index marks an to accept this workload type – and many of application that makes maximum use of a given them will. machine, is raising its temperature, but ﬁnishes the computation quickly. This behaviour may total in less “energy consumed/computation” Cool strategy: avoid air-condition use than the application which creates less heat/ time. Still heat/time is an important parameter Desktop Grids are the real Green Grids: lower from a green operations point of view. energy density than clusters results in less energy wasted for cooling. However, this may not longer be true if air-conditions are used to assure proper operation of Desktops. Principles Time-of-day and weather dependent energy of cooling: Energy consumption by air- tariffs conditions range from 30% to >200% of the The value of electrical energy is usually energy dissipated by the IT device (payload), changing according to the conditions of depending on the cool-reservoir temperature

– 44 – generation as well as by changing consumption. The cost of the energy consumption, and Accordingly, the tariffs for electricity are changing. The cost associated to the personnel needed to maintain the infrastructure. A typical profile in Central Europe would be 16 Eurocts/kWh during night, 23 Eurocts/kWh in Several factors are missing in this estimate, daytime and 80 Eurocts at noon +/- 1hour(private including: customer prices). Other regions with different climate, style-of living (=time to cook) and the cost of adapting the building to house the different energy production mix have completely cluster and the cooling system. These costs are different time-of-day profiles. too site specific but can be very significant.

Typically, the peak usage is covered by Oil and the cost and energy consumption of the Gas turbine power plants. Reducing Internet communication. The bandwidth required computational workload during those peak times to run a large volunteer computing network may save CO2 directly. be substantial. Also, the power consumption of the network equipment between the research In the example given, Denmark, the wind center and the volunteer is very hard to estimate. produced electricity is predictable for a couple of hours (up to ~1 day) ahead. An accordingly updated Desktop Grid project website could The cost of a Desktop Grid or a data centre is advice the donors to “contribute now” as his highly dependent on the energy consumption. energy price would be low at this point in time. We identified that there are three main To improve the energy cost situation and to take configurations that are used in Desktop grids: advantage of excess Green electricity, advice 1. Dedicated: The participating PC is turned on could be given to contributors how to configure to run only Desktop Grid applications. In this their Desktop Grid-Clients to prefer workload case 100 % of the energy used by the during low tariff times. machine will be accounted for.

2. Intermittent: The PC periodically runs Desktop Grid applications, when the machine 5.3 Measurements and is considered idle. When the machine is used for other purposes, no Desktop Grid comparisons with applications run. For example, it is possible to configure BOINC such that it only runs standard data centres computations when the user is absent or at night. In this case only the energy during the periods when the scientific applications run IDGF-SP partner and IDGF member Sony CSL should be taken into account. evaluated the cost of a Desktop Grid compared to the cost of a traditional data centre. The 3. Background: The scientific computation comparison takes as primary target, a small takes places concurrently with other research group that has to invest in a computing activities, for example, when the user is infrastructure to run CPU-intensive scientific editing texts, or when a server is running a models. web site. In this case, we must separate the energy used by the Desktop Grid from the Two solutions are compared, a dedicated energy required to run the other activities. computing cluster and a volunteer computing network. The comparison starts with a given In the current analysis for Desktop Grids, we did computing performance that is the same for both consider only the dedicated set-up and the solutions. This performance in expressed using background-computing set-up. the CFP2006 benchmark as a unit. We are interested in the range of possibilities: for The evaluation includes: each of the three solutions, the comparison therefore makes a low-end estimate and high- The purchase cost of the equipment, end estimate. Both solutions present many configuration options (best-of-class machines The installation cost, versus best-bargain machines, cost of salaries, and so on). The details of the parameters are

– 45 – discussed in detail on the Low energy BOINC than when the machines are dedicated to the wiki computation.

In the study, we estimated the cost for a fairly Initial measurements show that running ambitious computing project. The size of this additional BOINC clients on a computing cluster project is, however, very realistic. The used for other purposes required only 4.2% of performance of the infrastructure was set at additional energy. The measurements 600000 CFP2006 benchmarks. performed on laptops show an energy requirement that is more than 50% less than a The computing cluster requires between 8.800 dedicated set-up. and 10.400 computing cores (1.100 and 2.600 computing nodes). The volunteer computing network, to obtain a similar performance, required between 2.200 and 160.000 In case all the PCs in a volunteer computing participants. network are used 24/7 to execute the code of the scientiﬁc project then it is not surprising that Using this simple model we estimate that for the the energy consumption of this grid is roughly cost: equivalent to that of a cluster. However, such a conﬁguration of the volunteer computing a computing cluster would require between network is very unlikely apart from some 1.5 and 4 million Euro per year. enthusiastic participants.

an equivalent volunteer computing network Another result is that when the computation requires between 195.000 and 490.000 Euro takes place as a background task, the ﬁnancial per year. implication for the volunteers remains very moderate: they contribute between 2.6 and 39 For the energy consumption, we obtained the Euro per year (estimated average: 10€/yr) by following estimates: paying extra for their electricity bill.

the computing cluster consumes between These figures depend on the hardware 2.9 and 4.4 GWh (1 GWh = 1 million kWh) per specifications of the machines, their energy year. consumption, their performance, the BOINC client configuration and the set-up of the the background computing Desktop Grid volunteer computing project. consumes between 0.68 and 1.4 GWh.

the dedicated Desktop Grid consumes between 2.9 and 12.2 GWh per year.

What we see is that a Desktop Grid is always Links cheaper. This is due in big part because the machines in a Desktop Grid are already paid Energy-Star for to perform other activities. The biggest part of the cost of a Desktop Grid consists of the http://www.energystar.gov/index.cfm? salaries to maintain the on-site project servers. fuseaction=find_a_product.showProductGroup&pgw_code =CO For a Desktop Grid this maintenance cost is also lower than for a data centre because the Lesandro Ponciano, Francisco Brasileiro, On the Impact of number of on-site machines is significantly less Energy-saving Strategies in Opportunistic Grids, than for a data centre as most volunteers http://dx.doi.org/10.1109/GRID.2010.5698003 maintain their own machine. Lesandro Ponciano, Francisco Brasileiro, On the Impact of Energy-saving Strategies in Opportunistic Grids, In terms of energy consumption, a background- http://www.ens-lyon.fr/LIP/RESO/e2gc2_2010/slides/ computing Desktop Grid is three to four times e2gc2_lponciano.pdf less energy-hungry than a data centre. This significant reduction of energy is due to the fact Comission, European. GERMANY – Energy Mix Fact Sheet. [Online] 01 01 2007 that the computation takes place using the available CPU cycles while the PCs are running http://ec.europa.eu/energy/energy_policy/doc/factsheets/ mix/mix_de_en.pdf for other activities. In this case the marginal, or additional, energy that is required is much lower Commission, European. FRANCE – Energy Mix Fact Sheet. [Online] 01 01 2007

– 46 – http://ec.europa.eu/energy/energy_policy/doc/factsheets/ mix/mix_fr_en.pdf 6 Glossary Comission, European. DENMARK – Energy Mix Fact Sheet. [Online] AUP Acceptable Use Policy http://ec.europa.eu/energy/energy_policy/doc/factsheets/ CO Confidential mix/mix_dk_en.pdf CONDMAT International Conference Spot, Nord Pool. Nord Pool Spot implements negative price floor in Elspot from October 2009. Press release. Modern Problems of [Online] 04 02 2009. Condensed Matter http://www.nordpoolspot.com/Market_Information/ DEGISCO Desktop Grids for Exchange-information/No162009-Nord-Pool-Spot- International Scientific implements-negative-price-floor-in-Elspot-from- Collaboration October-2009-/ DG Desktop Grid Low Energy BOINC Wiki EADM EDGeS Application http://low-energy-boinc.cslparis.fr/ Development Methodology EC European Commission EDGeS Enabling Desktop Grids for e- Science EGI European Grid Initiative EGITF European Grid Initiative Technical Forum e-IRG e-Infrastructure Reflection Group GNU GNU NGI National Grid Initiative ICPC International Cooperation Partner Countries IDGF International Desktop Grid Federation IPR Intellectual Property Right ISGC International Symposium on Grids and Clouds OGF Open Grid Forum PU Public RSS Remote Site Syndication SG Service Grid SLA Service Level Agreement WP Workpackage

– 47 – A special thanks to all the volunteers around the world that use Desktop Grids to donate idle computing time and thus giving science a helping hand.

– 48 –