Short term roadmap

Enol Fernández

www.egi.eu This work by EGI.eu is licensed under a Creative Commons Attribution 4.0 International License. Short term roadmap

• Discussed during the F2F on 29-30 November • Seeks to improve reliability and usability of the federation • Starting now until Q2 2017 • Complete roadmap available at http://go.egi.eu/fedcloud-roadmap • Mid- and Long-term to be finalized early next year

12/20/16 2 VM Image Management

• CloudKeeper development – If you have any requirements, add them to the github issue tracker at https://github.com/Misenko/cloudkeeper/issues • [Q1 2017] First OpenNebula developments with complete functionality ready (CESNET) • [Q1 2017] Start of OpenStack developments (IPCH). • [Q2 2017] Assess development plan for (GRNET) • [Q2 2017] Roll into production

12/20/16 3 Monitoring

• [Jan 2017] Introduction of swift probes (SRCE) • [Jan 2017] Introduction of improved OCCI probes with image from AppDB (SRCE/CESNET) • [Q1 2017] Introduction of probe for availability of image lists (SRCE/CESNET) • [Q2 2017] Introduction of probe for block storage (SRCE/CESNET) • [Q2 2017] Improvement of accounting probe (SRCE/STFC) • [Q2 2017] Improvement of monitoring to include callback from started VMs to enable checking internal functionality (e.g. block storage is actually accessible from VMs) (SRCE/CESNET) • [Q2 2017] Introduction of monitoring for fedcloud.egi.eu and training.egi.eu VOs. Evaluate implications on the sites, define features to be tested and adjust probe periodicity to avoid overloading sites.(SRCE/EGI.eu)

12/20/16 4 Information Discovery

• [Dec 2016/Jan 2017] Finalise implementation of VO-based information (BIFI) • [Q2 2017] Further extend GlueSchema and renderings to better suit our use cases (or look for viable alternatives). Engage with tool developers to understand what information is missing (BIFI/WP4/EGI.eu) • [Q2 2017] Research on alternative ways of providing information discovery to overcome issues of BDII. Explore solutions based on publication/consumption of information based on messaging (WP4/INDIGO-DataCloud/CESNET) • [Q2 2017] Include networking information: capabilities and policies for each site must be available in the information discovery system so users can select appropriate sites for their use cases. Assess what users need from sites and include the information at the providers (WP4)

12/20/16 5 AAI

• [Jan 2017] Pilot OpenStack EGI CheckIn testbed with GUI and CLI tools (EGI.eu/LIP/GRNET). • [Q2 2017] Drop Keystone-VOMS (v2), move to Keystone v3 Federation with GridSite (WP4) • [Q2 2017] Document and provide clear guidelines for integration in OpenStack. (EGI.eu/LIP/GRNET) • [Q2 2017] Develop any missing functionality in the clients to support OIDC (WP4/INDIGO-Datacloud) • [Q1 2017] Assess the technical developments needed to support OIDC for OpenNebula providers (WP4, CESNET) • [Q1 2017] Assess the technical developments needed to support OIDC for Synnefo providers (WP4, GRNET) • [Q2 2017] Define transition roadmap from certificates to EGI CheckIn.

12/20/16 6 Accounting

• [Dec 2016] Organise meeting to further understand issues: missing information from probes, clearly define semantics of fields and how to account specific situations (suspension, CPU time, resizing, IPs), missing valuable information in the portal (flavors, images) • [Jan 2017] Deploy new accounting schema (WP4/STFC) • [Jan 2017] Assessment of the long running VMs accounting. Definition of new roadmap for this feature (STFC) • [Q1 2017] Improvement of portal views for • [Q1 2017] Packages for Ubuntu/CentOS 7 in CMD (STFC) • [Q2 2017] GPGPU accounting (STFC/IISAS)

12/20/16 7 Providers Operations

• [Dec 2016/Jan 2017] Clarify in documentation implications for site operations for joining the federation. Clearly document workflows and roles of the providers and EGI. (EGI.eu) • [Q2 2017] OLAs for fedcloud.egi.eu and training.egi.eu defined and agreed. (EGI.eu) • [Q2 2017] Provide templates and clear documentation on how to support new VOs. Reduce the steps as much as possible for this. (EGI.eu) • [Q2 2017] Make SLA overview publicly available to cloud providers so they can perform adequate capacity planning and join existing negotiations even if not planned initially. (EGI.eu)

12/20/16 8 User facing services

• [Jan 2017] Identify issues with current OCCI implementations, evaluate how to overcome them (WP4) • [Jan 2017] Survey user communities on satisfaction and used APIs/features and new requirements (check existing surveys also) so technical implementation plan can follow. (EGI.eu) • [Q1 2017] Enable training VO on AppDB VMOps, enable for pilot users (IASA) • [Q2 2017] Finalise AppDB VMOps Dashboard (IASA) • [Q2 2017] Integrate AppDB VMOps Dashboard with LToS (IASA/EGI.eu) • [Q1 2017] Finalize OCCI 1.2 implementations in all CMF, including snapshotting and resizing of images (WP4) • [Q2 2017] Provide an orchestration service for the EGI Federated Cloud that allows deployment of applications as collection of resources on the hybrid- cloud using the most suitable technology. • [Q2 2017] Provide orchestration templates for typical cases so users can easily deploy their applications on top (e.g. Hadoop, Kubernetes, etc.)

12/20/16 9 Hybrid Cloud

• [Q1 2017] Assess the available technology for IaaS provisioning (in a multi-cloud environment. Identify user requirements and possible solutions for the federations. Clearly Classify tools by capabilities. Already identified products: IM, SlipStream, Terraform, Scalr, OCCOPUS, INDIGO-DataCloud. Have 3-4 providers to compare features (SZTAKI and SURFSara interested). • [Q1 2017] Define the scenarios for usage of commercial providers (e.g. scaling up computation, procurement of resources, hybrid data platform) in the federation. Use cases: LHC, EO Data, DGrow. • [Q2 2017] Define a technical blueprint for enabling hybrid- cloud access and review the federation model to fit the scenarios of collaboration with commercial providers, focus on federated AAI, data portability & computing portability.

12/20/16 10 Data Management

• [Q2 2017] Define exploitation plan for the EGI DataHub: clarify EGI DataHub components required at the sites to provide the service, document how EGI DataHub data is accessed from the VMs (or containers), specially to provide a posix access to the shared data). Understand scalability and potential performance issues of OneData. Communities: LifeWatch, astronomy involving CANFAR.

12/20/16 11 Security

• [Q1 2017] Identify relevant orchestration and similar services in the infrastructure and register them in GOCDB with clear contact points. • [Jan 2017] Agree on a clear position on the VM operator role idea (FedCloud) • [Q2 2017] Define default policies regarding inbound/outbound connectivity and public IPs for sites • [Q2 2017] Review if existing policies allow snapshots (after implementation) • [Q2 2017] Perform security vulnerability assessment or endorsement of orchestrators (at least of the more commonly used in the infrastructure) and the deployed configuration templates.

12/20/16 12 Relations with other cloud initiatives

• [2017] Increase participation in relevant OpenStack forums, e.g. scientific working group, Boston Cloud declaration workshop, federation-related activities, etc. • [Q1 2017] Survey existing public founded and research clouds at European level and worldwide. • [Q2 2017] Survey services offered by the NGIs that can have interest from the research community and can be offered at the federation level.

12/20/16 13 Thank you for your attention.

Questions?

www.egi.eu This work by EGI.eu is licensed under a Creative Commons Attribution 4.0 International License.