Building a Private Cloud with Open Nebula

Total Page：16

File Type：pdf, Size：1020Kb

Lucia Short Mentors: Ryan Glenn Andree Jacobson ISTI-OFF Ross Nordeen David Kennel DCS-1 Building a private cloud with OpenNebula LA-UR 10-05197 Why use Virtualized Cloud Computing for HPC? • Support Legacy Software Stacks • Flexible Load Balancing and Energy Efficiency • On-Demand and Dynamic Provisioning of Clusters • Enhance Failover and Redundancy Solutions Challenges • CPU and Memory Overhead • Network Overhead – limitations of TCP/IP overhead – Support for Infiniband/Quadrics/Myrinet is missing • Scalability Implementing OpenNebula on our cluster Implementation on our cluster Open Source Technologies Used KVM and Libvirt (Virtualization Layer) • KVM is Full Virtualization Solution for Linux – Uses VT extensions – Loadable kernel module • Libvirt Provides Libraries to Manage Multiple Hypervisors – Provides a GUI and CLI interface Open Nebula (Management Layer) • Virtual Infrastructure manager • Flexible tool to build any cloud environment • Haizea was used inplace of OpenNebula’s scheduler DRBD (Storage Layer) • Distributed storage system for Linux • Similar to RAID 1, except over the network Results Management Interface Reliability and Availability • Live migration in Open Nebula • Redundant image repository Customer Experience Future with Open Nebula • Explore a hybrid cloud with vCloud, EC2, OCCI • A management interface for HPC clusters will be ready in Sept • OFED support to come soon • Open Nebula 2.0 released a week ago – Cluster support, load aware scheduling Conclusion • Virtualization will not replace conventional clusters – will always be overhead • OpenNebula is a feasible option – For capacity and legacy systems • Great potential in the field – However not capable of solving all of HPC’s needs Questions?.

Copy Link

Recommended publications

Google Cloud Issue Summary Multiple Products - 2020-08-19 All Dates/Times Relative to US/Pacific
Google Cloud Issue Summary Multiple Products - 2020-08-19 All dates/times relative to US/Pacific Starting on August 19, 2020, from 20:55 to 03:30, multiple G Suite and Google Cloud Platform products experienced errors, unavailability, and delivery delays. Most of these issues involved creating, uploading, copying, or delivering content. The total incident duration was 6 hours and 35 minutes, though the impact period differed between products, and impact was mitigated earlier for most users and services. We understand that this issue has impacted our valued customers and users, and we apologize to those who were affected. DETAILED DESCRIPTION OF IMPACT Starting on August 19, 2020, from 20:55 to 03:30, Google Cloud services exhibited the following issues: ● Gmail: The Gmail service was unavailable for some users, and email delivery was delayed. About 0.73% of Gmail users (both consumer and G Suite) active within the preceding seven days experienced 3 or more availability errors during the outage period. G Suite customers accounted for 27% of affected Gmail users. Additionally, some users experienced errors when adding attachments to messages. Impact on Gmail was mitigated by 03:30, and all messages delayed by this incident have been delivered. ● Drive: Some Google Drive users experienced errors and elevated latency. Approximately 1.5% of Drive users (both consumer and G Suite) active within the preceding 24 hours experienced 3 or more errors during the outage period. ● Docs and Editors: Some Google Docs users experienced issues with image creation actions (for example, uploading an image, copying a document with an image, or using a template with images).
[Show full text]
Hybrid Cloud Foundation
Hybrid Cloud Foundation Explore, Plan, Implement Hybrid Cloud Foundation is your on-ramp to Azure. This This offering includes modules focusing on Core Azure, Microsoft Services Solution Offering will help you learn Azure IaaS, Azure Operations, and Azure Workloads. The about Azure so that you can make informed design Operations module is a discovery and planning workshop. decisions leading to an Enterprise-grade implementation The Core, IaaS, and Workload modules include a technical to meet your business needs and strategies. design workstream, and a hands-on exploration and implementation workstream. Outcomes Reduce cloud Prepare for new Accelerate cloud adoption risk operating models adoption Benefit from cloud adoption Cloud completely changes how IT is Take your hybrid solution to market expertise shaped over thousands delivered and managed. Gain quickly with a structured approach of engagements with Fortune 500 insights and recommendations for and detailed guidance based on and other Enterprise organizations running and operating in the cloud. our past experience. around the world. Capabilities Hybrid Cloud Foundation Modules Related offerings Core Azure Design (3 weeks) Azure Workloads for SAP (6+ weeks): Focus on design decisions core to Azure: subscription Azure Workloads for SQL (6+ weeks): model, naming conventions, identity, security, and external Design and implement a specific VM-based workload. network connectivity. Azure Migration Pilot (3 weeks): Migrate your first VM- based workloads to Azure. Azure IaaS Design (6 weeks) Extend your core Azure design. Focus on the design Azure Operations Jumpstart (5 weeks): Deep planning to decisions to enable Azure IaaS capabilities: compute, help your people, process, technology shift for operating storage, networking, and extending on-premises in the cloud.
[Show full text]
Opennebula 5.4 Deployment Guide Release 5.4.15
OpenNebula 5.4 Deployment guide Release 5.4.15 OpenNebula Systems Jun 19, 2018 This document is being provided by OpenNebula Systems under the Creative Commons Attribution-NonCommercial- Share Alike License. THE DOCUMENT IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IM- PLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE DOCUMENT. i CONTENTS 1 Cloud Design 1 1.1 Overview.................................................1 1.2 Open Cloud Architecture.........................................2 1.3 VMware Cloud Architecture.......................................7 1.4 OpenNebula Provisioning Model.................................... 13 2 OpenNebula Installation 19 2.1 Overview................................................. 19 2.2 Front-end Installation.......................................... 19 2.3 MySQL Setup.............................................. 25 3 Node Installation 27 3.1 Overview................................................. 27 3.2 KVM Node Installation......................................... 28 3.3 vCenter Node Installation........................................ 34 3.4 Verify your Installation.......................................... 42 4 Authentication Setup 50 4.1 Overview................................................. 50 4.2
[Show full text]
Guest OS Compatibility Guide
Guest OS Compatibility Guide Guest OS Compatibility Guide Last Updated: September 29, 2021 For more information go to vmware.com. Introduction VMware provides the widest virtualization support for guest operating systems in the industry to enable your environments and maximize your investments. The VMware Compatibility Guide shows the certification status of operating system releases for use as a Guest OS by the following VMware products: • VMware ESXi/ESX Server 3.0 and later • VMware Workstation 6.0 and later • VMware Fusion 2.0 and later • VMware ACE 2.0 and later • VMware Server 2.0 and later VMware Certification and Support Levels VMware product support for operating system releases can vary depending upon the specific VMware product release or update and can also be subject to: • Installation of specific patches to VMware products • Installation of specific operating system patches • Adherence to guidance and recommendations that are documented in knowledge base articles VMware attempts to provide timely support for new operating system update releases and where possible, certification of new update releases will be added to existing VMware product releases in the VMware Compatibility Guide based upon the results of compatibility testing. Tech Preview Operating system releases that are shown with the Tech Preview level of support are planned for future support by the VMware product but are not certified for use as a Guest OS for one or more of the of the following reasons: • The operating system vendor has not announced the general availability of the OS release. • Not all blocking issues have been resolved by the operating system vendor.
[Show full text]
Main Application Version 7.18 This Documentation Is Provided Under Restrictions on Use and Are Protected by Intellectual Property Laws
Architecture Main application Version 7.18 This documentation is provided under restrictions on use and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this documentation, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. © 2021 Creatio. All rights reserved. Table of Contents | 3 Table of Contents Main application 4 Creatio layers 4 Creatio infrastructure 5 Horizontal scaling 9 Deployment options 11 © 2021 Creatio. All rights reserved. Main application | 4 Main application Beginner The classic three-tier Creatio architecture is cross-platform, flexibile, and scalable. The presentation layer lets users access Creatio UI via a web browser or a mobile app for Android or iOS. Creatio application layer is available for the .NET Framework and .NET Core platforms, and can be deployed on Windows, Linux, and macOS web servers. Creatio supports Microsoft SQL Server, Oracle, and PostgreSQL for persistent storage on the data layer. Creatio supports version control system server integration, which allows for multi-user development. The Creatio architecture supports horizontal scaling and different deployment methods: cloud and on-site (with or without fault tolerance). Creatio layers Creatio has a classic three-tier architecture with the following layers: data, application, presentation.
[Show full text]
Rackspace Service Blocks: Complex Cloud Operations Gain Advanced Operational Guidance and Support from Certified Cloud Specialists
Rackspace Service Blocks: Complex Cloud Operations Gain advanced operational guidance and support from certified cloud specialists. Large organizations and businesses running highly complex cloud About Rackspace environments often require customized cloud architecture and access Rackspace is your trusted partner across cloud, to highly skilled cloud resources. Finding the right level of technical applications, security, data and infrastructure. expertise can be challenging, because talent is scarce, expensive and hard to retain. Rackspace® Complex Cloud Operations help you manage • A leader in the 2018 Gartner Magic Quadrant ongoing operational challenges and keep you informed on ever-evolving for Public Cloud Infrastructure Managed cloud features. Service Providers, Worldwide • 3,000+ cloud experts Experts Dedicated to Your Success • Hosting provider for more than half of the Rackspace is a next-generation cloud services provider that manages the Fortune 100 full cloud lifecycle. Complex Cloud Operations offers access to advanced technical resources to better manage public cloud operations and • 20+ years of hosting experience infrastructure architecture. • Customers in 150+ countries • 3,000+ AWS accreditations Key Features • 1,000+ AWS certifications Rackspace offers a range of robust features to support complex cloud operations including: • 1,300+ Microsoft certifications worldwide • 200+ Microsoft-certified professionals Expertise: Expert technical resources, including solution architects, lead engineers, container and Kubernetes experts, and DevOps professionals • 150+ Microsoft Azure certifications worldwide help optimize public cloud operations and architectural infrastructure. • 100+ Microsoft Azure-certified professionals Optimized Custom Configurations: Custom configurations include • 13 Microsoft partner certifications monitoring services, OS optimization, patch management, alert response • Five-time Microsoft Hosting Partner of the Year runbooks and deployment management.
[Show full text]
An Experimental Study of Load Balancing of Opennebula Open-Source Cloud Computing Platform
An Experimental Study of Load Balancing of OpenNebula Open-Source Cloud Computing Platform A B M Moniruzzaman, StudentMember, IEEE Kawser Wazed Nafi Syed Akther Hossain, Member, IEEE & ACM Abstract— Cloud Computing is becoming a viable computing organization existing computing infrastructure into a private or solution for services oriented computing. Several open-source hybrid cloud [3]. Eucalyptus, CloudStack, OpenStack and cloud solutions are available to these supports. Open-source OpenNebula are the most popular open-source cloud solutions software stacks offer a huge amount of customizability without used as to build such community cloud platforms. Eucalyptus huge licensing fees. As a result, open source software are widely is open source cloud software for building AWS-compatible used for designing cloud, and private clouds are being built private and hybrid clouds [4]. Apache CloudStack is open increasingly in the open source way. Numerous contributions source software designed to deploy and manage large networks have been made by the open-source community related to of virtual machines Infrastructure as a Service (IaaS) cloud private-IaaS-cloud. OpenNebula - a cloud platform is one of the computing platform [5]. OpenStack is a cloud operating system popular private cloud management software. However, little has that controls large pools of compute, storage, and networking been done to systematically investigate the performance evaluation of this open-source cloud solution in the existing resources throughout a datacenter, all managed through a literature. The performance evaluation aids new and existing dashboard that gives administrators control while empowering research, industry and international projects when selecting their users to provision resources through a web interface [6].
[Show full text]
Google App Engine Overview
Google App Engine Overview Chris Schalk Developer Advocate http://twitter.com/cschalk What is cloud computing? 2 Cloud Computing Defined SaaS APaaS IaaS Source: Gartner AADI Summit Dec 2009 3 IaaS value proposition… 4 APaaS value proposition… 5 Google App Engine •Easy to build •Easy to maintain •Easy to scale 6 By the numbers 250M+ 250,000+100,000+ daily DevelopersApps Pageviews 7 7 In a word… via Wordle Word source: http://en.wikipedia.org/wiki/App_Engine 8 Some App Engine Partners 9 Socialwok 10 App Engine @ 11 Social networking at scale >62M Users 12 gigy Socialize 13 gigy Socialize Flexible Scalability "Although we typically host all our services in-house, on our own infrastructure, we felt that GAE would be a better fit for the live chat feature because of its unique traffic pattern, which is characterized by very low traffic most of the time with very high bursts during high profile events." Raviv Pavel, Gigya VP of Research and Development http://googleappengine.blogspot.com/2010/02/scalability-means-flexibility.html#links 14 Why App Engine? 15 Managing Everything is Hard 16 DIY Hosng means hidden costs •Idle capacity •Soware patches & upgrades •License fees •Lots of maintenance •Traﬃc & ulizaon forecasng •Upgrades 18 Cloud development in a box SDK & “The Cloud” Hardware Networking Operating system Application runtime Java, Python Static file serving Services Fault tolerance Load balancing 19 App Engine Details 20 Specialized Services Memcache Datastore URL Fetch Mail XMPP Task Queue Images Blobstore User Service 21 Language runtimes Duke, the Java mascot Copyright © Sun Microsystems Inc., all rights reserved.
[Show full text]
EXPRESSCLUSTER X 4.3 for Linux, Getting Started Guide
EXPRESSCLUSTER X 4.3 for Linux Getting Started Guide Release 1 NEC Corporation Apr 09, 2021 TABLE OF CONTENTS: 1 Preface 1 1.1 Who Should Use This Guide.......................................1 1.2 How This Guide is Organized......................................2 1.3 EXPRESSCLUSTER X Documentation Set...............................3 1.4 Conventions...............................................4 1.5 Contacting NEC.............................................5 2 What is a cluster system? 7 2.1 Overview of the cluster system.....................................8 2.2 High Availability (HA) cluster......................................9 2.3 Error detection mechanism........................................ 14 2.4 Taking over cluster resources...................................... 16 2.5 Eliminating single point of failure.................................... 18 2.6 Operation for availability......................................... 23 3 Using EXPRESSCLUSTER 25 3.1 What is EXPRESSCLUSTER?..................................... 26 3.2 EXPRESSCLUSTER modules...................................... 27 3.3 Software conﬁguration of EXPRESSCLUSTER............................ 28 3.4 Network partition resolution....................................... 34 3.5 Failover mechanism........................................... 35 3.6 What is a resource?............................................ 47 3.7 Getting started with EXPRESSCLUSTER............................... 52 4 Installation requirements for EXPRESSCLUSTER 53 4.1 Hardware................................................
[Show full text]
Cloud Computing with Nimbus February 2009
Cloud Computing with Nimbus February 2009 Kate Keahey ([email protected]) University of Chicago Argonne National Laboratory Cloud Computing SaaS Software-as-a-Service elasticity computing on demand PaaS Platform-as-a-Service capital expense operational expense IaaS Infrastructure-as-a-Service 8/25/2009 The Nimbus Toolkit: http//workspace.globus.org 2 The Quest Begins Code complexity Resource control 8/25/2009 The Nimbus Toolkit: http//workspace.globus.org 3 “Workspaces” Dynamically provisioned environments Environment control Resource control Implementations Via leasing hardware platforms: reimaging, configuration management, dynamic accounts… Via virtualization: VM deployment 8/25/2009 The Nimbus Toolkit: http//workspace.globus.org 4 A Brief History of Nimbus First STAR production run on EC2 Xen released EC2 goes online Nimbus Cloud comes online 2003 2006 2009 Research on First WSRF EC2 gateway Support for agreement-based Workspace Service available EC2 interfaces services release Context Broker release 8/25/2009 The Nimbus Toolkit: http//workspace.globus.org 5 Nimbus Overview Goal: cloud computing for science Open source, extensible IaaS implementation A platform for experimentation with features for scientific needs and interoperability Set up private clouds (privacy, expense considerations) Workspace Service Orchestration tools Focus on end-to-end picture Context Broker, gateway http://workspace.globus.org/ 8/25/2009 The Nimbus Toolkit: http//workspace.globus.org 6 The Workspace Service Pool Pool Pool node node node VWS Service Pool Pool Pool node node node Pool Pool Pool node node node Pool Pool Pool node node node 8/25/2009 The Nimbus Toolkit: http//workspace.globus.org 7 The Workspace Service The workspace service publishes information on each workspace Pool Pool Pool node node node VWS Service Pool Pool Pool node node node Users can query this information to find out things about their Pool Pool Pool workspace (e.g.
[Show full text]
Australian IT Channel Reveals Cloud Pain Points
Sponsored by OrionVM Sponsored by OrionVM RESULTS OF OUR SURVEY ABOUT CLOUD Australian IT channel 71.2% 56.3% 55.2% 51.7% see market demand for offer Software-as-a-Service offer private or hybrid of channel companies offer reveals cloud pain points cloud storage or Platform-as-a-Service cloud infrastructure public cloud infrastructure Australian IT channel companies are The channel also sees plenty of Security and compliance topped clamouring to sell cloud, but worry demand for cloud services. We asked that wishlist, followed by cloud vendors that incumbent cloud providers aren’t readers to identify the services or that don’t compete with resellers for WHICH CLOUD INFRASTRUCTURE PROVIDERS DO WHAT PROBLEMS DO YOU HAVE WITH YOUR CURRENT CLOUD INFRASTRUCTURE PROVIDER?* offering the structures needed to make applications experiencing greatest customers. YOU CURRENTLY USE OR RESELL?* cloud a viable proposition. end-customer demand, and more than Survey respondents also wanted I do not currently use any cloud 14% That’s one of the findings from a new two-thirds pointed to cloud storage. cloud infrastructure that complements infrastructure providers I am completely satisfied with my current survey of CRN readers, sponsored by Intriguingly, storage beat out tier one their businesses, full control over cloud cloud infrastructure provider 24% Other (please specify) 11% OrionVM and conducted in November and enterprise applications, databases and branding, enterprise-level performance Losing clients that go directly to my cloud provider and/or competitors 25% and resilient infrastructure – further proof December 2018. unified communications. Not enough control over branding and doesn’t allow The survey found that over 44 percent that low prices won’t solve the channel’s me to represent services and solutions as my own 21% None 12% of channel partners and distributors INCUMBENT PLATFORMS cloud needs.
[Show full text]
502 How the “Netbook” Is Changing Learning
Hottest e-Learning Trends and Research December 9 & 10, 2010 502 How the “Netbook” is Changing Learning Reuben Tozman, edCetra Training Hottest e-Learning Trends and Research Dec 9 & 10, 2010 How The Netbook Is Changing Learning Reuben Tozman, Chief Learning Officer December 10, 2010 What is cloud computing? What does cloud computing mean to you? How is it different to ASP, or is it? Type your answers in the chat pod. Session 502 – How the “Netbook” is Changing Learning – Reuben Tozman, Page 1 edCetra Training Hottest e-Learning Trends and Research Dec 9 & 10, 2010 What is cloud computing? Cloud computing describes a new supplement, consumption, and delivery model for IT services based on the Internet, and it typically involves over-the-Internet provision of dynamically scalable and often virtualized resources. Wikipedia Examples of Cloud Computing Applications Gliffy Google Docs Salesforce.com Are there any other examples you can think of? Session 502 – How the “Netbook” is Changing Learning – Reuben Tozman, Page 2 edCetra Training Hottest e-Learning Trends and Research Dec 9 & 10, 2010 Cloud Architecture Cloud Architecture – What’s the Key? Think about learning systems for a moment…. Think past the LMS…..whats the key? Scalability via dynamic ("on-demand") provisioning of resources on a fine-grained, self-service basis near real-time, without users having to engineer for peak loads. Performance is monitored, and consistent and loosely coupled architectures are constructed using web services as the system interface. Session 502 – How
[Show full text]