DOCSLIB.ORG

Big Data & Cloud

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Big Data & Cloud Big Data & Cloud 4th European Summit on the Future Internet António Miguel Ferreira, CEO, Lunacloud Aveiro, 13 to 14th June 2013 ? About About Cloud use case 1 Cloud use case 2 Big Data Lunacloud is a cloud infrastructure and platform services provider (IaaS + PaaS), with datacenters in the UK, Portugal, France and Russia (July 2013). About Cloud use case 1 Cloud use case 2 Big Data 1. The Cloud is a more efficient way of using IT resources. 2. High-end compute & storage resources are now widely available through the global Internet. 3. A whole new set of challenges may be addressed quickly and with lower costs. About Cloud use case 1 Cloud use case 2 Big Data Cloud use case 1 IT research About Cloud use case 1 Cloud use case 2 Big Data Altoros Product engineering in areas such as implementation of NoSQL and NewSQL storage systems, Hadoop distributed computing, etc. Offices in Silicon Valley (Sunnyvale, California), Norway, Denmark, Switzerland, UK, Eastern Europe (Minsk, Belarus) and South America (Buenos Aires, Santa Fe, Argentina). Challenge #1: NoSQL benchmarking Testing of different NoSQL databases against various types of workloads: Cassandra, MongoDB, Riak, Couchbase, MySQL Cluster, and Hbase. Yahoo! Cloud Serving Benchmark (YCSB) used to evaluate performance. Infrastructure needs 120 virtual machines with a total of 960 GB RAM + 920 CPU Cores + 12 TB local storage ... For only 15 days! Traditional IT capex = 100.000 € Cloud IaaS cost = 8.000 € About Cloud use case 1 Cloud use case 2 Big Data Altoros Product engineering in areas such as implementation of NoSQL and NewSQL storage systems, Hadoop distributed computing, etc. Offices in Silicon Valley (Sunnyvale, California), Norway, Denmark, Switzerland, UK, Eastern Europe (Minsk, Belarus) and South America (Buenos Aires, Santa Fe, Argentina). Challenge #2: Hadoop benchmarking Testing different Hadoop distribution packages to assess their performance, ease of use, functionality, etc: Apache Hadoop; CDH, Cloudera's Distribution, including Apache Hadoop; HDP, Hortonworks Data Platform; MapR M3 Edition; Intel Hadoop; Pivotal HD. TeraSort benchmark to measure performance. Infrastructure needs 250 virtual machines with a total of 2000 GB RAM + 2000 CPU Cores + 25TB local storage ... For only 25 days! Traditional IT capex = 200.000 € Cloud IaaS cost = 28.000 € About Cloud use case 1 Cloud use case 2 Big Data Concepts – Use case 1: 1.Immediate availability 2.Elasticity 3.Pay per use About Cloud use case 1 Cloud use case 2 Big Data Cloud use case 2 Entertainment About Cloud use case 1 Cloud use case 2 Big Data Music Stage Social network for independent bands and musicians. Challenge: Multimedia storage 10’s or 100’s of thousands of bands and artists want to upload 10’s or 100’s of musics and videos, each with 4 to 30 MB in size. Need for unlimited storage space with no upfront investment. Infrastructure needs Up to 100.000 bands x (100 musics + 10 videos) = 60TB, spread across the world Growing from 1 band to 100.000 bands, without worrying about storage, resilience and geographical coverage. About Cloud use case 1 Cloud use case 2 Big Data Concepts – Use case 2 1.Elasticity 2.On-demand self-service (API) 3.Broad network access About Cloud use case 1 Cloud use case 2 Big Data Big Data NoSQL – Our case About Cloud use case 1 Cloud use case 2 Big Data Cloud Storage Unlimited virtual disk accessible through a web interface or an API. Applications can use cloud storage to place any objects. Data is replicated in at least 3 different physical stores. Built over a NoSQL Cassandra database. Cloud Mongo MongoDB as a Service, with single or replicated instances, provisioned through a point-and-click web interface. It allows developers of applications to focus on code, not database management. About Cloud use case 1 Cloud use case 2 Big Data The future of the Cloud In our view IaaS is a commodity, a utility service, that is best when infrastructure is closer to customers, in nearby datacenters. PaaS is a differentiator and will accelerate innovation. SaaS is where most of the activity and innovation takes place. Thank you Email [email protected] Web www.lunacloud.com Twitter @lunacloud.
Load more

Recommended publications
  • Privacy Policy for the Websites of Etherisc Gmbh
    Privacy Policy for the websites of Etherisc GmbH Data protection With this privacy policy we would like to inform you about how we process personal data. We are aware of the importance of the processing of personal data for the user and, accordingly, comply with all relevant legal requirements. The protection of your privacy is of utmost importance to us. That is why it is a matter of course for us to comply with the legal provisions on data protection. 1. Collection, processing and use of personal information 1.1 Personal data Personal data is all information about personal and factual circumstances of a particular or identifiable person. For web page visitors it includes web analytics data, which is collected by the use of Cookies: - Cookie IDs - IP address - Country - operating system - when and which pages were opened - how long a user stayed on pages - browser version - browser language For newsletter subscription it includes: - name and - email address When we registered users for the Token Generating event, it includes: - first name - last name - Email - contact information - identity documents information which is used for Know your Customer procedure (KYC) - occupation, industry, annual income, origin of money, trading volume are used for Know your Customer procedure for certified investors. For request or proposal of the product idea: - Email 1.2 Legal basis The processing of your data is done on the following legal bases: - With regard to data that you specify in forms etc., with your consent, Art. 6 para. 1 lit. a) GDPR - in relation to the services you use, to carry out a contract with you (in particular, Token Generating event), Art.
    [Show full text]
  • Implementacao De Algoritmo De Escalabilidade Para
    Mestrado em Engenharia Informática Estágio Relatório Final 2014/2015 Implementação de algoritmo de escalabilidade para DNS em contexto cloud João Manuel Marques do Santos [email protected] Orientador DEI: Professor Doutor Edmundo Monteiro Orientadores OneSource: Doutor Bruno Sousa Doutor David Palma 2 de setembro de 2015 Implementação de algoritmo de escalabilidade para DNS em contexto cloud Relatório submetido no âmbito do estágio curricular do Mestrado em Engenharia Informática 2014/2015 Autor: João Manuel Marques dos Santos Orientador DEI: Professor Doutor Edmundo Monteio Orientadores OneSource: Doutores Bruno Sousa e David Palma Júri Arguente: Professor Doutor Fernando Boavida Júri Vogal: Professor Doutor Fernando José Barros Agradecimentos O presente relatório representa o culminar de um percurso de estudos em Engenharia Informática, aqui como forma de cumprimento dos requisitos necessários à obtenção do grau de Mestre e para o qual foram essenciais todos os conhecimentos adquiridos ao longo deste ciclo de estudos na Faculdade de Ciências e Tecnologia da Universidade de Coimbra, mas também, desde logo, durante a Licenciatura na Escola Superior de Tecnologia e Gestão de Oliveira do Hospital do Instituto Politécnico de Coimbra. É neste momento que importa também, para além do esforço de enriquecimento pessoal e académico, demonstrar o meu reconhecimento para com todos os que contribuíram para a materialização deste Relatório de Estágio. Expresso o meu agradecimento ao Professor Doutor Edmundo Monteiro e aos Doutores Bruno Sousa e David Palma pela orientação deste trabalho, pela competência, exigência e a compreensão do desafio, mas também pela disponibilidade, observações e sugestões que me incentivaram a trabalhar para alcançar um resultado que é o aproximado às nossas expectativas.
    [Show full text]
  • Guidelines for Cloud Computing Architecture:Development Process
    International Journal of Computer Trends and Technology (IJCTT) – Volume 67 Issue 8 – Aug 2019 Guidelines for Cloud Computing Architecture:Development Process Raja Muhammad Ubaid Ullah#1, Dr.Kevan A. Buckley#2, Dr. Mary Garvey#3, Dr. Jun Li#4. School of Mathematics and Computer Science University of Wolverhampton, Wulfruna Street, Wolverhampton WV1 1LY, UK. Abstract unacceptable” [3]. In CC service models provide Cloud computing (CC) has received significant different types of services to organisations to perform attention from different types of businesses and the business tasks. For instance, Infrastructure as a Industries and emerged as a new utility for businesses Service (IaaS), it offers various type of hardware activities. The philosophy behind CC shows a great resources in the form of service, for instance, Central potential to transform a major part of the IT industry, Processing Unit (CPU), virtual machine (VM), making computing environment and resources even memory, storage and network devices, etc. more attractive as a cost-effective service and changing [2,7]Platform as a Service (PaaS), it works with the the way IT hardware is designed and purchased. combination of both, infrastructure and application by Resulting day by day more small and medium and large CSP, to facilitate the user to develop and deploy their enterprise are adopting different types of CC services. respective applications [2, 8] and finally Software as a However, strong market competitive environment for Service (SaaS), it provides a software delivery model, converting existing IT services to CC environment to run various software applications through the imposed different types of challenges for the CC Internet without installing these in the client site [2, 9].
    [Show full text]
  • Moving to the Cloud? a Buyer’S Guide to Finding the Right Strategic IT Partner
    Moving to the Cloud? A Buyer’s Guide to Finding the Right Strategic IT Partner It is very tempting as a budget-friendly option to simply click hardware costs, optimally manage their complex enterprise a few keystrokes and instantly stand your systems up in a applications, and get better performance – all helping to public cloud-based environment; however, are you potentially drive their business’s bottom line. gambling with your most mission critical systems? ERP Where do you get started? There are many key cloud environments should not only be flexible, scalable, and considerations to take into account when migrating a cloud affordable, but also secure and expertly managed. based solution for your ERP environment. This buyer’s guide As an IT leader, putting your enterprise applications into will illuminate your path to the cloud by answering the most a public cloud model can be a risky proposition: with common buyer questions: today’s cyber security threats, outages potentially costing – Clearly defining a cloud solution companies millions of dollars in revenue, and the complexity of ERP systems that need to be expertly managed. Public, – Questions to Consider when Moving to the Cloud self-provisioning cloud solutions built on white box hardware – Putting the Cloud to Practical Use for Your Business concept should not make your short list. – Outlining vendor options However, the adoption of a cloud hosting model is a – Key Considerations to Choosing a Cloud Provider progressive outsourcing strategy many industry leading organizations are
    [Show full text]
  • Approach for Selecting and Integrating Cloud Services to Construct Hybrid Cloud
    J Grid Computing (2020) 18:441–469 https://doi.org/10.1007/s10723-020-09519-x Approach for Selecting and Integrating Cloud Services to Construct Hybrid Cloud Joonseok Park & Ungsoo Kim & Donggyu Yun & Keunhyuk Yeom Received: 28 September 2018 /Accepted: 26 April 2020 / Published online: 13 May 2020 # The Author(s) 2020 Abstract With the popularization of cloud computing, cloud services through a hybrid cloud service broker various cloud services have emerged, and hybrid clouds (hybrid CSB), which is an automation solution that can take advantage of combining public and private supporting hybrid cloud deployment. Moreover, the pro- clouds are attracting attention. However, because of their posed method is realized using a script including the variety, determining a combination of cloud services application programming interface of each cloud service. suitedtotheuser’s current environment and requirements The proposed C-SIP will be used as a core approach is expensive when deploying a hybrid cloud. Even if the toward the hybrid CSB, which is expected to facilitate required services are available, there is a lack of tools to the introduction of hybrid clouds and the acquisition of connect them, manage them in batches, and utilize the cloud strategies. integrated environment. To solve these problems, this paper proposes a cloud selection and integration process Keywords Cloud service brokerage . Cloud service (C-SIP), which selects and integrates a combination of selection . Cloud service integration . Cloud service evaluation . Hybrid cloud Ungsoo Kim and Donggyu Yun contributed equally to this work. J. Park Research Institute of Intelligent Logistics Big Data, Pusan 1 Introduction National University, Busandaehak-ro 63beon-gil, Geumjeong-gu, Busan 46241, South Korea e-mail: [email protected] Cloud computing is a computing paradigm that allows remote access via Internet without the need to directly U.
    [Show full text]
  • Building Storage-As-A-Service Businesses
    CLOUDIAN, INC. WHITE PAPER : BUILDING STORAGE-AS-A-SERVICE BUSINEssES WHITE PAPER SERVICE PROVIDERS’ GREATEST NEW GROWTH OPPORTUNITY: Building Storage-as-a-Service Businesses Today, with the availability of off-the-shelf cloud storage platforms, service providers of all sizes can now build cloud storage services with very little risk. Storage-as-a-Service (STaaS) can now be easily added to a service provider’s offerings with minimal cost, little deployment time, and significant ROI. As the enterprise market adopts the cloud model and more applications (and thus use cases) are created, STaaS stands poised to be a massive revenue generator. In addition, this same cloud storage system can reduce operating costs for the service provider by utilizing the same infrastructure for secondary storage for cloud compute systems. ACCORDING TO This whitepaper provides an overview of the market size for STaaS as well as describes the 451 RESEARCH, STORAGE AS A SERVICE technologies and ecosystem which make STaaS viable. The paper goes into detail about design REPRESENTS A LARGE best practices, using the Cloudian Community Edition—which allows a 100TB system to be AND RAPIDLY GROWING deployed with no software license fees—as an example of the value and promise of STaaS. MARKET WITH A CAGR OF 47% WITH A TOTAL MARKET OF NEARLY STaaS Market Size $6B IN 2015. According to 451 Research, Storage as a Service represents a large and rapidly growing market with a CAGR of 47% with a total market of nearly $6B in 2015. Of this $6B, online backup and archiving represents 20% (a 50% – 50% split between backup and archiving).
    [Show full text]
  • Cloud-Computing Strategies for Sustainable ICT Utilization: a Decision-Making Framework for Non-Expert Smart Building Managers
    Cloud-Computing Strategies for Sustainable ICT Utilization: A Decision-Making Framework for Non-Expert Smart Building Managers Karim Jamil Mualla (MSc) (BEng) Submitted for the degree of Doctor of Philosophy Heriot-Watt University School of Energy, Geoscience, Infrastructure and Society Edinburgh, United Kingdom March, 2016 The copyright in this thesis is owned by the author. Any quotation from the thesis or use of any of the information contained in it must acknowledge this thesis as the source of the quotation or information. Abstract Virtualization of processing power, storage, and networking applications via cloud- computing allows Smart Buildings to operate heavy demand computing resources off- premises. While this approach reduces in-house costs and energy use, recent case- studies have highlighted complexities in decision-making processes associated with implementing the concept of cloud-computing. This complexity is due to the rapid evolution of these technologies without standardization of approach by those organizations offering cloud-computing provision as a commercial concern. This study defines the term Smart Building as an ICT environment where a degree of system integration is accomplished. Non-expert managers are highlighted as key users of the outcomes from this project given the diverse nature of Smart Buildings’ operational objectives. This research evaluates different ICT management methods to effectively support decisions made by non-expert clients to deploy different models of cloud-computing services in their Smart Buildings ICT environments. The objective of this study is to reduce the need for costly 3rd party ICT consultancy providers, so non-experts can focus more on their Smart Buildings’ core competencies rather than the complex, expensive, and energy consuming processes of ICT management.
    [Show full text]
  • IEEE Paper Template in A4 (V1)
    IJRET: International Journal of Research in Engineering and Technology eISSN: 2319-1163 | pISSN: 2321-7308 A REVIEW TO ASSESS OPPORTUNITIES AND SECURITY RISK CHALLENGES IN CLOUD COMPUTING Shikha Gupta1, Anil K Saini2 1 College of Computing Sciences and Information Technology (CCSIT), TMU, Moradabad, India 2 University School of Management Studies, GGS Indraprasth University, Delhi, India Abstract In today’s competitive world, organizations are under increasing pressure to develop efficiently and convert their IT practices to achieve more with less. Cloud Computing is a flexible technology that can support a wide variety of application. In contrast to grid computing and clustering, cloud computing is service oriented, loosely coupled, TCP/IP based Business Model technology that provides high security and advantage of virtualization and strong fault tolerance. The purpose of the paper is to provide a review of the cloud computing concepts and challenges facing organizations in its adoption Keywords— Cloud Computing, flexible technology, Business Model technology, virtualization --------------------------------------------------------------------***--------------------------------------------------------------------- 1. INTRODUCTION Cloud Computing is an evolution of the widespread acceptance of virtualization, service-oriented architecture, In today’s competitive world, organizations are under automatic, and utility computing[2][3]. The actual term increasing pressure to develop efficiently and convert their “cloud” is borrowed from telecommunications companies, IT practices to achieve more with less. Business needs to who until the 1990s offered primarily dedicated point to have reduced time-to-market, better agility, higher point data circuits, began offering Virtual Private Network availability, and reduced expenditures to meet the changing (VPN) services with similar quality of service but at much business requirements. In this, cloud computing has proved lower cost.
    [Show full text]
  • Sharing Files Using Cloud Storage Services
    Sharing Files Using Cloud Storage Services Tiago Oliveira, Ricardo Mendes, and Alysson Bessani ftoliveira,[email protected], [email protected] Universidade de Lisboa, Faculdade de Ci^encias,LaSIGE { Portugal Abstract. The widespread use of cloud storage in the last few years can be attributed to the existence of appealing applications such as file backup, data archival and file sharing. File sharing in particular, is im- plemented in different ways by distinct cloud storage services. These differences can appear at the offered permission types and in the form they are applied. We present a survey of these differences for several popular cloud storage services. We also show how to realize secure data sharing using these services, allowing the implementation of equivalent data sharing features in different clouds, an important requirement for secure multi-cloud systems. 1 Introduction With more people accessing their files online, an important part of file sharing today is done by taking advantage of cloud storage. This can be done through personal file synchronization services like Dropbox [4], Google Drive [6], Mi- crosoft OneDrive [13], Box [3] or Ubuntu One [18], which store users' data in the cloud. These services have been extremely successful, as attested by the success of DropBox, which has announced last April that it reached 275 million users [5]. These systems perform file sharing through dedicated application servers which are responsible for controlling access to the files as well as user groups management, data deduplication, etc. It means that the security of the file shar- ing requires trusting not only the storage service (for instance, Dropbox is built on top Amazon S3 [4]), but also these application servers.
    [Show full text]
  • Position: Cloud Foundry / Devops Engineer (Seattle, WA) Location: Seattle, WA Even Those Who Don’T Know About Cloud Foundry Have Probably Heard of the Conveyor Belt
    Position: Cloud Foundry / DevOps Engineer (Seattle, WA) Location: Seattle, WA Even those who don’t know about Cloud Foundry have probably heard of the conveyor belt. Car factories began producing 10x more vehicles per month, slashing the cost and time to assemble an automobile. The conveyor belt has changed the world. Cloud Foundry, one of the fastest-growing open-source projects in history (according to the Linux Foundation), is now changing the way companies build their apps. It is a cloud-native platform that abstracts developers from the underlying infrastructure, so that they can focus on business logic and deliver apps in weeks instead of months. Cloud Foundry users include dozens of Fortune 500 and Global 2000 organizations, as well as the governments of the U.S., Australia, South Korea, etc. At Altoros, we bring “software assembly lines” into large organizations, helping them to cut the cost and time to produce software. A member of the Cloud Foundry Foundation, Altoros integrates solutions offered by the Cloud Foundry ecosystem. With 250+ employees across 8 countries in America and Europe, Altoros is behind some of the world’s largest Cloud Foundry deployments. Currently, we are expanding one of our project teams—located in Seattle, WA. Job description: We are looking for a DevOps Engineer who will become part of the global team building cloud-native systems and applying digital transformation to Fortune 500 / Global 2000 customers. You will be helping enterprises to make a revolutionary shift in the way they develop and deliver software, working side-by-side with some of the most experienced engineers in the Cloud Foundry community.
    [Show full text]
  • 2 What Kinds of Hosting Does the Market Offer?
    2 What kinds of hosting does the market offer? If you’ve spent at least 30 minutes searching for hosting solutions, you’ll have noticed that there are different types of hosting: PaaS, shared, VPS, dedicated, colocation, cloud. The hosting market tends to segment itself in those 5-6 categories: some of them have a clear definition, while others (like cloud) have so many different meanings that I’ve had to come up with my own definition for the book. 2.1 Shared hosting On a shared hosting account you get a directory where you can put your static files (HTML, images) and an interpreter for your dynamic pages (in 98% of the cases PHP): the server where your files live is also home to hundreds, if not thousands, of similar websites. You share with the other websites the Internet 7 bandwidth to and from the datacenter, the CPU, the memory and the disk I/O of the server. This is the most limited and cheapest form of hosting available on the market. The main advantage is that you don’t have to perform any system administra- tion tasks such as configuring the OS, the network, installing applications, and so on. Few providers of shared hosting give you the ability of logging in to the actual server with SSH: you get instead a web interface called a control panel which translates clicks to actions on the system. Installing applications is often a one-click process, but you can choose only from a fixed list of the applications made available by the hosting company.
    [Show full text]
  • Cloud Brokering : New Value-Added Services and Pricing Models Angel Felipe Diaz Sanchez
    Cloud brokering : new value-added services and pricing models Angel Felipe Diaz Sanchez To cite this version: Angel Felipe Diaz Sanchez. Cloud brokering : new value-added services and pricing models. Other [cs.OH]. Télécom ParisTech, 2014. English. NNT : 2014ENST0028. tel-01276552 HAL Id: tel-01276552 https://pastel.archives-ouvertes.fr/tel-01276552 Submitted on 19 Feb 2016 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. 2014-ENST-0028 EDITE - ED 130 Doctorat ParisTech T H È S E pour obtenir le grade de docteur délivré par TELECOM ParisTech Spécialité “Informatique et Réseaux” présentée et soutenue publiquement par Felipe DIAZ-SANCHEZ le 10 juin 2014 Cloud brokering : nouveaux services de valeur ajoutée et politique de prix Jury M. Christophe CERIN , Professor, Université Paris 13 Rapporteur M. Laurent LEFEVRE , Professor, ENS-Lyon, INRIA Rapporteur M. Jean-Philippe VASSEUR , Dr. Professeur Associé à Telecom, CISCO-USA Examinateur ! M. Charles LOOMIS , Dr., Laboratoire de l’accélérateur linéaire d’Orsay, CNRS Examinateur M. Jean-Pierre LAISNE , Président, société CompatibleOne Examinateur M. Maurice GAGNAIRE , Professor, Télécom ParisTech Directeur de thèse " TELECOM ParisTech école de l’Institut Mines-Télécom - membre de ParisTech 46 rue Barrault 75013 Paris - (+33) 1 45 81 77 77 - www.telecom-paristech.fr Cloud brokering: new value-added services and pricing models Abstract: Cloud brokering is a service paradigm that provides interoperability and portability of applications across multiple Cloud providers.
    [Show full text]