1
2
3
5
6
7 8 Welcome to Cloud OnBoard 9
10
11
12
13
14
15
16
1 Cloud OnBoard
2 3 Cloud OnBoard
10 ( ) Lunch 12:00 PM 11 (‘Module 3’) Virtual Machines in the Cloud 01:00 PM
12 (‘Module 4’) Storage in the Cloud 01:40 PM
13 (‘Module 5’) Containers in the Cloud 02:30 PM
14 ( ) Break 03:00 PM
15 (‘Module 6’) Applications in the Cloud 03:30 PM
16 (‘Module 7’) Developing, Deploying, and Monitoring in the Cloud 04:00 PM
17 (‘Module 8’) Big Data and Machine Learning in the Cloud 04:30 PM Closing Session | Continue learning with Google Cloud 05:00 PM 18 } 1 Cloud OnBoard
2 3 Certificate of Attendance 5
6
7
8
9
10
11
12
13
14
15
16
17
18 1
2
3
5
6
7 8 #GoogleCloudOnBoard 9
10
11
12
13
14
15
16 1
2
3
5
6 7 Module #1: Introducing 8 Google Cloud Platform 9
10
11
12
13
14
15
16 1 Cloud OnBoard
2 3 Agenda 5 6 Introduction to Google Cloud Platform 7
8 Quiz
9
10
11
12
13
14
15
16
17
18 Cloud OnBoard
Computing trends toward pay-as-you-go, fully automated services
Now Next
Storage Processing Memory Network Storage Processing Memory Network
Physical/Colo Virtualized Serverless
User-configured, managed, and maintained Fully automated Every company is a data company Cloud OnBoard
GCP offers a range of computing architectures
Kubernetes Cloud Managed Compute Engine App Engine Engine Functions services
IaaS Hybrid PaaS Serverless Automated elastic logic resources
Toward managed infrastructure Toward dynamic infrastructure Cloud OnBoard
Google network: 100,000s of km of fiber cable, 8 subsea cables
FASTER (US, JP, TW) 2016
Unity (US, JP) 2010
PLCN (HK, LA) 2019 SJC (JP, HK, SG) 2013
Monet (US, BR) 2017 Network
Network sea cable investments Junior (Rio, Santos) 2017 Edge points of presence >100 Tannat (BR, UY, AR) 2017 Edge node locations >1000
Indigo (SG, ID, AU) 2019 Cloud OnBoard
Google Cloud Platform is organized into regions and zones
Future region and number of zones
Current region and number of zones Cloud OnBoard
Google offers customer-friendly pricing innovations
Billing in sub-hour Discounts for Custom VM increments sustained use instance types
For virtual machines and Automatically applied to Pay only for the resources containers in the cloud; virtual machine use over you need for your application data processing and other 25% of a month services too Cloud OnBoard
Open APIs and open source mean flexibility
Open APIs; compatibility Open source for a rich Multi-vendor-friendly with open-source services ecosystem technologies
Cloud Bigtable Kubernetes Google Stackdriver
Forseti Security Cloud Dataproc Kubernetes Engine Cloud OnBoard
Security is designed into Google’s technical infrastructure
Layer Notable security measures (among others)
Intrusion detection systems; techniques to reduce insider risk; employee U2F use; software Operational security development practices
Internet communication Google Front End; designed-in Denial of Service protection
Storage services Encryption at rest
User identity Central identity service with support for U2F
Service deployment Encryption of inter-service communication
Hardware infrastructure Hardware design and provenance; secure boot stack; premises security Google Cloud Platform enables developers to build, test, and deploy applications on Google’s highly secure, reliable, and scalable infrastructure. Cloud OnBoard
Review: Google Cloud Platform offers a range of compute services Compute
Compute Kubernetes App Engine Cloud Engine Engine Functions Cloud OnBoard
Google Cloud Platform offers a range of storage services
Compute Storage
Cloud Cloud Compute Kubernetes App Engine Cloud Bigtable Cloud Cloud SQL Spanner Datastore Engine Engine Functions Storage Cloud OnBoard
Google Cloud Platform offers services for getting value from data
Compute Storage
Cloud Cloud Compute Kubernetes App Engine Cloud Bigtable Cloud Cloud SQL Spanner Datastore Engine Engine Functions Storage
Big Data Machine Learning
BigQuery Pub/Sub Dataflow Dataproc Datalab Natural Vision API Machine Speech Translate Language Learning API API API 1 Cloud OnBoard
2 3 Agenda 5 6 Introduction to Google Cloud Platform 7
8 Quiz
9
10
11
12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6
7
8
9
10
11
12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 ● Sub-hour billing 7 ● Sustained-use discounts
8 ● Compute Engine custom machine types
9
10
11
12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 ● Sub-hour billing 7 ● Sustained-use discounts
8 ● Compute Engine custom machine types
9
10
11
12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 ● Sub-hour billing 7 ● Sustained-use discounts
8 ● Compute Engine custom machine types
9
10
11 ● Commitment to environmental responsibility 12 ● Commitment to open-source technologies 13 ● Robust infrastructure
14
15
16
17
18 1 Cloud OnBoard
2 3 More resources 5
6
7 Why Google Cloud Platform? Google Cloud Platform product overview 8 https://cloud.google.com/why-google/ http://cloud.google.com/products/ 9
10 Pricing philosophy Google Cloud Platform solutions 11 https://cloud.google.com/pricing/philosophy/ http://cloud.google.com/solutions/
12
13 Data centers
14 https://www.google.com/about/datacenters/
15
16
17
18 1
2
3
5
6
7 8 #GoogleCloudOnBoard 9
10
11
12
13
14
15
16 1
2
3
5
6 7 Module #2: Getting Started with 8 Google Cloud Platform 9
10
11
12
13
14
15
16 Cloud OnBoard
On- Infrastructure Platform as a Managed Responsibility premises as a Service Service services
Content Cloud security requires collaboration Access policies
Usage ● Google is responsible Deployment for Managing its infrastructure Security. Web application security
Identity ● You are responsible for Operations Securing your data.
Access and authentication ● Google helps you with best Network security practices, templates, OS, data, and content products, and solutions.
Audit logging
Network Customer-managed Google-managed Storage and encryption
Hardware 1 Cloud OnBoard
2 3 Agenda 5 6 Google Cloud Platform resource hierarchy 7
8 Identity and Access Management (IAM)
9
10 Cloud Identity
11 Interacting with Google Cloud Platform 12
13 GCP Marketplace 14
15 Quiz
16
17
18 1 Cloud OnBoard
2 3 Projects organize resources 5
6 ● Global resource collection ○ 7 ○ 8 ○ 9 ○
10
11 ● Provides an isolation boundary
12 between resources ○ 13
14
15 ● All Google Cloud Platform services
16 you use are associated with one
17 and only project
18 1 Cloud OnBoard
2 3 Resource hierarchy levels define 5 trust boundaries 6
7 ● Group your resources with folders 8 and projects according to your 9 organization structure 10
11 ● Levels of the hierarchy provide trust 12 boundaries and resource isolation
13
14
15
16
17
18 Cloud OnBoard Cloud OnBoard
The organization node organizes projects
● The organization node is the root node for Google Cloud resources
● Notable organization roles: [email protected] ○ Organization Admin ■
○ ■
Create
Ex Mail [email protected] Ex Drive Project Creator 1 Cloud OnBoard
2 An example IAM resource hierarchy 3
5
6 ● A policy is set on a resource ○ 7 Organization
8
9
10 ● Resources inherit policies Folders
11 from parent
12 ○ Projects 13
14 15 ● A less restrictive parent
16 policy overrides a more Resources 17 restrictive resource policy
18 1 Cloud OnBoard
2 3 Agenda 5 6 Google Cloud Platform resource hierarchy 7
8 Identity and Access Management (IAM)
9
10 Interacting with Google Cloud Platform
11 GCP Marketplace 12
13 Quiz 14
15
16
17
18 1 Cloud OnBoard
2 3 Cloud Identity 5
6
7 ● Integrate your cloud and on-premises 8 directories in one IDaaS platform 9
10 ● Single sign-on supports SAML 2.0, 11 OAuth 2.0 and OpenID
12
13 ● Google grade security and scale
14
15 ● Suspicious activity detection ○ 16 ○ 17 ○
18 Cloud OnBoard
Google’s Current Offering
Access (SAML + OIDC)
Provisioning Cloud Identity
sync Cloud OnBoard
One independent platform to host and manage identity Cloud OnBoard
Managing Identity And Access
For managing users and For granting authorization authentication to cloud resources
Cloud Identity Cloud Console IAM admin console
● User accounts ● Defining Identity and Access Management roles ● Groups ● Authentication options for developers Cloud OnBoard
Each action in your environment needs to answer 3 questions
who can do what on which resource 1 Cloud OnBoard
2 3 The most common ways to identify 5 users or machines are 6 who
7
8 Organization-managed users hosted of Google’s secure 9 IDaaP Including GSuite Users 10 [email protected] 11
12 User managed Google account 13 test@gmail.com 14
15 Service account 16 test@project_id.iam.gserviceaccount.com 17
18 1 Cloud OnBoard
2 3 Service Account 5 Belongs to your application or a virtual machine (VM),
6 instead of to an individual end user
7 ● Provide a machine identity for carrying out server-to-server/service 8 interactions 9
10 ● Default service accounts managed by Google 11
12
13 ● User Defined Service Accounts
14
17
18 1 Cloud OnBoard
2
3
5 There are three types of IAM roles
6 can do what
7
8
9
10
11
12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 IAM predefined roles 5 A set of permissions that grouped together
6
7 can do what on resources in this project, folder, or org 8
9
10 InstanceAdmin Role
11 compute.instances.delete 12 compute.instances.get 13 compute.instances.list 14 compute.instances.setMachineType 15 compute.instances.start 16 compute.instances.stop 17 example.com 18
2 3 IAM Custom roles 5 lets you define a precise set of permissions
6
7 can do what on resources in this project, folder, or org 8
9
10 SecurityAudit Role
11 compute.instances.get 12 compute.instances.list 13 containers.pods.getLogs 14 appengine.instances.get 15 logging.logs.list
16
17 example.com
18 Cloud OnBoard
IAM primitive roles apply across all GCP services in a project
can do what on all project resources Cloud OnBoard
IAM primitive roles offer fixed, coarse-grained levels of access
Viewer Editor Owner Billing Admin Access
x x Manage billing
x x Add and remove administrators
x x x Read-only access A project can have multiple x x Configure services owners, editors, x x Modify code viewers, and billing administrators. x Deploy applications
x Invite members
x Remove members
x Delete projects 1 Cloud OnBoard
2 3 Google Groups Best Practices 5
6
7 ● Assign permissions to groups rather than ● Create Groups for each team in your 8 individuals organisation 9
10 ● Make Groups own resources and projects ● Nest Groups for fine grain control 11 for continuity 12 ● Groups can also contain service accounts
13
14 Developers NetOps 15 SecOps
16
17 App A App B 18 1 Cloud OnBoard
2 3 Audit Logs 5
6
7
8
9
10
11 Cloud console 12 activity page
13
14
15 Stackdriver 16 logging
17
18 Cloud OnBoard
Principle of least privilege
Everybody Owner Organization
Security Admin Security Project A Group A Admin Role Cloud OnBoard
Example: Service Accounts and IAM
● VMs running FrontEnd are granted Ex Mail Ex Drive Editor access to project_b using Service Account 1
● VMs running BackEnd are granted FrontEnd VM Service objectViewer access to bucket_1 Account 1 Editor using Service Account 2
● Service account permissions can be changed without BackEnd VM Service recreating VMs Account 2 Storage. objectViewer
bucket_1 1 Cloud OnBoard
2 3 Agenda 5 6 Google Cloud Platform resource hierarchy 7
8 Identity and Access Management (IAM)
9
10 Interacting with Google Cloud Platform
11 GCP Marketplace 12
13 Quiz 14
15
16
17
18 Cloud OnBoard
There are four ways to interact with GCP
>_ Cloud OnBoard
Google Cloud Platform Console
● Centralized console for all project data
● Developer tools ○ ○ ○
● Access to product APIs
● Manage and create projects Cloud OnBoard
Google Cloud SDK
● SDK includes CLI tools for Cloud Platform products and services ○
● Available as Docker image
● Available via Cloud Shell ○ 1 Cloud OnBoard
2 3 RESTful APIs 5
6 ● Programmatic access to products and services
7 ○
8 ○
9
10 ● Enabled through the Google Cloud Platform Console
11
12 ● Most APIs include daily quotas and rates (limits) that can be
13 raised by request
14 ○
15 ● Experiment with APIs Explorer 16
17
18 Cloud OnBoard
Cloud Console Mobile App
● Manage virtual machines and database instances
● Manage apps in Google App Engine
● Manage your billing
● Visualize your projects with a customizable dashboard 1 Cloud OnBoard
2 3 APIs Explorer 5
6 ● The APIs Explorer is an interactive tool that lets you easily try Google
7 APIs using a browser.
8
9 ● With the APIs Explorer, you can:
10 ○
11 ○
12
13 ○ ○ 14
15
16
17
18 1 Cloud OnBoard
2 3 Client Libraries 5
6 ● Cloud Client Libraries
7 ○
8
9 ● Google API Client Libraries
10 ○ ○ 11 ■ 12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Agenda 5 6 Google Cloud Platform resource hierarchy 7
8 Identity and Access Management (IAM)
9
10 Interacting with Google Cloud Platform
11 GCP Marketplace 12
13 Quiz 14
15
16
17
18 Cloud OnBoard
GCP Marketplace gives quick access to solutions
● A solution marketplace containing pre-packaged, ready-to-deploy solutions ○ ○
● You pay for the underlying GCP resource usage. ○ 1 Cloud OnBoard
2 3 Agenda 5 6 Google Cloud Platform resource hierarchy 7
8 Identity and Access Management (IAM)
9
10 Interacting with Google Cloud Platform
11 GCP Marketplace 12
13 Quiz 14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6
7
8
9
10
11
12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 False: Policies are a union of the parent and 7 the resource. If a parent policy is less
8 restrictive, it overrides a more restrictive resource policy. 9
10
11
12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 False: Policies are a union of the parent and 7 the resource. If a parent policy is less
8 restrictive, it overrides a more restrictive resource policy. 9
10
11
12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 False: Policies are a union of the parent and 7 the resource. If a parent policy is less
8 restrictive, it overrides a more restrictive resource policy. 9
10
11
12
13
14 True: All Google Cloud Platform resources are associated with a project. 15
16
17
18 1 Cloud OnBoard
2 3 Quiz: Service Accounts 5
6 Service accounts are used to provide which of the following?
7
8 ❏ Authentication between Google Cloud Platform services
9
10 ❏ Key generation and rotation when used with App Engine and Compute Engine 11
12 ❏ A way to restrict the actions a resource (such as a VM) can perform 13
14 ❏ A way to allow users to act with service account permissions 15
16 ❏ All of the above
17
18 1 Cloud OnBoard
2 3 Quiz: Service Accounts 5
6 Service accounts are used to provide which of the following?
7
8 ❏ Authentication between Google Cloud Platform services
9
10 ❏ Key generation and rotation when used with App Engine and Compute Engine 11
12 ❏ A way to restrict the actions a resource (such as a VM) can perform 13
14 ❏ A way to allow users to act with service account permissions 15
16 ✓ All of the above
17
18 1 Cloud OnBoard
2 3 More resources 5
6
7 Google Cloud Platform security Cloud SDK installation and quick start https://cloud.google.com/security/ https://cloud.google.com/sdk/#Quick_Start 8
9
10 Configuring permissions Google Cloud Platform solutions 11 https://cloud.google.com/docs/permissions- http://cloud.google.com/solutions/ 12 overview
13
14
15 Identity and Access Management (IAM)
16 https://cloud.google.com/iam/
17
18 1
2
3
5
6
7 8 #GoogleCloudOnBoard 9
10
11
12
13
14
15
16 1
2
3
5
6
7 8 [Lunch] 9
10
11
12
13
14
15
16 1
2
3
5
6 7 Demo: 8 Get Hands-on with Qwiklabs 9
10
11
12
13
14
15
16 Cloud OnBoard
Getting started on your Google Cloud learning journey
1 2 3
Today Tomorrow Future Google Cloud Platform Complete hands-on labs: Find more training online Fundamentals: GCP Essentials Quest cloud.google.com/training Core Infrastructure google.qwiklabs.com Cloud OnBoard
Qwiklabs provides a hands-on labs environment to learn GCP
1 2 3 4
Sign in to Choose a Lab Get a time bound Master a Skill Qwiklabs 150+ labs live GCP 18 Quests Environment Cloud OnBoard
1 month free access to Qwiklabs
1 Receive a follow up email after this event
2 Follow the link to Qwiklabs and enroll in a Quest
3 Create your Qwiklabs account if you don’t have one already
4 Log in and take your first lab 1
2
3
5
6 7 Module #3: 8 Virtual Machines in the Cloud 9
10
11
12
13
14
15
16 1 Cloud OnBoard
2 3 Agenda 5 6 Virtual Private Cloud (VPC) Network 7
8 Compute Engine
9
10 Operations and tools
11 Quiz 12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Virtual Private Cloud Network 5
6
7
8 ● Build your own VPC on top Google’s physical global fiber network, powered by
9 Andromeda
10
11 ● Gives you the flexibility to build solutions that scale and communicate across regions and continents 12
13 ● Abstracts away lower level management so you can focus on creating an 14 elastic, flexible and cost effective solution 15
16
17
18 1 Cloud OnBoard
2 3 Virtual Private Cloud Network 5
6 Key Features
7
8 ● Global Load Balancer with a single anycast IP
9
10 ● Subnets span across Regions
11 ● Software defined Routers 12
13 ● Easy ways to share and peer your network 14
15 ● Flexible Firewall Rules
16
17 ● Up to 5 global networks per project
18 Cloud OnBoard
Google Cloud VPCs are global; subnets are regional
us-east1
us-east1-b us-east1-c my-subnet1
10.0.0.0/24 10.0.0.2 10.0.0.3 Cloud OnBoard
Project
Network Prod Network #2 Network Dev Network Corp Network #5
asia-east1
B europe-west1
us-central1
us-west1
A C D us-east1
The Internet
● A and B can communicate over internal IPs even though they are in different regions. ● C and D must communicate over external IPs even though they are in the same region. Cloud OnBoard
Google Cloud Platform offers many connectivity options
VPN Interconnect Direct Peering 1 Cloud OnBoard
2 3 Cloud Global Load Balancing: 5
6 HTTP(S)
7
8 ● Global single anycast IP address routes traffic based on location or URL path
9
10 ● Balance HTTP(S) traffic across multiple Compute Engine regions
11 ● Integrated health checks means traffic is directed only to healthy instances 12
13 ● Terminates SSL at the load balancer 14
15 ● Scalable, requires no pre-warming and provides resilience, fault tolerant 16 architecture
17
18 Cloud OnBoard
Global Load Balancer 1 Cloud OnBoard
2 3 Cloud Load Balancing Content Base 5
6
7
8 Spread HTTP(S) traffic over different pool of Global Load Balancer
9 instances based on the URL Pattern.
10
11 Traffic is directed only to instances that pass health checks and has capacity 12 Scalable, requires no pre-warming 13
14
15
16
17 Default /video 18 Handler 1 Cloud OnBoard
2 3 Cloud Load Balancing - 5 TCP/SSL and UDP 6
7
8 ● Spread TCP/SSL and UDP traffic over pool of instances within a
9 Compute Engine region
10
11 ● Traffic is directed only to instances that pass health checks
12 ● Scalable, requires no pre-warming 13
14
15
16
17
18 1 Cloud OnBoard
2 3 Cloud Load Balancers 5
6
7
8
9
10
11 Global HTTP(S) SSL Proxy Proxy
12
13
14 Regional Network Internal 15
16
17
18 1 Cloud OnBoard
2 3 Cloud DNS is cost-effective way to 5 make your applications and services 6
7 available to your users
8 ● Low latency solution 9
10 ● Fast Anycast Name Servers 11
12 ● Automatic Scaling
13
14 ● 100% SLA
15
16 ● Create managed zones, then add, edit, delete DNS records ○ 17
18 1 Cloud OnBoard
2 3 Cloud CDN (Content Delivery Network) 5
6
7
8 ● Use Google's globally distributed edge caches to cache HTTP(S) load-balanced content far
9 closer to your users than your instances ○ 10
11 ● Cloud CDN uses caches at network locations to store responses generated by instances 12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Agenda 5 6 Virtual Private Cloud (VPC) Network 7
8 Compute Engine
9
10 Operations and tools
11 Quiz 12
13
14
15
16
17
18 Cloud OnBoard 1 Cloud OnBoard
2 3 Compute Engine 5
6 Scalable, High Performance Virtual Machines
7
8 ● Performance consistency, boots quickly
9
10 ● Custom Machine Types and industry-leading local SSD Performance
11 ● Resize disks with no downtime 12
13 ● Scales from single instances to global, load-balanced cloud computing 14
15 ● Various pricing innovation
16
17 ● Transparent Maintenance, automatic restarts and health-checks
18 1 Cloud OnBoard
2 3 Compute Engine 5
6 IaaS that is build for the Cloud era
7
8 ● Managed instance groups that will horizontally scale your fleet
9 ○ ○ 10 ○ 11
12 ● Recommendation Engine
13
14 ● Inferred instance discount
15
16 ● Preemptible Machines
17 ● Sustained Use Discounts 18 Cloud OnBoard
Automatic Sustained Use Discounts
-10% 100% -20% -30% 75% Price
50% 24% Monthly Usage 25%
0% 25% 50% 75% 100% Cloud OnBoard
Rightsizing Recommendations Optimize for your usage
2 instances could be resized to save an estimated $33 per month Cloud OnBoard
Resource based instances discount Cloud OnBoard
Preemptible VMs Up to 80% cheaper for short-lived instances CPU and GPU 1 Cloud OnBoard
2 3 Agenda 5 6 Virtual Private Cloud (VPC) Network 7
8 Compute Engine
9
10 Operations and tools
11 Quiz 12
13
14
15
16
17
18 Cloud OnBoard
VPC Network offers many internetworking features
● Fine-grained networking policies
● Fine-grained IP address range selection
● Routes
● Firewalls
● Virtual Private Network (VPN)
● Cloud Router 1 Cloud OnBoard
2
3 Google Cloud Platform offers many 5 interconnect options
6
7
8
9
10 Carrier Interconnect Direct Peering CDN Interconnect
11 Enterprise-grade Connect your business Allows select CDN providers to establish direct
12 connections provided by directly to Google interconnect links with Google’s edge network at carrier service providers various locations 13
14
15
16
17 Google Cloud Interconnect 18 Cloud OnBoard
Cloud DNS is highly available and scalable
● Create managed zones, then add, edit, delete DNS records ○ Cloud OnBoard
Cloud Load Balancing: HTTP(S)
● Balance HTTP-based traffic across multiple Compute Engine regions
● Global, external IP address routes traffic
● Traffic is directed only to instances that pass health checks
● Scalable, requires no pre-warming and provides resilience, fault tolerance Cloud OnBoard
Cloud Load Balancing: TCP/SSL, UDP
● Spread TCP/SSL and UDP traffic over pool of instances within a Compute Engine region
● Traffic is directed only to instances that pass health checks
● Scalable, requires no pre-warming Cloud OnBoard
Cloud CDN (Content Delivery Network)
● Use Google's globally distributed edge caches to cache HTTP(S) load-balanced content far closer to your users than your instances ○
● Cloud CDN uses caches at network locations to store responses generated by instances 1 Cloud OnBoard
2 3 Agenda 5 6 Virtual Private Cloud (VPC) Network 7
8 Compute Engine
9
10 Operations and tools
11 Quiz 12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 Name 3 robust networking services
7 available to your applications on Google Cloud Platform. 8
9
10
11
12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 Name 3 robust networking services Cloud Virtual Network, Cloud
7 available to your applications on Google Interconnect, Cloud DNS, Cloud Load Cloud Platform. Balancing, and Cloud CDN. 8
9
10
11
12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 Name 3 robust networking services Cloud Virtual Network, Cloud
7 available to your applications on Google Interconnect, Cloud DNS, Cloud Load Cloud Platform. Balancing, and Cloud CDN. 8
9
10
11 Name 3 Compute Engine pricing
12 innovations.
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 Name 3 robust networking services Cloud Virtual Network, Cloud
7 available to your applications on Google Interconnect, Cloud DNS, Cloud Load Cloud Platform. Balancing, and Cloud CDN. 8
9
10
11 Name 3 Compute Engine pricing Per-second billing, custom machine
12 innovations. types, preemptible instances.
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 Name 3 robust networking services Cloud Virtual Network, Cloud
7 available to your applications on Google Interconnect, Cloud DNS, Cloud Load Cloud Platform. Balancing, and Cloud CDN. 8
9
10
11 Name 3 Compute Engine pricing Per-second billing, custom machine
12 innovations. types, preemptible instances.
13
14 True or False: Google Cloud Load 15 Balancing lets you balance HTTP traffic 16 across multiple Compute Engine regions. 17
18 1 Cloud OnBoard
2 3 Quiz 5
6 Name 3 robust networking services Cloud Virtual Network, Cloud
7 available to your applications on Google Interconnect, Cloud DNS, Cloud Load Cloud Platform. Balancing, and Cloud CDN. 8
9
10
11 Name 3 Compute Engine pricing Per-second billing, custom machine
12 innovations. types, preemptible instances.
13
14 True or False: Google Cloud Load True. 15 Balancing lets you balance HTTP traffic 16 across multiple Compute Engine regions. 17
18 1 Cloud OnBoard
2 3 More resources 5
6 Google Compute Engine 7 https://cloud.google.com/compute/docs/ 8
9 Google Cloud Platform VPC
10 https://cloud.google.com/compute/docs/vpc/
11
12 Google Cloud Stackdriver https://cloud.google.com/stackdriver/docs/ 13
14 Google Cloud Source Repositories gcloud tool guide 15 https://cloud.google.com/source-repositories/docs/ 16
17
18 1
2
3
5
6
7 8 #GoogleCloudOnBoard 9
10
11
12
13
14
15
16 1
2
3
5
6 7 Module #4: 8 Storage in the Cloud 9
10
11
12
13
14
15
16 1 Cloud OnBoard
2 3 Agenda 5 6 Cloud Storage 7
8 Cloud SQL and Cloud Spanner
9
10 Cloud Bigtable
11 Cloud Datastore 12
13 Comparing storage options 14
15 Integrations with other services
16
17 Quiz
18 Cloud OnBoard
Google Cloud Platform
Machine Operations Compute Networking Big Data Storage Learning and Tools
Cloud Cloud Cloud Cloud Cloud Storage SQL Spanner Datastore Bigtable Cloud OnBoard
Cloud Storage is binary large-object storage
● High performance, internet-scale
● Simple administration ○
● Data encryption at rest
● Data encryption in transit by default from Google to endpoint
● Online and offline import services are available 1 Cloud OnBoard
2 3 Your Cloud Storage files are organized 5 into buckets 6
7 Bucket attributes: Bucket contents:
8 ● Globally unique name ● Files (in a flat namespace) 9 ● Storage class ● Access Control Lists 10 ● Location 11 ○ 12 ● IAM policies or 13
14 ● Access Control Lists
15 ● Object versioning setting 16 ● Object lifecycle management rules
17
18 Cloud OnBoard
Choosing among Cloud Storage classes
Multi-regional Regional Nearline Coldline
Intended for data Most frequently Accessed frequently Accessed less than Accessed less than that is... accessed within a region once a month once a year
Availability SLA 99.95% 99.90% 99.00% 99.00%
Access APIs Consistent APIs
Access time Millisecond access Price per GB stored per month Storage price
Total price per GB transferred Retrieval price
Use cases Content storage and In-region analytics, Long-tail content, Archiving, delivery transcoding backups disaster recovery “Thanks to Google Cloud Platform and the Google Genomics team, the greatest minds in science from around the world will be able to study trillions of data points in one single database.” 100 200 PER RAW UPLOADED TERABYTES UP TO GIGABYTES GENOME of data from more than MSSNG project 1,300 WHOLE could easily surpass a
GENOMES PETABYTE OF DATA to Google Cloud Storage
WHOLE GENOMES FROM Making MSSNG world’s largest single repository of autism-related DNA sequencing data 10,000 PEOPLE 1 Cloud OnBoard
2 3 Agenda 5 6 Cloud Storage 7
8 Cloud SQL and Cloud Spanner
9
10 Cloud Bigtable
11 Cloud Datastore 12
13 Comparing storage options 14
15 Integrations with other services
16
17 Quiz
18 Cloud OnBoard
Cloud SQL is a managed RDBMS
● Offers MySQL and PostgreSQLBeta databases as a service
● Automatic replication
● Managed backups
● Vertical scaling (read and write)
● Horizontal scaling (read)
● Google security Cloud OnBoard
Cloud Spanner is a horizontally scalable RDBMS
Cloud Spanner supports:
● Automatic replication
● Strong global consistency
● Managed instances with high availability
● SQL (ANSI 2011 with extensions) 1 Cloud OnBoard
2 3 Agenda 5 6 Cloud Storage 7
8 Cloud SQL and Cloud Spanner
9
10 Cloud Bigtable
11 Cloud Datastore 12
13 Comparing storage options 14
15 Integrations with other services
16
17 Quiz
18 Cloud OnBoard
Cloud Bigtable is managed NoSQL
● Fully managed NoSQL, wide-column database service for terabyte applications
● Integrated ○ ○ Cloud OnBoard
Why choose Cloud Bigtable?
● Replicated storage
● Data encryption in-flight and at rest
● Role-based ACLs
● Drives major applications such as Google Analytics and Gmail Overview: Challenges: Data to process: Data in the Consolidated Audit Trail (CAT). How to process the CAT and organize 100 billion A data repository of all equities and options orders, quotes, market events into an “order lifecycle” in a 4 hour and events window. Store 6 years (~30PB) of data
Cloud Bigtable to process and run queries and tolerate volume increases
6 BILLION 1.7 GIGs 10 BN 1.7 GIGABYTES PER SECOND PER SECOND MARKET EVENTS WRITTEN PER HOUR BURSTS 10 TERABYTES WRITTEN PER HOUR 6 TBs PER HOUR PER HOUR Cloud OnBoard
Bigtable Access Patterns
Application API
Streaming
Batch Processing Cloud Bigtable 1 Cloud OnBoard
2 3 Agenda 5 6 Cloud Storage 7
8 Cloud SQL and Cloud Spanner
9
10 Cloud Bigtable
11 Cloud Datastore 12
13 Comparing storage options 14
15 Integrations with other services
16
17 Quiz
18 Cloud OnBoard
Cloud Datastore is a horizontally scalable NoSQL DB
NoSQL designed for application backends
● Fully managed
● Uses a distributed architecture
● to automatically manage scaling
● Built-in redundancy
● Supports ACID transactions Cloud OnBoard
Google Cloud Datastore: benefits
● Schemaless access ○
● Local development tools
● Includes a free daily quota
● Access from anywhere through a RESTful interface 1 Cloud OnBoard
2 3 Agenda 5 6 Cloud Storage 7
8 Cloud SQL and Cloud Spanner
9
10 Cloud Bigtable
11 Cloud Datastore 12
13 Comparing storage options 14
15 Integrations with other services
16
17 Quiz
18 Cloud OnBoard
Comparing storage options: technical details
Cloud Cloud Cloud Bigtable Cloud SQL BigQuery Datastore Storage Spanner
Type NoSQL NoSQL Blobstore Relational SQL Relational SQL Relational SQL document wide column for OLTP for OLTP for OLAP
Transaction Yes Single-row No Yes Yes No s
Complex No No No Yes Yes Yes queries
Capacity Terabytes+ Petabytes+ Petabytes+ 500 GB Petabytes Petabytes+
Unit size 1 MB/entity ~10 MB/cell 5 TB/object Determined by 10,240 MiB/ 10 MB/row ~100 MB/row DB engine row Cloud OnBoard
Comparing storage options: technical details
Cloud Cloud Cloud Bigtable Cloud SQL BigQuery Datastore Storage Spanner
Type NoSQL NoSQL Blobstore Relational SQL Relational SQL Relational SQL document wide column for OLTP for OLTP for OLAP
Best for Getting “Flat” data, Structured and Web Large-scale Interactive started, App Heavy read/ unstructured frameworks, database querying, offline Engine write, events, binary or object existing applications (> analytics applications analytical data data applications ~2 TB)
Use cases Getting AdTech, Images, large User Whenever high Data started, App Financial and media files, credentials, I/O, global warehousing Engine IoT data backups customer consistency is applications orders needed 1 Cloud OnBoard
2 3 Agenda 5 6 Cloud Storage 7
8 Cloud SQL and Cloud Spanner
9
10 Cloud Bigtable
11 Cloud Datastore 12
13 Comparing storage options 14
15 Integrations with other services
16
17 Quiz
18 Cloud OnBoard
Cloud Storage is integrated with other GCP services
Import and Startup scripts, export tables images, and general object storage
BigQuery Compute Engine
Cloud Object storage, Storage Import and logs, and export tables Datastore backups
App Cloud SQL Engine Cloud OnBoard
Cloud Bigtable is integrated with other GCP services
Google Cloud Dataflow Use Cloud Dataflow connector for Bigtable for batch and streaming operations in pipelines.
Google Cloud Dataproc Use Bigtable HBase client to integrate Hadoop jobs with Cloud Dataproc. Cloud Bigtable On-premises, cloud-based Hadoop External service Use Bigtable HBase client to integrate with Hadoop clusters. Cloud OnBoard
Cloud SQL is integrated with other GCP services
External service
Cloud SQL can be used Compute Engine instances Cloud SQL can be used with with App Engine using can be authorized to access external applications and clients. standard drivers. Cloud SQL instances using an external IP address. Standard tools can be used to You can configure a Cloud SQL administer databases. instance to follow an App Cloud SQL instances can be Engine application. configured with a preferred zone. External read replicas can be configured. 1 Cloud OnBoard
2 3 Agenda 5 6 Cloud Storage 7
8 Cloud SQL and Cloud Spanner
9
10 Cloud Bigtable
11 Cloud Datastore 12
13 Comparing storage options 14
15 Integrations with other services
16
17 Quiz
18 1 Cloud OnBoard
2 3 Quiz 5
6 Your application transcodes large video
7 files. Which storage service should you consider first? 8
9
10
11
12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 Your application transcodes large video Google Cloud Storage
7 files. Which storage service should you consider first? 8
9
10
11
12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 Your application transcodes large video Google Cloud Storage
7 files. Which storage service should you consider first? 8
9
10
11 You stream huge amounts of data from
12 devices with sensors. Which storage service should you consider first? 13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 Your application transcodes large video Google Cloud Storage
7 files. Which storage service should you consider first? 8
9
10
11 You stream huge amounts of data from Google Cloud Bigtable
12 devices with sensors. Which storage service should you consider first? 13
14
15
16
17
18 1 Cloud OnBoard
2 3 More resources 5
6 Overview of Cloud Storage Cloud Spanner 7 https://cloud.google.com/storage/ https://cloud.google.com/spanner/docs/ 8
9 Getting started with Google Cloud SQL Cloud Datastore
10 https://cloud.google.com/sql/docs/quickstart https://cloud.google.com/datastore/docs/
11
12 Cloud Bigtable https://cloud.google.com/stackdriver/docs/ 13
14
15
16
17
18 1
2
3
5
6
7 8 #GoogleCloudOnBoard 9
10
11
12
13
14
15
16 1
2
3
5
6 7 Module #5: 8 Containers in the Cloud 9
10
11
12
13
14
15
16 Cloud OnBoard
Review: IaaS and PaaS
Toward Compute Engine Kubernetes Engine App Engine Toward managed managed infrastructure services IaaS PaaS Raw compute, storage, and network Preset run-times More granular control Java, Go, PHP, Python... Focus is application logic
Pay for what you allocate Pay for what you use More management overhead Less management overhead 1 Cloud OnBoard
2 3 Agenda 5 6 Introduction to Containers 7
8 Kubernetes
9
10 Kubernetes Engine
11 Quiz 12
13
14
15
16
17
18 Cloud OnBoard
How do virtual machines and containers differ?
Hypervisor-based virtualization Container-based virtualization
App App App
Libs Libs Libs App App App
Libs Libs Libs
Guest Guest Guest Container runtime OS OS OS Host OS Kernel
Hardware Hypervisor
Hardware 1 Cloud OnBoard
2 3 Why use containers? 5
6
7
8 Consistency Loose coupling Workload migration Agility 9
10
11
12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Agenda 5 6 Introduction to Containers 7
8 Kubernetes
9
10 Kubernetes Engine
11 Quiz 12
13
14
15
16
17
18 Cloud OnBoard
Kubernetes is a container cluster orchestration system
● Automates deployment, scaling, and operations for container clusters
● Open source, based on Google’s experience over 10+ years
● Built for a multi-cloud world ○ Cloud OnBoard
Google Cloud Datastore: benefits
● Workload portability ○
○
● Rolling updates ○
● Persistent storage ○ Cloud OnBoard
Kubernetes makes applications more elastic
● Multi-zone clusters ○
● Load balancing ○
● Autoscaling ○ 1 Cloud OnBoard
2 3 Agenda 5 6 Introduction to Containers 7
8 Kubernetes
9
10 Kubernetes Engine
11 Quiz 12
13
14
15
16
17
18 Cloud OnBoard
Kubernetes Engine manages and runs containers
● Fully managed cluster management and orchestration system for running containers ○ ○
● Uses a declarative syntax to manage applications ○ Cloud OnBoard
Why use Kubernetes Engine?
● Decouples operational, development concerns
● Manages and maintains ○
● Easily update Kubernetes versions as they are released 1 Cloud OnBoard
2 3 Kubernetes Engine’s complementary services 5
6
7
8
9
10
11
12
13
14
15
16
17
18 Cloud OnBoard
Treeptik
“Our platform sometimes has to be deployed on a cluster. How do we enable containers to communicate from different hosts? Google has the answer: Kubernetes. This awesome tool helps us manage our clusters of containers as if they were a single system.”
scale speed -30% Docker containers REST APIs speed provisioning of Administrative costs automate scalability new instances; JAVA applications reduced by 30% can be deployed in minutes 1 Cloud OnBoard
2
3 Deploying Apps: Kubernetes Engine vs App Engine
5
6
7 Kubernetes App Engine App Engine Flexible 8 Engine Standard
9 Language support Any Java, Python, Go & Any 10 PHP 11
12 Service model Hybrid PaaS PaaS
13 Container-based Web and mobile Web and mobile 14 Primary use case workloads applications applications, 15 container-based 16 workloads
17
18 Cloud OnBoard
Free Course: Learn more about Kubernetes Engine on Coursera Activate your free voucher now
Go to 1 https://www.coursera.org/voucher/Cloudonboard
Activate voucher and sign 2 up for a free account
Enroll in Getting Started with Google 3 Kubernetes Engine -Limited time offer!!
Explore other Courses at Coursera.org/Googlecloud 1 Cloud OnBoard
2 3 Agenda 5 6 Introduction to Containers 7
8 Kubernetes
9
10 Kubernetes Engine
11 Quiz 12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 Name two reasons for deploying
7 applications using containers.
8
9
10
11
12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 Name two reasons for deploying Consistency across development,
7 applications using containers. testing,and production environments; Simpler to migrate workloads; Loose 8 coupling; Agility 9
10
11
12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 Name two reasons for deploying Consistency across development,
7 applications using containers. testing,and production environments; Simpler to migrate workloads; Loose 8 coupling; Agility 9
10
11 True or False: Kubernetes lets you manage container clusters in multiple 12 cloud providers. 13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 Name two reasons for deploying Consistency across development,
7 applications using containers. testing,and production environments; Simpler to migrate workloads; Loose 8 coupling; Agility 9
10
11 True or False: Kubernetes lets you True manage container clusters in multiple 12 cloud providers. 13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 Name two reasons for deploying Consistency across development,
7 applications using containers. testing,and production environments; Simpler to migrate workloads; Loose 8 coupling; Agility 9
10
11 True or False: Kubernetes lets you True manage container clusters in multiple 12 cloud providers. 13
14
15 True or False: GCP provides a private,
16 high-speed container image storage service for use with Kubernetes Engine. 17
18 1 Cloud OnBoard
2 3 Quiz 5
6 Name two reasons for deploying Consistency across development,
7 applications using containers. testing,and production environments; Simpler to migrate workloads; Loose 8 coupling; Agility 9
10
11 True or False: Kubernetes lets you True manage container clusters in multiple 12 cloud providers. 13
14
15 True or False: GCP provides a private, True
16 high-speed container image storage service for use with Kubernetes Engine. 17
18 1 Cloud OnBoard
2 3 More resources 5
6 Kubernetes Engine Google Cloud Container Builder 7 https://cloud.google.com/container- https://cloud.google.com/container- 8 engine/docs/ builder/docs/
9
10 Kubernetes Engine tutorials Google Container Registry
11 https://cloud.google.com/container-engine/ https://cloud.google.com/container-
12 docs/tutorials registry/docs/
13 Kubernetes 14 http://kubernetes.io/ 15
16
17
18 1
2
3
5
6
7 8 #GoogleCloudOnBoard 9
10
11
12
13
14
15
16 1
2
3
5
6
7 8 [Break] 9
10
11
12
13
14
15
16 1
2
3
5
6 7 Module #6: 8 Applications in the Cloud 9
10
11
12
13
14
15
16 1 Cloud OnBoard
2 3 Agenda 5 6 Google App Engine 7
8 Google App Engine Standard Environment
9
10 Google App Engine Flexible Environment
11 Google Cloud Endpoints and Apigee Edge 12
13 Quiz 14
15
16
17
18 Cloud OnBoard
App Engine is a PaaS for building scalable applications
● App Engine makes deployment, maintenance, and scalability easy so you can focus on innovation
● Especially suited for building scalable web applications and mobile backends 1 Cloud OnBoard
2 3 Agenda 5 6 Google App Engine 7
8 Google App Engine Standard Environment
9
10 Google App Engine Flexible Environment
11 Google Cloud Endpoints and Apigee Edge 12
13 Quiz 14
15
16
17
18 Cloud OnBoard
App Engine standard environment
● Easily deploy your applications
● Autoscale workloads to meet demand
● Economical ○ ○
● SDKs for development, testing and deployment Cloud OnBoard
App Engine standard environment: Requirements
● Specific versions of Java, Python, PHP, and Go are supported
● Your application must conform to sandbox constraints: ○ ○ ○ Cloud OnBoard
Example App Engine standard workflow: Web applications
App Engine automatically App Engine can access a 3 scales & reliably serves your variety of services using dedicated APIs 1 Develop & test the web web application application locally Project Memcache App Engine App Servers Task queues Application Use the SDK to deploy to 2 instances Scheduled App Engine tasks Application instances Search Application instances Logs 1 Cloud OnBoard
2 3 Agenda 5 6 Google App Engine 7
8 Google App Engine Standard Environment
9
10 Google App Engine Flexible Environment
11 Google Cloud Endpoints and Apigee Edge 12
13 Quiz 14
15
16
17
18 Cloud OnBoard
App Engine flexible environment
● Build and deploy containerized apps with a click
● No sandbox constraints
● Can access App Engine resources
● Standard runtimes: Python, Java, Go, Node.js
● Custom runtime support: Any language that supports HTTP requests
● Package your runtime as a Dockerfile Cloud OnBoard
Comparing the App Engine environments
Standard Environment Flexible Environment
Instance startup Milliseconds Minutes
SSH access No Yes (although not by default)
Scaling Manual, basic, automatic Manual, automatic
Write to local disk No Yes (but writes are ephemeral)
Support for 3rd-party No Yes binaries
Network access Via App Engine services Yes
Pricing model After free daily use, pay per instance class, Pay for resource allocation per hour; no with automatic shutdown automatic shutdown Cloud OnBoard
CompaDeploying Apps: Kubernetes Engine vs App Engine ring the AppKubernetes Engine Engine environmentsApp Engine Flexible App Engine Standard
Language Any Any Java, Python, Go, PHP support
Service model Hybrid PaaS PaaS
Primary use Container-based workloads Web and mobile applications, Web and mobile applications case container-based workloads
Toward managed infrastructure Toward dynamic infrastructure 1 Cloud OnBoard
2 3 Agenda 5 6 Google App Engine 7
8 Google App Engine Standard Environment
9
10 Google App Engine Flexible Environment
11 Google Cloud Endpoints and Apigee Edge 12
13 Quiz 14
15
16
17
18 Cloud OnBoard
Cloud Endpoints helps you create and maintain APIs
● Distributed API management through an API console
● Expose your API using a RESTful interface
● Control access and validate calls with JSON Web Tokens and Google API keys
● Identify web, mobile users with Auth0 and Firebase Authentication
● Generate client libraries Cloud OnBoard
Cloud Endpoints: Supported platforms
● Supports App Engine standard or flexible environment, Compute Engine, Kubernetes Engine
● Use Java or Python open-source Frameworks or any other framework and language
● Supports iOS, Android, and JavaScript clients 1 Cloud OnBoard
2 3 Apigee Edge helps you secure and 5 monetize APIs 6
7 1. A platform for making APIs available to your customers 8
9 and partners
10
11 2. Contains analytics, monetization, and a developer portal
12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Agenda 5 6 Google App Engine 7
8 Google App Engine Standard Environment
9
10 Google App Engine Flexible Environment
11 Google Cloud Endpoints and Apigee Edge 12
13 Quiz 14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 Name 3 advantages of using the App
7 Engine flexible environment over App Engine standard. 8
9
10
11
12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 Name 3 advantages of using the App The flexible environment allows SSH
7 Engine flexible environment over App access, allows disk writes, and Engine standard. supports third-party binaries (also 8 allows stack customization and 9 background processes). 10
11
12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 Name 3 advantages of using the App The flexible environment allows SSH
7 Engine flexible environment over App access, allows disk writes, and Engine standard. supports third-party binaries (also 8 allows stack customization and 9 background processes). 10
11
12 What is the difference between Cloud Endpoints and Apigee Edge? 13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 Name 3 advantages of using the App The flexible environment allows SSH
7 Engine flexible environment over App access, allows disk writes, and Engine standard. supports third-party binaries (also 8 allows stack customization and 9 background processes). 10
11
12 What is the difference between Cloud Cloud Endpoints helps you create and Endpoints and Apigee Edge? maintain APIs; Apigee Edge helps you 13 secure and monetize APIs. 14
15
16
17
18 1 Cloud OnBoard
2 3 More resources 5
6 Google App Engine Google Cloud Endpoints 7 https://cloud.google.com/appengine/docs/ https://cloud.google.com/endpoints/docs/ 8
9 Google App Engine Flexible Environment Apigee Edge
10 https://cloud.google.com/appengine/ http://docs.apigee.com/api-services/content/
11 docs/flexible/ what-apigee-edge
12 Google App Engine Standard Environment 13 https://cloud.google.com/appengine/ 14 docs/standard/ 15
16
17
18 1
2
3
5
6
7 8 #GoogleCloudOnBoard 9
10
11
12
13
14
15
16 1
2
3
5
6 7 Module #7: 8 Developing, Deploying, and Monitoring in the Cloud 9
10
11
12
13
14
15
16 1 Cloud OnBoard
2 3 Agenda 5 6 Development in the cloud 7
8 Deployment: Infrastructure as code
9
10 Monitoring: Proactive instrumentation
11
12
13
14
15
16
17
18 Cloud OnBoard
Cloud Source Repositories
● Fully featured Git repositories hosted on Google Cloud Platform
● Supports collaborative development of cloud apps
● Includes integration with Stackdriver Debugger Cloud OnBoard
Cloud Functions
● Create single-purpose functions that respond to events without a server or runtime ○
● Written in Javascript; execute in managed Node.js environment on Google Cloud Platform 1 Cloud OnBoard
2 3 Agenda 5 6 Development in the cloud 7
8 Deployment: Infrastructure as code
9
10 Monitoring: Proactive instrumentation
11
12
13
14
15
16
17
18 Cloud OnBoard
Deployment Manager
● Infrastructure management service
● Create a .yaml template describing your environment and use Deployment Manager to create resources
● Provides repeatable deployments 1 Cloud OnBoard
2 3 Agenda 5 6 Development in the cloud 7
8 Deployment: Infrastructure as code
9
10 Monitoring: Proactive instrumentation
11
12
13
14
15
16
17
18 Monitoring Logging Debug
Error Reporting Trace 1 Cloud OnBoard
2
3 Stackdriver offers capabilities in six areas
5
6 Monitoring Logging Profiler 7 Platform, system, and Platform, system, and Statistical, low-overhead profiler 8 application metrics application logs that continuously gathers CPU 9 usage and memory-allocation Uptime/health checks Log search, view, filter, 10 information from your and export production applications 11 Dashboards and alerts Log-based metrics 12
13
14 Error Reporting Debugger Trace
15 Error notifications Debug applications Latency reporting and sampling 16 Error dashboard 17 Per-URL latency and statistics
18 1 Cloud OnBoard
2 3 More resources 5
6 Cloud Source Repositories 7 https://cloud.google.com/source-repositories/docs/
8
9 Deployment Manager
10 https://cloud.google.com/deployment-manager/docs/
11
12 Google Stackdriver https://cloud.google.com/stackdriver/docs/ 13
14
15
16
17
18 1
2
3
5
6
7 8 #GoogleCloudOnBoard 9
10
11
12
13
14
15
16 1
2
3
5
6 7 Module #8: 8 Big Data and Machine Learning in the Cloud 9
10
11
12
13
14
15
16 1 Cloud OnBoard
2 3 Agenda 5 6 Google Cloud Big Data Platform 7
8 Google Cloud Machine Learning Platform
9
10 Quiz
11
12
13
14
15
16
17
18 Cloud OnBoard
Google Cloud’s big data services are fully managed and scalable
I Cloud OnBoard
Cloud Dataproc is managed Hadoop
● Fast, easy, managed way to run Hadoop and Spark/Hive/Pig on Google Cloud Platform
● Create clusters in 90 seconds or less on average.
● Scale clusters up and down even when jobs are running. Cloud OnBoard
Why use Cloud Dataproc?
● Easily migrate on-premises Hadoop jobs to the cloud.
● Quickly analyze data (like log data) stored in Cloud Storage; create a cluster in 90 seconds or less on average, and then delete it immediately.
● Use Spark/Spark SQL to quickly perform data mining and analysis.
● Use Spark Machine Learning Libraries (MLlib) to run classification algorithms. Cloud OnBoard
Cloud Dataflow offers managed data pipelines
● Processes data using Compute Engine instances. ○ ○
● Write code once and get batch and streaming.
● Transform-based programming model. Cloud OnBoard
Dataflow pipelines flow data from a source through transforms
Source
BigQuery
Transforms
Sink
Cloud Storage Cloud OnBoard
Why use Cloud Dataflow?
● ETL (extract/transform/load) pipelines to move, filter, enrich, shape data
● Data analysis: batch computation or continuous computation using streaming
● Orchestration: create pipelines that coordinate services, including external services
● Integrates with GCP services like Cloud Storage, Cloud Pub/Sub, BigQuery, and Bigtable ○ Cloud OnBoard
BigQuery is a fully managed data warehouse
● Provides near real-time interactive analysis of massive datasets (hundreds of TBs)
● Query using SQL syntax (SQL 2011)
● No cluster maintenance is required. Cloud OnBoard
BigQuery runs on Google’s high-performance infrastructure
● Compute and storage are separated with a terabit network in between
● You only pay for storage and processing used
● Automatic discount for long-term data storage BigQuery at UC Denver Anschutz Medical Personalized medicinal care for each patient Campus
6 million Genetic makeup, health history and treatments of over 6 million patients
8 hrs 15 min BigQuery reduced time to execute complex analyses on the entire set of patient records to just 15 minutes Cloud OnBoard
Cloud Pub/Sub is scalable, reliable messaging
● Supports many-to-many asynchronous messaging ○
● Includes support for offline consumers
● Based on proven Google technologies
● Integrates with Cloud Dataflow for data processing pipelines Cloud OnBoard
Why use Cloud Pub/Sub?
● Building block for data ingestion in Dataflow, Internet of Things (IoT), Marketing Analytics
● Foundation for Dataflow streaming
● Push notifications for cloud-based applications
● Connect applications across Google Cloud Platform (push/pull between Compute Engine and App Engine) Cloud OnBoard
Cloud Datalab offers interactive data exploration
● Interactive tool for large-scale data exploration, transformation, analysis, and visualization
● Integrated, open source ○ ○ Cloud OnBoard
Why use Cloud Datalab?
● Create and manage code, documentation, results, and visualizations in intuitive notebook format. ○
● Analyze data in BigQuery, Compute Engine, and Cloud Storage using Python, SQL, and JavaScript.
● Easily deploy models to BigQuery. 1 Cloud OnBoard
2 3 Agenda 5 6 Google Cloud Big Data Platform 7
8 Google Cloud Machine Learning Platform
9
10 Quiz
11
12
13
14
15
16
17
18 Cloud OnBoard
Machine Learning APIs enable apps that see, hear, and understand Cloud OnBoard
Cloud Machine Learning Platform
● Open source tool to build and run neural network models ○
● Fully managed machine learning service ○ ○ Cloud ML ● Pre-trained machine learning models built by Google ○
○
Machine Learning APIs ○ ○ 1 Cloud OnBoard
2
3 Why use the Cloud Machine Learning platform?
5 6 For structured data For unstructured data 7
8 Image and video analytics 9 Classification and regression
10
11 Text analytics 12 Recommendation
13
14
15 Anomaly detection 16
17
18 Cloud OnBoard
Cloud Vision API
● Analyze images with a simple REST API ○
● With the Cloud Vision API, you can: ○ ○ ○ ○ Cloud OnBoard
Cloud Speech API
● Recognizes over 80 languages and variants
● Can return text in real time
● Highly accurate, even in noisy environments
● Access from any device
● Powered by Google’s machine learning Cloud OnBoard
Cloud Natural Language API
● Uses machine learning models to reveal structure and meaning of text.
● Extract information about items mentioned in text documents, news articles, and blog posts.
● Analyze text uploaded in request or integrate with Cloud Storage. Cloud OnBoard
Cloud Translation API
● Translate arbitrary strings between thousands of language pairs
● Programmatically detect a document’s language
● Support for dozens of languages Cloud OnBoard
Cloud Video Intelligence APIBeta
● Annotate the contents of videos
● Detect scene changes
● Flag inappropriate content
● Support for a variety of video formats 1 Cloud OnBoard
2 3 Agenda 5 6 Google Cloud Big Data Platform 7
8 Google Cloud Machine Learning Platform
9
10 Quiz
11
12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 When would you use Cloud Dataproc?
7
8
9
10
11
12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 When would you use Cloud Dataproc? You can use it to migrate on-premises
7 Hadoop jobs to the cloud. You can also use it for data mining and 8 analysis of cloud-based data. 9
10
11
12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 When would you use Cloud Dataproc? You can use it to migrate on-premises
7 Hadoop jobs to the cloud. You can also use it for data mining and 8 analysis of cloud-based data. 9
10
11 Name two use cases for Cloud Dataflow.
12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 When would you use Cloud Dataproc? You can use it to migrate on-premises
7 Hadoop jobs to the cloud. You can also use it for data mining and 8 analysis of cloud-based data. 9
10
11 Name two use cases for Cloud Dataflow. ETL, orchestration
12
13
14
15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 When would you use Cloud Dataproc? You can use it to migrate on-premises
7 Hadoop jobs to the cloud. You can also use it for data mining and 8 analysis of cloud-based data. 9
10
11 Name two use cases for Cloud Dataflow. ETL, orchestration
12
13
14 Name three use cases for the Google machine learning platform. 15
16
17
18 1 Cloud OnBoard
2 3 Quiz 5
6 When would you use Cloud Dataproc? You can use it to migrate on-premises
7 Hadoop jobs to the cloud. You can also use it for data mining and 8 analysis of cloud-based data. 9
10
11 Name two use cases for Cloud Dataflow. ETL, orchestration
12
13
14 Name three use cases for the Google Fraud detection, sentiment analysis, machine learning platform. content personalization 15
16
17
18 1 Cloud OnBoard
2 3 More resources 5
6 Google Big Data Platform 7 https://cloud.google.com/products/big-data/
8
9 Google Machine Learning Platform
10 https://cloud.google.com/products/machine-learning/
11
12
13
14
15
16
17
18 1
2
3
5
6
7 8 #GoogleCloudOnBoard 9
10
11
12
13
14
15
16 1
2
3
5
6
7 8 Closing Session 9
10
11
12
13
14
15
16 Cloud OnBoard
1 month free access to Qwiklabs
1 Receive a follow up email after this event
2 Follow the link to Qwiklabs and enroll in a Quest
3 Create your Qwiklabs account if you don’t have one already
4 Log in and take your first lab Cloud OnBoard
Make Google Cloud certification your goal!
Find study guides, tips, practice Associate Cloud Engineer exams, and testing sites Deploys applications, monitors operations, and maintains projects
Professional Cloud Architect Designs, builds and manages solutions
Professional Data Engineer Develops data processing systems and creates machine learning models cloud.google.com/certification 1 Cloud OnBoard
2 3 Google Cloud Developer Communities 5
6 Start or join a community near you
7 Connect with developers, DevOps, 8 IT pros and architects, through in 9 person meetups.
10
11 Train, share, and learn about the 12 latest features and newest updates 13 on Google Cloud.
14
15 [email protected] 16 cloud.google.com/community/meetups 50 chapters globally and growing
17
18