Getting Started with Google Cloud

Home , Google APIs, Google Cloud Datastore

Getting Started With Google Cloud

Cloud OnBoard

Welcome to Cloud OnBoard

#GoogleCloudOnBoard

©Google Inc. or its affiliates. All rights reserved. Do not distribute. ©Google Inc. or its affiliates. All rights reserved. Do not distribute. May only be taught by Google Cloud Platform Authorized Trainers.

Cloud OnBoard

Page 1 Agenda Why choose Google Cloud Platform?

Google Cloud Platform enables developers to build, test and 1 Introduction to Google Cloud Platform deploy applications on Google’s highly-scalable, secure, and reliable infrastructure. 2 Quiz Choose from computing, storage, big data/machine learning, and application services for your web, mobile, analytics, and backend solutions.

The Future of Cloud Computing GCP is organized into regions and zones

Now Next ● Regions: collections of zones ○ Specific geographical locations where you can run resources ○ Regions are interconnected using Google’s global, meshed backbone network

Storage Processing Memory Network Storage Processing Memory Network ● Zones: isolated deployment areas in a region

Physical/Colo Virtualized Serverless/No-Ops ● Your resources can be regional, zonal, or in some cases multi-regional User-configured, managed, and maintained Fully automated

Page 2 Page 3 Google’s infrastructure: the most powerful on earth Google’s commitment to environmental responsibility

Belgium 2 Finland London 3 3 Tokyo (2016) Developing our infrastructure while respecting Oregon 2 3 Iowa 4 Frankfurt 3 N Virginia 3 3 our ecosystem Taiwan S Carolina 3 3 Mumbai • Pioneering data center efficiency 2 Singapore • Largest private investor in renewables (wind, solar) Network path 3 São Paulo • First data centers to receive ISO 14001 certification Sydney 3 Current regions # and number of zones • 100% carbon neutral since 2007 Committed regions for 2017 # and number of zones

https://peering.google.com https://cloud.google.com/compute/docs/regions-zones/regions-zones Image by Connie Zhou

Google’s network: Innovative, Customer-Friendly Pricing the world’s largest software-defined network

Belgium 2 Finland ● Billing in sub-hour increments London 3 3 Tokyo (2016) Oregon 2 3 Iowa 4 Frankfurt 3 N Virginia 3 3 ● Discounts for sustained-use Taiwan S Carolina 3 3 Mumbai ○ Automatically reward users who run virtual machines for 2 Singapore over 25% of any calendar month

Network path 3 São Paulo Points of presence ● Custom virtual-machine types Sydney 3 Current regions # and number of zones ○ Pay only for the resources you need for your application Committed regions for 2017 # and number of zones

https://peering.google.com https://cloud.google.com/compute/docs/regions-zones/regions-zones Image by Connie Zhou

Page 4 Page 5 Google Cloud Platform Google’s commitment to open APIs and open source Compute Storage

Cloud Cloud Compute Container App Engine Cloud Bigtable Cloud Cloud SQL Spanner Datastore Engine Engine Functions Storage Beta Beta Big Data Machine Learning

Customers should use us because they love us, BigQuery Pub/Sub Dataflow Dataproc Datalab Natural Vision API Machine Speech Translate API not because they are unable to migrate Language API Learning API

GCP offers a range of compute architectures

Stackdriver helps engineers keep their cloud-powered applications fast and available.

Compute Container App Cloud Managed Engine Engine Engine Functions Beta services Monitoring Logging Debug IaaS Hybrid PaaS Serverless Automated logic elastic resources Error Reporting Trace

Toward managed infrastructure Toward dynamic infrastructure

Page 6 Page 7 Agenda Resources

● Why Google Cloud Platform? https://cloud.google.com/why-google/ 1 Introduction to Google Cloud Platform ● Pricing philosophy https://cloud.google.com/pricing/philosophy/ 2 Quiz ● Data Centers https://www.google.com/about/datacenters/ ● Google Cloud Platform product overview http://cloud.google.com/products/ ● Google Cloud Platform solutions http://cloud.google.com/solutions/

Quiz Cloud OnBoard

Name 3 pricing innovations that make Google Cloud Platform the most cost-effective public cloud.

Sub-hour billing, sustained-use discounts, Compute Engine custom machine types

In addition to innovative pricing, name 3 benefits of using Google Cloud Platform.

Commitment to environmental responsibility, commitment to open source technologies, robust infrastructure

©Google Inc. or its affiliates. All rights reserved. Do not distribute. ©Google Inc. or its affiliates. All rights reserved. Do not distribute. 16 ©Google Inc. or its affiliates. All rights reserved. Do not distribute. May only be taught by Google Cloud Platform Authorized Trainers.

Page 8 Page 9 Agenda Projects are uniquely named

● Projects use three identifying attributes: 1 Google Cloud Platform Projects ○ Project Name ○ Project Number 2 Identity and Access Management (IAM) ○ Project ID ■ Also known as Application ID 3 Interacting with Google Cloud Platform ● Interact with projects using the Cloud Console 4 Quiz or the Cloud Resource Manager API

Projects organize resources Agenda

● All Google Cloud Platform services are associated with a project that is 1 Google Cloud Platform Projects used to: 2 ○ Track resource and quota usage Identity and Access Management (IAM) ○ Enable billing ○ Manage permissions and credentials 3 Interacting with Google Cloud Platform ○ Enable services and APIs 4 Quiz & Lab

Page 10 Page 11 Identity and Access Management You can define “curated roles”

List of Permissions Google Group ✔ compute.instances.delete ✔ compute.instances.get ✔ InstanceAdmin compute.instances.list ✔ Role compute.instances.setMachineType Who can do what on which resource ✔ compute.instances.start ✔ compute.instances.stop project_a . . .

IAM predefines “primitive roles” IAM Resource Hierarchy

● A policy is set on a resource example.com ○ Each policy Organization contains: Set of roles, role members

Owner Editor Viewer Billing administrator ● Resources inherit Project bookshelf static-assets stream-ingest policies from parent Invite members Deploy applications Read-only Manage billing Policy Inheritance Remove members Modify code ○ Resource policies access Add administrators are a union of Can delete project Configure services Remove parent and resource Compute App Cloud Cloud Cloud BigQuery Includes Editor rights Includes Viewer rights administrators Engine Engine Storage Storage Pub/Sub ● If parent policy less

restrictive, overrides Resources more restrictive A project can have multiple owners, editors, viewers and billing administrators. resource policy instance_a queue_a bucket_a bucket_b topic_a dataset_a

Page 12 Page 13 Organization node organizes projects Service Accounts and IAM

● Organization node is root node Visibility ● Service accounts for Google Cloud resources Control authenticate with keys Identity IAM Role Resource ○ Can be managed via Cloud ○ Google manages keys, key Resource Manager API rotation for Compute Engine [email protected] example.com and App Engine Service Account InstanceAdmin Role Compute Instances Organization ● Can assign an IAM role to the ● 2 organization roles: Admin ○ Organization Admin - Control service account Create over all cloud resources ● Can also assign ○ Project Creator - Controls ServiceAccountActor role project_1 project_2 project creation [email protected] User/Group ServiceAccountActor Role Service Account Project Creator to users/groups

Service Accounts Example: Service Accounts and IAM

project_a project_b ● Provide an identity for carrying out server-to-server ● VMs running component_1 are granted Editor access to interactions in a project project_b using Service Account 1 component_1 Service Account 1 ● Used to authenticate from one service to another ● VMs running component_2 are Editor objectViewer ● Can be used with primitive and curated roles granted access to bucket_1 using Service Account 2 ● Identified with an email address: ● Service account permissions can component_2 Service Account 2 [email protected] be changed without recreating Storage. objectViewer [email protected] VMs bucket_1

Page 14 Page 15 Agenda Google Cloud Platform Console

● Centralized console for all project data 1 Google Cloud Platform Projects ● Developer tools Cloud Source Repositories 2 Identity and Access Management (IAM) ○ ○ Cloud Shell ○ Test Lab (mobile app testing) 3 Interacting with Google Cloud Platform ● Access to product APIs 4 Quiz ● Manage, create projects

Interacting with Google Cloud Platform Google Cloud SDK

Cloud SDK/ ● SDK includes CLI tools for Cloud Cloud Console >_ Web user interface Cloud Shell Platform products and services command-line interfaces ○ gcloud, gsutil (Cloud Storage), bq (BigQuery) ● Available as Docker image ● Available via Cloud Shell Cloud Console REST-based API Mobile App ○ Containerized version of Cloud SDK running For Android and iOS on Compute Engine instance

Page 16 Page 17 RESTful APIs APIs Explorer

● Programmatic access to products and services ● The APIs Explorer is an interactive tool that lets you easily ○ Typically use JSON as an interchange format try Google APIs using a browser ○ Use OAuth 2.0 for authentication and authorization ● With the APIs Explorer, you can: ● Enabled through the Google Cloud Platform Console ○ Browse quickly through available APIs and versions. ● Most APIs include daily quotas and rates (limits) that can be raised by ○ See methods available for each API and what parameters request they support along with inline documentation. ○ Important to plan ahead to manage your required capacity ○ Execute requests for any method and see responses in real ● Experiment with APIs Explorer time. ○ Make authenticated and authorized API calls with ease.

Cloud Console Mobile App Client Libraries

● Manage virtual machines and ● Google Cloud Client Libraries database instances ○ Community-owned, hand-crafted client libraries ● Manage apps in Google App Engine ● Manage your billing ● Google APIs Client Libraries ● Visualize your projects with a ○ Open source, generated customizable dashboard ○ Support various languages ■ Java, Python, JavaScript, PHP, .NET, Go, Node.js, Ruby, Objective-C, Dart

Page 18 Page 19 Agenda Quiz: Service Accounts

Service accounts are used to provide which of the following? 1 Google Cloud Platform Projects

❏ Authentication between Google Cloud Platform services 2 Identity and Access Management (IAM) ❏ Key generation and rotation when used with App Engine and Compute Engine 3 Interacting with Google Cloud Platform ❏ A way to restrict the actions a resource (such as a VM) can perform ❏ A way to allow users to act with service account permissions ❏ All of the above 4 Quiz

Quiz: Policies and Projects Resources

Cloud SDK installation and quick start True or False: If a Google Cloud IAM policy gives you Owner permissions at ● https://cloud.google.com/sdk/#Quick_Start the project level, your access to a resource in the project may be restricted by a more restrictive policy on that resource. ● gcloud tool guide https://cloud.google.com/sdk/gcloud/ False: Policies are a union of the parent and the resource. If a parent policy is less restrictive, it overrides a more restrictive resource policy. ● IAM https://cloud.google.com/iam/ True or False: All Google Cloud Platform resources are associated with a project. ● Configuring permissions on Google Cloud Platform https://cloud.google.com/docs/permissions-overview True: All Google Cloud Platform resources are associated with a project. ● Google Cloud Platform security https://cloud.google.com/security/

Page 20 Page 21 Cloud OnBoard Google Cloud Platform

Machine Operations Compute Networking Big Data Storage Learning and Tools

Compute Container App Cloud Engine Engine Engine Functions Beta

©Google Inc. or its affiliates. All rights reserved. Do not distribute. ©Google Inc. or its affiliates. All rights reserved. Do not distribute. ©Google Inc. or its affiliates. All rights reserved. Do not distribute. 45 May only be taught by Google Cloud Platform Authorized Trainers.

Agenda What is Google App Engine? 1 Overview and Customer Stories ● A platform (platform as a service) for 2 Google App Engine Standard Environment building scalable web applications and 3 Google App Engine Flexible Environment mobile backends ● App Engine makes deployment, 4 Google Cloud Endpoints maintenance, and scalability easy so 5 Google Cloud Datastore you can focus on innovation

6 Quiz

Page 22 Page 23 IaaS and PaaS Agenda 1 Overview and Customer Stories

Compute Engine App Engine 2 Google App Engine Standard Environment Towards Towards managed managed infrastructure services 3 Google App Engine Flexible Environment IaaS PaaS Raw compute, storage and Preset run-times network Java, Go, PHP, Python... 4 Google Cloud Endpoints More granular control Focus is application logic 5 Google Cloud Datastore Pay for what you use Pay for what you allocate Less management overhead More management overhead 6 Quiz

App Engine Standard Environment “App Engine enabled us to focus on developing the application. We wouldn’t have gotten here without the Snapchat ease of development that App Engine gave us.” ● Easily deploy your applications Bobby Murphy, CTO ● Autoscale workloads to meet demand Economical Snapchat sends ● ○ Free daily quota 700 million ○ Usage based pricing photos and videos each day Google App Engine Small team is able ● SDKs for development, testing and scaled seamlessly to innovate quickly during growth to and expand deployment millions of users globally

Page 24 Page 25 App Engine Standard Environment: requirements Agenda 1 Overview and Customer Stories ● Specific versions of Java, Python, PHP, and Go are 2 Google App Engine Standard Environment supported 3 Google App Engine Flexible Environment

● Need to conform to sandbox constraints: 4 Google Cloud Endpoints ○ No writing to local file system ○ Request timeouts at 60 seconds 5 Google Cloud Datastore ○ Limit on 3rd-party software installations 6 Quiz

Example App Engine Standard Workflow - App Engine Flexible Environment Web Applications App Engine automatically App Engine can access a ● Build, deploy containerized apps with a 3 scales & reliably serves your variety of services using 1 Develop & test the web web application dedicated APIs click application locally ● Standard runtimes - Python, Java, Go, Project Memcache Node.js App Engine Task App Servers queues ○ No sandbox constraints Application Use the SDK to deploy to Scheduled ○ Can access App Engine resources 2 instances App Engine tasks Application ● Custom runtime support instances Search Any language that supports HTTP Application ○ instances Logs requests

Page 26 Page 27 App Engine Standard vs Flexible Environment Google Cloud Endpoints: distributed API management Standard Environment Flexible Environment An API console to help you create and Instance startup Milliseconds Minutes ● maintain APIs SSH access No Yes (not default) ● Expose your API using a RESTful interface Scaling Manual, basic, automatic Manual, automatic

Write to local disk No Yes (ephemeral) ● Control access and validate calls with

Support for 3rd party binaries No Yes JSON Web Tokens and Google API keys

Network access Via App Engine services Yes ○ Identify web, mobile users with Auth0 and Firebase Authentication Customizable stack No Yes

After free daily use, pay per instance Pay for resource allocation per hour; Generate client libraries Pricing model ● class, with automatic shutdown no automatic shutdown

Agenda Google Cloud Endpoints: supported platforms

1 Overview and Customer Stories ● Supports App Engine Standard or Flexible Environment, Compute Engine, 2 Google App Engine Standard Environment Container Engine 3 Google App Engine Flexible Environment ● Use Java or Python open source Frameworks or any other framework 4 Google Cloud Endpoints and language

5 Google Cloud Datastore ● Supports iOS, Android, and JavaScript clients 6 Quiz

Page 28 Page 29 Agenda Google Cloud Datastore: scalable NoSQL DB 1 Overview and Customer Stories ● Database designed for application 2 Google App Engine Standard Environment backends ● NoSQL store with automatic scaling 3 Google App Engine Flexible Environment to billions of rows 4 Google Cloud Endpoints ● Fully managed ● Built-in redundancy 5 Google Cloud Datastore ● Supports ACID transactions 6 Quiz

Google Cloud Platform Google Cloud Datastore: benefits

● Schemaless access No need to think about underlying data Machine Operations ○ Compute Networking Big Data Storage Learning and Tools structure ● Local development tools ● Includes a free daily quota Cloud Cloud Cloud Cloud Access from anywhere through a Storage SQL Datastore Bigtable ● RESTful interface

Page 30 Page 31 Agenda Resources 1 Overview and Customer Stories ● Overview: App Engine https://cloud.google.com/appengine/ 2 Google App Engine Standard Environment ● DevBytes - Your app, at scale with Google App Engine https://www.youtube.com/watch?v=ytT2-kL9v2o 3 Google App Engine Flexible Environment ● Datastore Concepts Overview 4 Google Cloud Endpoints https://cloud.google.com/datastore/docs/concepts/overview ● Getting started with Google Cloud Datastore API 5 Google Cloud Datastore https://cloud.google.com/datastore/docs/datastore-api-tutorial 6 Quiz

Quiz Cloud OnBoard

Name 3 advantages of using the App Engine Flexible Environment over App Engine Standard. The Flexible Environment allows SSH access, allows disk writes, and supports third-party binaries. It allows stack customization and background processes. True or False: Google Cloud Datastore supports ACID transactions. True. Google Cloud Datastore supports ACID transactions.

©Google Inc. or its affiliates. All rights reserved. Do not distribute. ©Google Inc. or its affiliates. All rights reserved. Do not distribute. 64 ©Google Inc. or its affiliates. All rights reserved. Do not distribute. May only be taught by Google Cloud Platform Authorized Trainers.

Page 32 Page 33 Agenda Google Cloud Storage

1 Google Cloud Storage ● High performance, internet-scale, immutable BLOB (binary large object) 2 Google Cloud Bigtable storage ● Simple administration 3 Google Cloud SQL and Google Cloud Spanner ○ Does not require capacity management 4 Comparing Storage Options ● Data encryption in-flight and at rest ● Four storage classes give customers 5 Quiz flexibility

Google Cloud Platform Choosing among Cloud Storage classes

Class Multi-regional Regional Nearline Coldline

Intended for Most frequently Accessed frequently Accessed less than Accessed less than data that is... accessed within a region once a month once a year Machine Operations Compute Networking Big Data Storage Learning and Tools Availability SLA 99.95% 99.90% 99.00% 99.00%

Access APIs Consistent APIs

Access time Millisecond access

Storage price Price per GB stored per month

Retrieval price Cloud Cloud Cloud Cloud Cloud Total price per GB transferred Storage SQL Spanner Datastore Bigtable Use cases Content storage and In-region data analytics, Long-tail content, Archiving, delivery transcoding backups disaster recovery

Page 34 Page 35 Cloud Storage Integration

“Thanks to Google Cloud Platform and the Google Genomics team, the greatest minds in Import and Startup science from around the world will be able to scripts, study trillions of data points in one single export database.” tables images and general object 100 200 PER RAW BigQuery Compute storage UPLOADED TERABYTES UP TO GIGABYTES GENOME Engine of data from more than MSSNG project 1,300 WHOLE could easily surpass a Object

GENOMES PETABYTE OF DATA to Google Cloud Storage storage, logs, Cloud Storage Import and Datastore export WHOLE GENOMES FROM Making MSSNG world’s backups tables largest single repository of autism-related DNA sequencing data 10,000 PEOPLE App Cloud SQL Engine

Agenda Cloud Storage 1 Google Cloud Storage Features 2 Google Cloud Bigtable Offline import (third Regional buckets Object versioning party) 3 Google Cloud SQL and Google Cloud Spanner

Online cloud import Object lifecycle Object change ACLs (Cloud Storage 4 Comparing Storage Options management notification Transfer Service) 5 Quiz

Page 36 Page 37 Google Cloud Platform Google Cloud Bigtable: protected, proven

● Replicated storage Machine Operations Compute Networking Big Data Storage Learning and Tools ● Data encryption in-flight and at rest ● Role-based ACLs ● Drives major applications such as Google Analytics and Gmail Cloud Cloud Cloud Cloud Cloud Storage SQL Spanner Datastore Bigtable

Google Cloud Bigtable: managed NoSQL

Overview: Challenges: Data to process: Data in the Consolidated Audit Trail (CAT). How to process the CAT and organize 100 billion market ● Fully managed, NoSQL, wide-column A data repository of all equities and options orders, quotes, events into an “order lifecycle” in a 4 hour window and events Store 6 years (~30PB) of data database service for large-workload

applications - Terabytes to petabytes Cloud Bigtable to process and run queries ● Integrated and tolerate volume increases ○ Accessed using HBase API ○ Native compatibility with big data, 6 BILLION 1.7 GIGs 10 BN 1.7 GIGABYTES PER SECOND WRITTEN PER SECOND Hadoop ecosystems MARKET EVENTS PER HOUR BURSTS 10 TERABYTES WRITTEN PER HOUR 6 TBs PER HOUR PER HOUR

Page 38 Page 39 Bigtable Access Patterns Agenda

Application API 1 Google Cloud Storage Data can be read from and written to Cloud Bigtable through a data service layer like: Managed VMs, the HBase REST Server, a Java Server using the HBase client. Typically this will be to serve data to applications, dashboards and data services. 2 Google Cloud Bigtable Streaming Data can be streamed in (written event by event) through a variety of popular stream processing frameworks like: 3 Google Cloud SQL and Google Cloud Spanner Cloud Dataflow Streaming, Spark Streaming, and Storm.

Batch Processing Cloud Bigtable 4 Comparing Storage Options Data can be read from and written to Cloud Bigtable through batch processes like: Hadoop MapReduce, Dataflow, Spark. Often, summarized or newly calculated data is written back to Cloud Bigtable or to a downstream database. 5 Quiz

Cloud Bigtable Integration Google Cloud Platform

Google Cloud Dataflow Use Cloud Dataflow connector for Bigtable for batch and streaming operations in pipelines Machine Operations Compute Networking Big Data Storage Learning and Tools Google Cloud Dataproc Use Bigtable HBase client to integrate Hadoop jobs with Cloud Dataproc

Cloud Bigtable On-premises, cloud-based Hadoop Cloud Cloud Cloud Cloud Cloud External Storage SQL Spanner Datastore Bigtable service Use Bigtable HBase client to integrate with Hadoop clusters

Page 40 Page 41 Google Cloud SQL: managed RDBMS Google Cloud Platform

● Offers MySQL and PostgreSQLBeta

Machine Operations databases as a service Compute Networking Big Data Storage Learning and Tools ● Automatic replication ● Managed backups ● Vertical scaling (read and write) ● Horizontal scaling (read) Cloud Cloud Cloud Cloud Cloud ● Google security Storage SQL Spanner Datastore Bigtable

Cloud SQL Integration Google Cloud SpannerBeta

● Cloud Spanner is a External service horizontally-scalable and strongly consistent relational database. Cloud SQL can be used with Compute Engine instances Cloud SQL can be used with App Engine using standard can be authorized to external applications and ● Cloud Spanner supports: drivers. access Cloud SQL clients. ○ Automatic replication instances using an external Standard tools can be used to ○ Strong global consistency You can configure a Cloud SQL IP address. administer databases. instance to follow an App ○ Managed instances with high Cloud SQL instances can Engine application. be configured with a External read replicas can be availability preferred zone. configured. ○ SQL (ANSI 2011 with extensions)

Page 42 Page 43 Agenda Comparing Storage Options: use cases

1 Google Cloud Storage BLOB NoSQL SQL

2 Google Cloud Bigtable

Cloud Cloud Cloud Cloud SQL Cloud 3 Google Cloud SQL and Google Cloud Spanner Storage Datastore Bigtable Spanner Good for: Good for: Good for: Good for: Good for: Structured and Getting started, App “Flat” data, Heavy Web frameworks, Large-scale database unstructured binary Engine applications read/write, events, existing applications applications (> ~2 TB) 4 Comparing Storage Options or object data analytical data

Use cases: Use cases: Use cases: Use cases: Use cases: Images, large media User profiles, AdTech, Financial and User credentials, Whenever high I/O, 5 Quiz files, backups product catalog IoT data customer orders global consistency is needed

Comparing Storage Options: Technical details Agenda

Cloud Cloud Cloud SQL Cloud Spanner Bigtable Datastore Storage (1st and 2nd Generation) 1 Google Cloud Storage

Storage type NoSQL, Object Relational SQL Relational SQL NoSQL, wide-column document (BLOB) 2 store Google Cloud SQL and Google Cloud Spanner

Overall Terabytes + Petabytes + up to 500 GB Petabytes Petabytes + capacity 3 Google Cloud SQL Unit size 1 megabyte 5 TB / Standard MySQL 10,240 MiB / row Recommended: ~10 / entity object limits MB per cell, ~100 MB for all values per row 4 Comparing Storage Options Transactions Yes No Yes Yes No Complex No No Yes Yes No 5 Quiz queries 88

Page 44 Page 45 Quiz Cloud OnBoard

You are developing an application that transcodes large video files. Which storage option is the best choice for your application?

Google Cloud Storage You manufacture devices with sensors and need to stream huge amounts of data from these devices to a storage option in the cloud. Which Google Cloud Platform storage option is the best choice for your application?

Google Cloud Bigtable

©Google Inc. or its affiliates. All rights reserved. Do not distribute. ©Google Inc. or its affiliates. All rights reserved. Do not distribute. 91 ©Google Inc. or its affiliates. All rights reserved. Do not distribute. May only be taught by Google Cloud Platform Authorized Trainers.

Resources Agenda

● Overview: Cloud Storage https://cloud.google.com/storage/ 1 Introduction to Containers ● DevBytes - File storage in the cloud https://www.youtube.com/watch?v=vyIap827rHs 2 Kubernetes ● Cloud SQL: Features, case studies, pricing, & documentation https://cloud.google.com/sql/ 3 Google Container Engine ● Getting started with Google Cloud SQL https://cloud.google.com/sql/docs/quickstart ● Overview of Cloud Bigtable 4 Quiz https://cloud.google.com/bigtable/docs/overview

Page 46 Page 47 What is a Container? Agenda

● Virtualization at the operating Virtual Machine system layer 1 Introduction to Containers ● Separates operating system from application code and 2 Kubernetes

dependencies Container Container Container Code and libraries Code and libraries Code and libraries 3 Google Container Engine ● Isolates individual processes Shared libraries ● Popular implementations 4 Quiz include Docker and rkt Operating system

Why Use Containers? Kubernetes (“k8s”)

● Open source container cluster ● Support consistency across development, testing, and orchestration system production environments ○ Automates deployment, scaling, and ● Loose coupling between application and operating system operations for container clusters layers ● Based on Google’s experience over ● Much simpler to migrate workloads between on-premises 10+ years and cloud environments ● Built for a multi-cloud world: ● Support agile development and operations ○ Public, private, hybrid

Page 48 Page 49 Kubernetes eases application maintenance Agenda

● Workload portability ○ Run in many environments, across cloud 1 Introduction to Containers providers ○ Implementation is open and modular 2 Kubernetes ● Rolling updates ○ Upgrade application with zero downtime 3 Google Container Engine ● Persistent storage 4 Quiz ○ Abstracts details of how storage is provided from how it is consumed

Kubernetes makes applications more elastic Google Cloud Platform

● Multi-zone clusters ○ Run a single cluster in multiple zones Machine Operations Compute Networking Big Data Storage ● Load balancing Learning and Tools ○ External IP address routes traffic to correct port ● Autoscaling Compute Container App Cloud Engine Engine Engine Functions ○ Automatically adapt to changes in workload Beta

Page 50 Page 51 IaaS and PaaS Why use Google Container Engine?

● Decouples operational, development concerns Compute Engine Container Engine App Engine Towards Towards managed managed ● Manages and maintains infrastructure services IaaS PaaS ○ Logging, health management, monitoring Raw compute, storage and Preset run-times network Java, Go, PHP, Python... ● Easily update Kubernetes versions as More granular control Focus is application logic they are released

Pay for what you use Pay for what you allocate Less management overhead More management overhead

Google Container Engine manages GKE’s complementary services and runs containers Google Cloud Container Builder ● Fully managed cluster management and ● orchestration system for running containers ○ Create Docker container images from ○ Based on Kubernetes app code in Google Cloud Storage ○ Uses Compute Engine instances and resources ● Uses a declarative syntax to manage applications ● Google Container Registry ○ Declare desired application configuration, ○ Secure, private Docker image storage Container Engine implements, manage

Page 52 Page 53 Agenda “Our platform sometimes has to be deployed on a cluster. How do we enable containers to communicate from different hosts? Google has the answer: Treeptik Kubernetes. This awesome tool helps us manage our clusters of containers as if they were a single system.” 1 Introduction to Containers

2 Kubernetes

scale speed -30% 3 Google Container Engine

Docker containers automate scalability REST APIs speed provisioning of new Administrative costs reduced by 30% instances; JAVA applications can be 4 Quiz deployed in minutes

Deploying Apps: Container Engine vs App Engine Quiz

Name two reasons for deploying applications using containers. Container App Engine App Engine Engine Standard Flexible Consistency across development, testing, production environments; Simpler to migrate workloads; Loose coupling; Agility Language Any Java, Python, Go Any True or False: Kubernetes lets you manage container clusters in multiple cloud & PHP support providers. Service model Hybrid PaaS PaaS True. Primary use case Container-based Web and mobile Web and mobile True or False: GCP provides a secure, high-speed container image storage workloads applications applications, service for use with Container Engine. container-based workloads True.

Page 54 Page 55 Resources Agenda

● Container Engine Overview 1 Google Compute Engine Overview https://cloud.google.com/container-engine/

● Container Engine tutorials 2 Google Cloud Networking https://cloud.google.com/container-engine/docs/tutorials ● Kubernetes 3 Operations and Tools http://kubernetes.io/ 4 Comparing Compute Options

5 Quiz

Cloud OnBoard Google Cloud Platform

Machine Operations Compute Networking Big Data Storage Learning and Tools

Compute Container App Cloud Engine Engine Engine Functions Beta

©Google Inc. or its affiliates. All rights reserved. Do not distribute. ©Google Inc. or its affiliates. All rights reserved. Do not distribute. ©Google Inc. or its affiliates. All rights reserved. Do not distribute.114 May only be taught by Google Cloud Platform Authorized Trainers.

Page 56 Page 57 Google Compute Engine: managed VMs Google Compute Engine: innovative pricing

● Per-minute billing, sustained use ● High CPU, high memory, standard and discounts shared-core machine types ● Preemptible instances ● Persistent disks ● High throughput to storage at no extra ○ Standard, SSD, local SSD cost ○ Snapshots ● Custom machine types - Only pay for ● Robust networking features the hardware you need

Google Compute Engine: many VM options Compute Engine reduces render farm load during periods of peak production ● High CPU, high memory, standard and Consumes processing Faster rendering time shared-core machine types power of up to 15,000 means visual designers Intel cores at peak can get results and make ● Persistent disks rendering times tweaks more quickly

Standard, SSD, local SSD saved due to eliminating idle cores Framestore ○ $300,000+ ○ Snapshots during production “quiet times” ● Resize disks, migrate instances with “By adding Compute Engine to our workflow and allowing our in-house capacity to focus on the studio no downtime work, everyone’s project gets computing time – and the creative team can get as imaginative as they want to, ● Instance metadata and startup scripts with fast views of new iterations.”

Page 58 Page 59 Agenda Cloud Virtual Network

1 Google Compute Engine Overview ● Managed networking functionality for Cloud Platform resources 2 Google Cloud Networking ● Provision Google Cloud Platform 3 Operations and Tools resources, connect them to each other and isolate them from one 4 Comparing Compute Options another in a Virtual Private Cloud (VPC) 5 Quiz

Google Cloud Platform Cloud Virtual Network’s internetworking features

● Fine-grained networking policies

Machine Operations ● Granular IP address range selection Compute Networking Big Data Storage Learning and Tools ● Routes ● Firewalls ● Virtual Private Network (VPN) ● Cloud Router Cloud Cloud Cloud Load Cloud Virtual Interconnect DNS Balancing CDN Network

Page 60 Page 61 Google Cloud Interconnect Google Cloud Load Balancing: HTTP(s)

● Balance HTTP-based traffic across Carrier Interconnect Direct Peering CDN Interconnect multiple Compute Engine regions Enterprise-grade Connect your Allows select CDN providers to ● Global, external IP address routes connections provided business directly establish direct interconnect links with traffic by carrier service to Google Google’s edge network at various providers locations ● Scalable, requires no pre-warming and provides resilience, fault tolerance

Google Cloud Interconnect

Google Cloud DNS Google Cloud Load Balancing: TCP/SSL, UDP

● Highly available and scalable DNS ● Spread TCP/SSL and UDP traffic over ○ Translates domain names into IP pool of instances within a Compute addresses Engine region Create managed zones, then add, edit, ● ○ Ensures only healthy instances handle delete DNS records traffic ○ Programmatically manage zones and ● Scalable, requires no pre-warming records using RESTful API or command-line interface

Page 62 Page 63 Google Cloud CDN (Content Delivery Network) Google Cloud Platform

● Use Google's globally distributed edge caches to cache HTTP(S) load

balanced content far closer to your Machine Operations Compute Networking Big Data Storage and Tools users than your instances Learning ○ Faster delivery of content to users while reducing costs

● Cloud CDN uses caches at network Cloud Deployment Stackdriver Source Manager locations to store responses generated Repositories by instances

Agenda Google Stackdriver

1 Google Compute Engine Overview ● Integrated monitoring, logging, diagnostics 2 Google Cloud Networking ● Powerful data, analytics tools

3 Operations and Tools ● Collaborations with PagerDuty, BMC, Splunk, othersWorks across Google 4 Comparing Compute Options Cloud Platform, Amazon Web Services ● Open source agents, integration 5 Quiz

Page 64 Page 65 Google Stackdriver’s areas of focus Google Cloud Source Repositories Beta

Monitoring Logging Platform, system, and Platform, system, and ● Fully-featured Git repositories hosted application metrics application logs on Google Cloud Platform Uptime/health checks Log search/view/filter Dashboards and alerts Log-based metrics ● Supports collaborative development of Trace Error Reporting cloud apps Latency reporting and Error notifications sampling Error dashboard ● Includes integration with Stackdriver Per-URL latency and debugger statistics Debugger Debug applications

Google Cloud Deployment Manager Agenda

1 Google Compute Engine Overview ● Infrastructure management service ● Create a .yaml template describing 2 Google Cloud Networking your environment and use Deployment Operations and Tools Manager to create resources 3

● Provides repeatable deployments 4 Comparing Compute Options

5 Quiz

Page 66 Page 67 Google Cloud FunctionsBeta Agenda

● Create single-purpose functions that 1 Google Compute Engine Overview respond to events without a server or runtime 2 Google Cloud Networking ○ Event examples: New instance created, file added to Cloud Storage 3 Operations and Tools ● Written in Javascript, execute in 4 Comparing Compute Options managed Node.js environment on Google Cloud Platform 5 Quiz

Comparing compute options Quiz

Service Compute Container App Engine App Engine Cloud Name 3 robust networking services available to your applications on Engine Engine Standard Flex FunctionsBeta Google Cloud Platform. Language Any Any Java, Python, Go Java, Python, JavaScript Cloud Virtual Network, Cloud Interconnect, Cloud DNS, Cloud support and PHP Node.js, Ruby, PHP, .NET core, and Go; Load Balancing, and Cloud CDN. or supply your own runtime Name 3 Compute Engine pricing innovations

Service IaaS Hybrid PaaS PaaS Serverless Per-minute billing, custom machine types, preemptible instances. model

Use cases General computing Container-based Web and mobile Web and mobile Ephemeral True or False: Google Cloud Load Balancing lets you balance HTTP workloads workloads applications applications, functions traffic across multiple Compute Engine regions. container-based responding to workloads events True.

Page 68 Page 69 Resources Agenda

● Google Compute Engine https://cloud.google.com/compute/docs/ ● Google Cloud CDN 1 Google Cloud Big Data Platform https://cloud.google.com/cdn/docs/ ● Google Cloud Stackdriver 2 Google Cloud Machine Learning Platform https://cloud.google.com/stackdriver/docs/ Google Cloud Deployment Manager ● 3 Quiz https://cloud.google.com/deployment-manager/docs/ ● Google Cloud Source Repositories https://cloud.google.com/source-repositories/docs/

Cloud OnBoard Google Cloud Big Data Platform Reduces integration risk, accelerates time to value

Integrated, NoOps cloud data platform for building scalable, secure and reliable data-driven applications that transform businesses and user experiences.

● Faster time-to-value ● Real-time applications ● Access to innovation, including machine learning ● Completeness ©Google Inc. or its affiliates. All rights reserved. Do not distribute. ©Google Inc. or its affiliates. All rights reserved. Do not distribute. ©Google Inc. or its affiliates. All rights reserved. Do not distribute.142 May only be taught by Google Cloud Platform Authorized Trainers.

Page 70 Page 71 Google Cloud Platform BigQuery: a fully-managed data warehouse

● Provides near real-time interactive

Machine Operations analysis of massive datasets Compute Networking Big Data Storage Learning and Tools (hundreds of TBs) ● Query using SQL syntax (SQL 2011) ● Zero administration for performance and scale Cloud Cloud Cloud Cloud BigQuery Pub/Sub Dataflow Dataproc Datalab

Big Data Services BigQuery runs on Google’s secure, high-performance infrastructure

● Compute and storage are separated with a terabit network in between BigQuery Pub/Sub Dataflow Dataproc ● You only pay for storage and Analytics database; Scalable & flexible Stream & batch Managed Hadoop Stream data at 100,000 enterprise messaging processing; Unified and MapReduce, Spark, processing used rows per second simplified pipelines Pig, and Hive service ● Automatic discount for long term data storage Fully Managed, NoOps Services

Page 72 Page 73 Personalized medicinal care for Why use Google Cloud Pub/Sub? BigQuery at UC Denver each patient Anschutz Medical Campus ● Use cases:

6 million ○ Building block for data ingestion in Dataflow, Genetic makeup, health history and Internet of Things (IoT), Marketing Analytics treatments of over 6 million patients ○ Foundation for Dataflow streaming ○ Push notifications for cloud-based applications ○ Connect applications across Google Cloud Platform (push/pull between Compute Engine 8 hrs 15 min BigQuery reduced time to execute and App Engine) complex analyses on the entire set of patient records to just 15 minutes

Google Cloud Pub/Sub: scalable, reliable messaging Google Cloud Dataflow: managed data pipelines

● Supports many-to-many asynchronous ● Processes data using Compute Engine messaging instances ○ Application components make ○ Clusters are sized for you push/pull subscriptions to topics ○ Automated scaling, no instance ● Includes support for offline consumers provisioning required ● Based on proven Google technologies ● Write code once and get batch and streaming ● Integrates with Cloud Dataflow for data processing pipelines ○ Transform-based programming model

Page 74 Page 75 Why use Google Cloud Dataflow? Why use Google Cloud Dataproc?

● Use cases: ● Use cases: ○ ETL (extract/transform/load) pipelines to move, ○ Easily migrate on-premises Hadoop jobs to filter, enrich, shape data the cloud ○ Data analysis - batch computation or continuous ○ Quickly analyze data (like log data) stored in computation using streaming Cloud Storage - create a cluster in less than 2 ○ Orchestration - create pipelines that coordinate minutes then delete it immediately services, including external services ○ Use Spark/Spark SQL to quickly perform data ● Integrates with GCP services like Cloud mining and analysis Storage, Cloud Pub/Sub, BigQuery, Bigtable ○ Use Spark Machine Learning Libraries (MLlib) to run classification algorithms ○ Open source Java and Python SDKs

Google Cloud Dataproc: managed Hadoop Google Cloud Datalab: interactive data exploration ● Fast, easy, managed way to run ● Interactive tool for large-scale data Hadoop and Spark/Hive/Pig on Google exploration, transformation, analysis, Cloud Platform visualization ● Create clusters in 90 sec or less ● Integrated, open source ● Scale clusters up and down even when ○ Runs on Google App Engine jobs are running ○ Built on Jupyter (formerly IPython)

Page 76 Page 77 Why use Google Cloud Datalab? Google Cloud Platform

● Create and manage code, documentation, results, visualizations in intuitive notebook

format Machine Operations Compute Networking Big Data Storage ○ Use Google Charts or matplotlib for easy Learning and Tools visualizations ● Analyze data in BigQuery, Compute Engine, and Cloud Storage using Python, SQL, and JavaScript Speech Translate Natural Cloud Vision API ● Easily deploy models to BigQuery Machine API API Language Learning API

Agenda Google Cloud Machine Learning Platform

Open source tool to build and run neural network models

● Wide platform support: CPU or GPU; mobile, server, or cloud ● Developed by researchers and engineers at Google Brain

1 Google Cloud Big Data Platform Fully managed machine learning service

● Faster training, better accuracy versus competing systems 2 Google Cloud Machine Learning Platform ● Familiar notebook-based developer experience Cloud ML ● Optimized for Google infrastructure; Integrates with BigQuery and Cloud Storage 3 Quiz Pre-trained machine learning models built by Google ● Speech: Stream results in real-time, detects 80 languages ● Vision: Identify objects, landmarks, text, content Machine Learning APIs ● Translate: Language translation including detection ● Natural Language: Structure, meaning of text

Page 78 Page 79 Google Cloud Machine Learning Use Cases Speech API Beta

Structured Data Unstructured Data ● Recognizes over 80 languages and Classification/ Regression Image Analytics ● Customer churn analysis ● Identify damaged shipments variants ● Product diagnostics ● Explicit content classification ● Forecasting ● Identify “styles” in images ● Can return text in real-time

Recommendation Text Analytics ● Highly accurate, even in noisy ● Content personalization ● Call center log analysis ● Product X-sells/up-sells ● Language identification environments ● Topic classification Anomaly Detection ● Access from any device Sentiment analysis ● Fraud detection ● Asset sensor diagnostics ● Powered by Google’s machine learning ● Log metric anomalies

Vision API Natural Language API

● Analyze images with a simple REST API ● Uses machine learning models to reveal structure, meaning of text ○ Face detection, logo detection, label ● Extract information about people, detection, and so on places, events mentioned in text ● With the Cloud Vision API, you can: documents, news articles, blog posts ○ Gain insight from images ● Analyze text uploaded in request or ○ Detect inappropriate content integrate with Cloud Storage ○ Analyze sentiment ○ Extract text ©Google Inc. or its affiliates. All rights reserved. Do not distribute.160 ©Google Inc. or its affiliates. All rights reserved. Do not distribute.162

Page 80 Page 81 Translate API Agenda

● Translate arbitrary strings between thousands of language pairs 1 Google Cloud Big Data Platform ● Programmatically detect a document’s language 2 Google Cloud Machine Learning Platform ● Support for dozens of languages 3 Quiz

Machine Learning APIs Quiz

Enable apps that see, hear, and understand. Name two use cases for Google Cloud Dataproc. Migrate on-premises Hadoop jobs to the cloud, data mining/analysis Name two use cases for Google Cloud Dataflow. ETL, orchestration Name three use cases for the Google machine learning platform Fraud detection, sentiment analysis, content personalization

Page 82 Page 83 Resources Google Cloud Technical Training and Certification

● Google Big Data Platform https://cloud.google.com/products/#big-data G Suite Admin What New with What New with G Suite Fundamentals G Suite Admin G Suite Admin ● Google Machine Learning Platform https://cloud.google.com/products/#machine-learning Cloud GCP GCP for Architecting with Architecting with Infrastructure Fundamentals - Systems GCP: Core GCP: Design and Core Infrastructure Operations Infrastructure Process

Data & Machine GCP Data Fundamentals - Engineering Learning Big Data & ML On GCP

Application GCP Developing Fundamentals - Solutions with Development Core Infrastructure GCP

Cloud OnBoard Training and Certification Schedule

cloud.google.com/training

©Google Inc. or its affiliates. All rights reserved. Do not distribute. ©Google Inc. or its affiliates. All rights reserved. Do not distribute. ©Google Inc. or its affiliates. All rights reserved. Do not distribute. May only be taught by Google Cloud Platform Authorized Trainers.

Page 84 Page 85 Google Cloud Self Paced Qwiklabs

6 7 See you 8 9 next time 10

13 google.qwiklabs.com 14 15

Google Cloud Coursera

coursera.org/googlecloud

Page 86 Page 87