Google Cloud Storage. | 1
Total Page:16
File Type:pdf, Size:1020Kb
google cloud storage. | 1 how to use google cloud storage. *this is a guide that was created from lecture videos and is used to help you gain an understanding of how to use google cloud storage. Understanding Google Cloud Storage Google Cloud Platform- App Engine, Compute Engine, Cloud Storage, Cloud Datastore, Cloud SQL, BigQuery Cloud Storage- Global regional hosting of assets and data, edge caching = reduced latency = faster access, guaranteed up-time of 99.5% backup and restore options, no cap on storage limits, enhanced security, OAuth 2.0 authentication, group-based access controls such as ACL’s Cloud Storage is based on buckets and objects, you create a bucket that holds one or more objects. Access to the buckets are handled with the API via XML in Java or Python or JSON through Java, JavaScript, Python, go, and PHP. You can also access through the Online Google Cloud Console or the gsutil Command line Cloud Storage is great for files. Cloud SQL is best for data not contained in a discrete file- relational database based on MySQL that can be hosted in US, Europe or Asia with up to 100GB storage and 16GB of RAM. Cloud SQL has automatic database replication to multiple locations and point in time backup recovery. Mysqldump, MySQL Wire Protocol, JBDC and database instances can be spun up as needed. You access the database instances through Google Cloud Console interface, Standard PHP/MySQL connections, MySQL Client through command line or the JSON API mind movement machine. google cloud storage. | 2 Cloud Datastore is for non relational data based off NoSQL. It has automatic replication.Supports ACID (Atomicity, Consistency, Isolation and Durability) transactions for reliable processing. You can access Cloud Datastore with Google Cloud Console interface, gcd command line tool or JSON API BigQuery is used to analyze massive amounts of data. Access to BigQuery is through the UI or REST Interface. BigQuery is accessible through a host of client libraries for Java, .NET, Python, Go, Ruby, PhP, JavaScript. A SQL style interface is available which is accessed through the client libraries or web user interface Google App Engine is a PaaS compared to Google Compute Engine which has the infrastructure as a service (IaaS) model. App Engine can be used to build, deploy and maintain web and mobile applications and games App Engine provides complete server side management and automatically scales to unlimited users. Auto scaling will allow instances as needed, databases shared on demand and bandwidth expanded when necessary Google Cloud Storage is ideal for storing assets and data as a resource to Google App Engine. You can store, retrieve, display and delete assets with Cloud Storage. You can also read and write flat datasheets that will be kept in cloud storage. App Engine has full connectivity to Cloud SQL and Cloud Datastore and they both can be queried. You can manage your data with the Cloud SQL relational database or with Cloud Datastore non relational database. mind movement machine. google cloud storage. | 3 App Engine can access BigQuery wia SQL dialect and you can utilize the analysis prowess of BigQuery through the language specific API REST interfaces. Google Compute Engine is an IaaS for large scale computing needs. It has the same infrastructure as Google Search, Gmail and Ads. Compute Engine provides virtual machines on demand that give fast computations, has efficient scaling and you can specify how many core instances (1, 2, 4, 8, 16) Each core has up to 3.75 GB memory per core. Difference between App Engine vs Compute Engine App Engine gives developers the ability to control Compute Engine cores, and provide a web front end for Google Compute Engine data processing applications. Compute Engine offers direct and complete operating system management for your virtual machines Signup for Google Cloud -> Sign in or Create a Google account -> cloud.google.com -> get started -> enter information needed and payment method -> Create project -> Project ID is a unique identifier that Google provides but you can change the name. Enabled APIs under API & Auth -> API -> Enabled: BigQuery, Debuglet Controller, Cloud SQL, Cloud Storage, Cloud Storage JSON API -> you can enable any API as needed that are provided for Google Cloud. If you need more information on getting the foundations of how to use google cloud storage, visit our article laying the foundations of Google Cloud. Google Cloud Storage Architecture Unlimited file storage, unlimited file size, different pricing levels and are assigned on the mind movement machine. google cloud storage. | 4 bucket level. Google Cloud Storage uses Google’s infrastructure, with high security and globally placed datacenters and fast retrieval due to edge caching for minimal latency Google Cloud Storage is used for storage of digital content, retrieval from any capable computer/device, secure sharing that supports access control lists and OAuth 2.0 Authentication. You can perform data analysis of stored data with Google Cloud integration with BigQuery. You can integrate App Engine for mobile gaming and app development. You can also host static website content in Google Cloud Storage. Google Cloud Storage Access from the Google Cloud Storage Manager web interface, HTTP standard methods such as get, put and post, the REST interface or gsutil, a command line interface. Google Cloud Storage is project based and have buckets within projects, And within buckets, there are objects. Projects -> Buckets -> Objects Google Cloud Storage buckets are the main container, they provide access level control and are project specific. You cannot share buckets from one project to another. And you cannot NEST buckets. You cannot have a bucket inside of a bucket. Google Cloud Storage objects are the content that resides in a bucket. Objects are bucket specific and cannot be shared between buckets. An object is made up of the object and the objects metadata, which is a series of name/value pairs that describe the object. Objects are mind movement machine. google cloud storage. | 5 verified when uploaded and they must be readable to be acceptable Once read, the objects are available. Once deleted, they are gone. Google Cloud Storage supports browser authenticated file transfer, pause and resume uploading and downloading, and automatic pause/resume when necessary. Google Cloud Storage can be programmatically user controlled. Installing Google Cloud SDK on a Mac- cloud.google.com/sdk -> ensure you have Python on your machine -> follow installation instructions Google Cloud Storage: Buckets Ensure API for Google Cloud Storage is enabled -> Google Cloud Storage -> create bucket -> bucket name must be unique to Cloud Storage namespace and conform to DNS naming conventions (3 to 63 characters long) ->upload files -> select file you want to upload to the created object -> you can also upload folders Manipulating objects in buckets Cloud Storage: Storage Browser -> select bucket you want to utilize objects for -> edit metadata -> -> save -> edit permissions will allow permissions for user account management -> to access object go to storage.cloud.google.com/bucket name/object name -> ‘shared publicly’ will allow you to publish through storage.googleapis.com/bucketname/objectname To get a better understanding of Google Cloud Buckets,, go to the googlecloudplatform on github. mind movement machine. google cloud storage. | 6 Google Cloud Storage: gsutil Authenticate with Google Cloud SDK -> specify project with gsutil: gcloud config set project (project id) -> gsutil ls will list buckets in project -> gsutil ls -L will give you the details about how many buckets you have in a JSON format, including who can access it, the access is the ACL To see what is inside of a bucket with gsutil: gsutil ls gs://name of bucket -> you can use gsutil ls -l to see object information Gsutil command to create a bucket- gsutil mb gs://unique bucket name -> check in Cloud Storage to ensure bucket was created Files are moved from local to remote folders with cp command -> gsutil cp name-of-file gs://destination-of-file gs://unique-bucket-name -> will copy and upload files to buckets Rearrange objects from bucket to bucket -> mv gs://source-bucket source-object gs://destination bucket cloud.google.com/storage/docs/gsutil for full list of commands and documentation Build static website with Cloud Storage (not using server side language/not database driven) Verify you are owner of website through google search console -> setup CNAME alias that points to the domain/subdomain to a Google API Service, you can do this under your current hosting provider in the DNS Zone file -> Add Record: CNAME and fill out information -> ensure you have created a new CNAME alias -> Cloud Storage: Storage Browser -> Add new bucket -> bucket name MUST be the same name of the subdomain/domain that you are mind movement machine. google cloud storage. | 7 going to use ex mind.mindmovementmachine.com -> Create bucket -> add website contents/files to your root -> open command line of root -> gsutil -m cp -r name-of-file–path gs://bucketname to move assets to Google Cloud Storage bucket -> ensure your files have downloaded into the appropriate Google Cloud Storage bucket -> gsutil -m acl set -r public- read gs://bucketname to make files publicly available -> specify which page is default (which should be homepage for example) command: gsutil web set -m homepage -e error-page gs://bucket-name to set the main homepage and the error page Use gsutil Create new bucket copy files to bucket -> move files to gallery bucket -> delete empty bucket Google Cloud SQL Fully managed relational database service that uses MySQL Data automatically replicated across multiple datacenters. MySQL is widely in used and you can import databases via mysqldump or export to mysqldump format You can create multiple database instances with 100GB per instance and 16GB of RAM per instance Cloud SQL Access can be done with Java, Python, the command line tool or the SQL prompt in the Google Cloud Console Cloud SQL only supports MySQL 5.5 or higher, the instance size is limited to 500GB.