DOCSLIB.ORG

Develop and Deploy Apps More Easily with Cloud Spanner and Cloud Bigtable Updates | Google Cloud Blog

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Develop and Deploy Apps More Easily with Cloud Spanner and Cloud Bigtable Updates | Google Cloud Blog 8/23/2020 Develop and deploy apps more easily with Cloud Spanner and Cloud Bigtable updates | Google Cloud Blog Blog Menu D ATAB ASES Develop and deploy apps more easily with Cloud Spanner and Cloud Bigtable updates FinDde aenp tai rStricivlaes..t.ava Product Manager, Cloud Spanner Misha Brukman Latest storiPersoduct Manager, Cloud Bigtable October 11, 2018 Products Topics About One of our goals at Google Cloud is to offer all the database choices that you want as managed services in order to remove operational complexity and toil from your day. RSS Feed We’ve got a full range of managed database services to address a variety of your workload needs. We’re always working to add features to our databases to make your https://cloud.google.cwomo/rbklogb/puroildduicntsg/datapbpassese/daesveielorp-and-deploy-apps-more-easily-with-cloud-spanner-and-cloud-bigtable-updates 1/6 8/23/2020 Develop and deploy apps more easily with Cloud Spanner and Cloud Bigtable updates | Google Cloud Blog work building apps easier. Today at Next London ‘18, we’re excited to announce the general availability of two important new features for two of our cloud-native database offerings: Cloud Spanner Blog and Cloud Bigtable. Menu Cloud Spanner: We’re adding enhancements to Cloud Spanner’s SQL capabilities to make it easier to read and write data in Cloud Spanner databases using SQL, and use off-the-shelf drivers and tooling. Cloud Spanner’s API now supports INSERT, UPDATE, and DELETE SQL statements. Cloud Bigtable: We’re offering Key Visualizer so you can see key access patterns in heatmap format to optimize your Cloud Bigtable schemas for improved performance. We’re adding this feature so you can build optimized data models to get the most out of your database. Key Visualizer is now generally available. To build the databases that you want, our process is simple. We listen to you, and then we build features and products to address your needs. If you have something you want us to build, let us know. Read on for more details on these useful new database capabilities, which add compatibility and ease of use for your cloud workloads. Cloud Spanner DML Cloud Spanner is a relational database built for ease of use at any scale, and relied on by leading enterprise, SaaS, and gaming applications. It offers massive scalability when you need it, along with relational semantics such as strong consistency and transactions across regions and continents. The standard way to manipulate data in relational databases is the SQL data manipulation language (DML). Cloud Spanner already supports SELECT queries, and today’s announcement adds support for INSERT, UPDATE, and DELETE statements in SQL as well. Here’s what you’ll see with this new query support: Find an article... Latest stories Products Topics About RSS Feed https://cloud.google.com/blog/products/databases/develop-and-deploy-apps-more-easily-with-cloud-spanner-and-cloud-bigtable-updates 2/6 8/23/2020 Develop and deploy apps more easily with Cloud Spanner and Cloud Bigtable updates | Google Cloud Blog Blog Menu Querying a Cloud Spanner database using DML operations through the Google Cloud Platform (GCP) console We’ve gotten great user feedback on the benefits of using DML in Cloud Spanner, including from cryptocurrency leader Blockchain. “Since we implemented Cloud Spanner for our teams, we've been impressed by its ease of use and simple integration with our existing tooling,” says Lewis Tuff, Engineering Lead at Blockchain. “One thing that has been missing for us is DML in SQL to support easier ad hoc querying, and now that we have it, we're excited to ship code even faster.” Along with DML support in Cloud Spanner, we are also releasing a new version of our JDBC driver that now supports writes, in addition to SELECTs. This update makes it easier to reuse existing code and tool chains, so you can integrate with popular object- relational mappers (ORMs). This reduces the learning curve for new Cloud Spanner users and makes porting existing applications easier. To get started, see the documentation. Key Visualizer for Cloud Bigtable The Key Visualizer feature, now generally available, helps visualize key access patterns to enable schema and performance optimization in Cloud Bigtable, our fully managed NoSQL key-value and wide-column database service. Cloud Bigtable is built for high throughput and low latency at any scale and is relied on for time-series, graph, and IoT workloads. We've heard from developers the desire for more visibility into their application behavior to optimize database schemas and get the most performance from Fintdh eainr daartiacblea.s..e—and we've built Key Visualizer to address this need. Key Visualizer has helped many teams here at Google and Google Cloud customers by providing them with insights into their database usage patterns, without impacting the serving capabilities Latest storioefs the database itself. This means you can design and refine your data model to best take advantage of the horizontal scalability of our GCP services.This functionality hasn’t Products been previously available for databases, but it’s an essential part of maximizing the performance of your applications. Topics Key Visualizer helps debug performance issues that are caused by unbalanced access patterns across the key space, or single rows that are either too large or receiving too About much read or write activity. With Key Visualizer, you get a heat map visualization of access patterns over time, along with the ability to zoom into specific key or time ranges RSS Feed to find more details about the hotspotting keys. This tool provides clear insights into the access patterns of a table, helping you resolve performance problems due to schema https://cloud.google.com/blog/products/databases/develop-and-deploy-apps-more-easily-with-cloud-spanner-and-cloud-bigtable-updates 3/6 8/23/2020 Develop and deploy apps more easily with Cloud Spanner and Cloud Bigtable updates | Google Cloud Blog design quickly. Key Visualizer is free and works at any scale, without any impact to Cloud Bigtable cluster performance. The Key Visualizer documentation provides guidance on what to do once a problem is Blog Menu detected. Key Visualizer creates scans for all Cloud Bigtable tables that meet at least one of the minimum criteria for size or activity. You can see more on how it works here: Key Visualizer for Cloud Bigtable We’ve heard great feedback from customers who have been using Key Visualizer in beta to achieve their performance objectives. “Key Visualizer for Google Cloud Bigtable is an amazing tool to quickly and visually identify hotspots in Bigtable,” says Joeri De Turck, CTO, OTA Insight, which provides revenue management tools for the hotel industry. “It helped us greatly to compare the performance of multiple possible key schemas and evaluate which one is the most efficient for our workload.” Find an article... Retailer eBay has also used Key Visualizer in beta. “With close to 200TB of data, 15 billion rows, and 10GB/s throughput, we are always working to better understand our Latest stories data access patterns at the storage level,” says Neeraj Jain, Data Architect, New Product Development at eBay. “One area we commonly look for is high read/write pressure in our Products Bigtable queries. Using Key Visualizer for Cloud Bigtable, we were able to get more details at the table level, which helped us perform faster diagnostics and make further Topics optimizations to how we store our data.” About With this launch, we are also adding a new feature: Multimetric views. This lets you view multiple metrics at the same time when you want to compare metrics or find RSS Feed correlations between them. Here's what a multi-metric display looks like, with miniature heatmaps on the right: https://cloud.google.com/blog/products/databases/develop-and-deploy-apps-more-easily-with-cloud-spanner-and-cloud-bigtable-updates 4/6 8/23/2020 Develop and deploy apps more easily with Cloud Spanner and Cloud Bigtable updates | Google Cloud Blog Blog Menu For more information on using Key Visualizer, check out this Google Cloud Next ‘18 presentation by Steve Niemitz, Staff Software Engineer at Twitter, together with Misha Brukman and Wim De Pauw from our team here at Google. These features continue to bring a better development experience to our managed database offerings so you can build apps faster. We look forward to seeing what you will build next! POSTED IN: DATABASES—GOOGLE CLOUD PLATFORM Find an article... Latest stories RELATED ARTICLES Products Databases that transform businesses 3 reasons to consider Cloud Spanner Topics —What happened at Google Cloud for your next project Next ‘20: OnAir About RSS Feed Follow Us https://cloud.google.com/blog/products/databases/develop-and-deploy-apps-more-easily-with-cloud-spanner-and-cloud-bigtable-updates 5/6 8/23/2020 Develop and deploy apps more easily with Cloud Spanner and Cloud Bigtable updates | Google Cloud Blog Blog Menu Privacy Terms About Google Google Cloud Products Language Help https://cloud.google.com/blog/products/databases/develop-and-deploy-apps-more-easily-with-cloud-spanner-and-cloud-bigtable-updates 6/6.
Load more

Recommended publications
  • F1 Query: Declarative Querying at Scale
    F1 Query: Declarative Querying at Scale Bart Samwel John Cieslewicz Ben Handy Jason Govig Petros Venetis Chanjun Yang Keith Peters Jeff Shute Daniel Tenedorio Himani Apte Felix Weigel David Wilhite Jiacheng Yang Jun Xu Jiexing Li Zhan Yuan Craig Chasseur Qiang Zeng Ian Rae Anurag Biyani Andrew Harn Yang Xia Andrey Gubichev Amr El-Helw Orri Erling Zhepeng Yan Mohan Yang Yiqun Wei Thanh Do Colin Zheng Goetz Graefe Somayeh Sardashti Ahmed M. Aly Divy Agrawal Ashish Gupta Shiv Venkataraman Google LLC [email protected] ABSTRACT 1. INTRODUCTION F1 Query is a stand-alone, federated query processing platform The data processing and analysis use cases in large organiza- that executes SQL queries against data stored in different file- tions like Google exhibit diverse requirements in data sizes, la- based formats as well as different storage systems at Google (e.g., tency, data sources and sinks, freshness, and the need for custom Bigtable, Spanner, Google Spreadsheets, etc.). F1 Query elimi- business logic. As a result, many data processing systems focus on nates the need to maintain the traditional distinction between dif- a particular slice of this requirements space, for instance on either ferent types of data processing workloads by simultaneously sup- transactional-style queries, medium-sized OLAP queries, or huge porting: (i) OLTP-style point queries that affect only a few records; Extract-Transform-Load (ETL) pipelines. Some systems are highly (ii) low-latency OLAP querying of large amounts of data; and (iii) extensible, while others are not. Some systems function mostly as a large ETL pipelines. F1 Query has also significantly reduced the closed silo, while others can easily pull in data from other sources.
    [Show full text]
  • Containers at Google
    Build What’s Next A Google Cloud Perspective Thomas Lichtenstein Customer Engineer, Google Cloud [email protected] 7 Cloud products with 1 billion users Google Cloud in DACH HAM BER ● New cloud region Germany Google Cloud Offices FRA Google Cloud Region (> 50% latency reduction) 3 Germany with 3 zones ● Commitment to GDPR MUC VIE compliance ZRH ● Partnership with MUC IoT platform connects nearly Manages “We found that Google Ads has the best system for 50 brands 250M+ precisely targeting customer segments in both the B2B with thousands of smart data sets per week and 3.5M and B2C spaces. It used to be hard to gain the right products searches per month via IoT platform insights to accurately measure our marketing spend and impacts. With Google Analytics, we can better connect the omnichannel customer journey.” Conrad is disrupting online retail with new Aleš Drábek, Chief Digital and Disruption Officer, Conrad Electronic services for mobility and IoT-enabled devices. Solution As Conrad transitions from a B2C retailer to an advanced B2B and Supports B2C platform for electronic products, it is using Google solutions to grow its customer base, develop on a reliable cloud infrastructure, Supports and digitize its workplaces and retail stores. Products Used 5x Mobile-First G Suite, Google Ads, Google Analytics, Google Chrome Enterprise, Google Chromebooks, Google Cloud Translation API, Google Cloud the IoT connections vs. strategy Vision API, Google Home, Apigee competitors Industry: Retail; Region: EMEA Number of Automate Everything running
    [Show full text]
  • Spanner: Becoming a SQL System
    Spanner: Becoming a SQL System David F. Bacon Nathan Bales Nico Bruno Brian F. Cooper Adam Dickinson Andrew Fikes Campbell Fraser Andrey Gubarev Milind Joshi Eugene Kogan Alexander Lloyd Sergey Melnik Rajesh Rao David Shue Christopher Taylor Marcel van der Holst Dale Woodford Google, Inc. ABSTRACT this paper, we focus on the “database system” aspects of Spanner, Spanner is a globally-distributed data management system that in particular how query execution has evolved and forced the rest backs hundreds of mission-critical services at Google. Spanner of Spanner to evolve. Most of these changes have occurred since is built on ideas from both the systems and database communi- [5] was written, and in many ways today’s Spanner is very different ties. The first Spanner paper published at OSDI’12 focused on the from what was described there. systems aspects such as scalability, automatic sharding, fault tol- A prime motivation for this evolution towards a more “database- erance, consistent replication, external consistency, and wide-area like” system was driven by the experiences of Google developers distribution. This paper highlights the database DNA of Spanner. trying to build on previous “key-value” storage systems. The pro- We describe distributed query execution in the presence of reshard- totypical example of such a key-value system is Bigtable [4], which ing, query restarts upon transient failures, range extraction that continues to see massive usage at Google for a variety of applica- drives query routing and index seeks, and the improved blockwise- tions. However, developers of many OLTP applications found it columnar storage format.
    [Show full text]
  • Economic and Social Impacts of Google Cloud September 2018 Economic and Social Impacts of Google Cloud |
    Economic and social impacts of Google Cloud September 2018 Economic and social impacts of Google Cloud | Contents Executive Summary 03 Introduction 10 Productivity impacts 15 Social and other impacts 29 Barriers to Cloud adoption and use 38 Policy actions to support Cloud adoption 42 Appendix 1. Country Sections 48 Appendix 2. Methodology 105 This final report (the “Final Report”) has been prepared by Deloitte Financial Advisory, S.L.U. (“Deloitte”) for Google in accordance with the contract with them dated 23rd February 2018 (“the Contract”) and on the basis of the scope and limitations set out below. The Final Report has been prepared solely for the purposes of assessment of the economic and social impacts of Google Cloud as set out in the Contract. It should not be used for any other purposes or in any other context, and Deloitte accepts no responsibility for its use in either regard. The Final Report is provided exclusively for Google’s use under the terms of the Contract. No party other than Google is entitled to rely on the Final Report for any purpose whatsoever and Deloitte accepts no responsibility or liability or duty of care to any party other than Google in respect of the Final Report and any of its contents. As set out in the Contract, the scope of our work has been limited by the time, information and explanations made available to us. The information contained in the Final Report has been obtained from Google and third party sources that are clearly referenced in the appropriate sections of the Final Report.
    [Show full text]
  • Spanner: Google's Globally-Distributed Database
    Spanner: Google's Globally-Distributed Database Corbett, Dean, et al. Jinliang Wei CMU CSD October 20, 2013 Jinliang Wei (CMU CSD) Spanner: Google's Globally-Distributed Database Corbett,October 20,Dean, 2013 et 1 al./ 21 What? - Key Features I Globally distributed I Versioned data I SQL transactions + key-value read/writes I External consistency I Automatic data migration across machines (even across datacenters) for load balancing and fautl tolerance. Jinliang Wei (CMU CSD) Spanner: Google's Globally-Distributed Database Corbett,October 20,Dean, 2013 et 2 al./ 21 External Consistency I Equivalent to linearizability I If a transaction T1 commits before another transaction T2 starts, then T1's commit timestamp is smaller than T 2. I Any read that sees T2 must see T1. I The strongest consistency guarantee that can be achieved in practice (Strict consistency is stronger, but not achievable in practice). Jinliang Wei (CMU CSD) Spanner: Google's Globally-Distributed Database Corbett,October 20,Dean, 2013 et 3 al./ 21 Why Spanner? I BigTable I Good performance I Does not support transaction across rows. I Hard to use. I Megastore I Support SQL transactions. I Many applications: Gmail, Calendar, AppEngine... I Poor write throughput. I Need SQL transactions + high throughput. Jinliang Wei (CMU CSD) Spanner: Google's Globally-Distributed Database Corbett,October 20,Dean, 2013 et 4 al./ 21 Spanserver Software Stack Figure: Spanner Server Software Stack Jinliang Wei (CMU CSD) Spanner: Google's Globally-Distributed Database Corbett,October 20,Dean, 2013 et 5 al./ 21 Spanserver Software Stack Cont. I Spanserver maintains data and serves client requests.
    [Show full text]
  • Scalable Crowd-Sourcing of Video from Mobile Devices
    Scalable Crowd-Sourcing of Video from Mobile Devices Pieter Simoens†, Yu Xiao‡, Padmanabhan Pillai•, Zhuo Chen, Kiryong Ha, Mahadev Satyanarayanan December 2012 CMU-CS-12-147 School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 †Carnegie Mellon University and Ghent University ‡Carnegie Mellon University and Aalto University •Intel Labs Abstract We propose a scalable Internet system for continuous collection of crowd-sourced video from devices such as Google Glass. Our hybrid cloud architecture for this system is effectively a CDN in reverse. It achieves scalability by decentralizing the cloud computing infrastructure using VM-based cloudlets. Based on time, location and content, privacy sensitive information is automatically removed from the video. This process, which we refer to as denaturing, is executed in a user-specific Virtual Machine (VM) on the cloudlet. Users can perform content-based searches on the total catalog of denatured videos. Our experiments reveal the bottlenecks for video upload, denaturing, indexing and content-based search and provide valuable insight on how parameters such as frame rate and resolution impact the system scalability. Copyright 2012 Carnegie Mellon University This research was supported by the National Science Foundation (NSF) under grant numbers CNS-0833882 and IIS-1065336, by an Intel Science and Technology Center grant, by the Department of Defense (DoD) under Contract No. FA8721-05-C-0003 for the operation of the Software Engineering Institute (SEI), a federally funded research and development center, and by the Academy of Finland under grant number 253860. Any opinions, findings, conclusions or recommendations expressed in this material are those of the authors and do not necessarily represent the views of the NSF, Intel, DoD, SEI, Academy of Finland, University of Ghent, Aalto University, or Carnegie Mellon University.
    [Show full text]
  • Trusting Your Data with Google Cloud Platform 2
    Google Cloud whitepaper September 2019 Trusting your data with Google Cloud Platform 2 Table of contents 1. Introduction 3 2. Managing your data lifecycle on GCP 4 2.1 Data usage 2.2 Data export/archive/backup 2.3 Data governance 2.4 Data residency 2.5 Security configuration management 2.6 Third party security solutions 2.7 Incident detection & response 3. Managing Google's access to your data 9 3.1 How does Google safeguard your data from unauthorized access? 3.2 Data export/archive/backup 3.2 Customer controls over Google access to data 3.3 Data access transparency 3.4 Google employee access authorization 3.5 What happens if we get a lawful request from a government for data? 4. Security and compliance standards 13 4.1 Independent verification of our control framework 4.2 Compliance support for customers Conclusion 14 Appendix: URLs 15 The information contained herein is intended to outline general product direction and should not be relied upon in making purchasing decisions nor shall it be used to trade in the securities of Alphabet Inc. The content is for informational purposes only and may not be incorporated into any contract. The information presented is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Any references to the development, release, and timing of any features or functionality described for these services remains at Google’s sole discretion. Product capabilities, time frames and features are subject to change and should not be viewed as Google commitments. 3 1. Introduction At Google Cloud we’ve set a high bar for what it means to host, serve, and protect customer data.
    [Show full text]
  • Spanner: Becoming a SQL System
    Spanner: Becoming a SQL System David F. Bacon Nathan Bales Nico Bruno Brian F. Cooper Adam Dickinson Andrew Fikes Campbell Fraser Andrey Gubarev Milind Joshi Eugene Kogan Alexander Lloyd Sergey Melnik Rajesh Rao David Shue Christopher Taylor Marcel van der Holst Dale Woodford Google, Inc. ABSTRACT this paper, we focus on the “database system” aspects of Spanner, Spanner is a globally-distributed data management system that in particular how query execution has evolved and forced the rest backs hundreds of mission-critical services at Google. Spanner of Spanner to evolve. Most of these changes have occurred since is built on ideas from both the systems and database communi- [5] was written, and in many ways today’s Spanner is very different ties. The first Spanner paper published at OSDI’12 focused on the from what was described there. systems aspects such as scalability, automatic sharding, fault tol- A prime motivation for this evolution towards a more “database- erance, consistent replication, external consistency, and wide-area like” system was driven by the experiences of Google developers distribution. This paper highlights the database DNA of Spanner. trying to build on previous “key-value” storage systems. The pro- We describe distributed query execution in the presence of reshard- totypical example of such a key-value system is Bigtable [4], which ing, query restarts upon transient failures, range extraction that continues to see massive usage at Google for a variety of applica- drives query routing and index seeks, and the improved blockwise- tions. However, developers of many OLTP applications found it columnar storage format.
    [Show full text]
  • How to Integrate Google Cloud Spanner with Denodo
    How to integrate Google Cloud Spanner with Denodo Revision 20210421 NOTE This document is confidential and proprietary of Denodo Technologies. No part of this document may be reproduced in any form by any means without prior written authorization of Denodo Technologies. Copyright © 2021 Denodo Technologies Proprietary and Confidential How to integrate Google Cloud Spanner with Denodo 20210421 2 of 15 CONTENTS 1 INTRODUCTION.................................................................2 1.1 SAMPLE GOOGLE CLOUD SPANNER DATABASE.................................3 2 CREATING A SERVICE ACCOUNT AND KEY IN GOOGLE CLOUD SPANNER............................................................................4 3 CONNECTING TO GOOGLE CLOUD SPANNER USING THE JDBC DRIVER...............................................................................8 3.1 DOWNLOAD THE JDBC DRIVER........................................................8 3.2 INSTALL THE JDBC DRIVER IN THE DENODO PLATFORM....................8 3.3 CONNECT TO GOOGLE CLOUD SPANNER........................................10 3.4 CREATE A JDBC DATA SOURCE.......................................................10 3.5 CREATE A BASE VIEW...................................................................13 4 REFERENCES...................................................................16 1 INTRODUCTION Spanner is a NewSQL database designed, built, and deployed at Google for OLTP systems. It is a fully managed, mission-critical, relational database service that offers transactional consistency at a global
    [Show full text]
  • Spanner: Google's Globally- Distributed Database
    Today’s Reminders • Discuss Project Ideas with Phil & Kevin – Phil’s Office Hours: After class today – Sign up for a slot: 11-12:30 or 3-4:20 this Friday Spanner: Google’s Globally-Distributed Database Phil Gibbons 15-712 F15 Lecture 14 2 Spanner: Google’s Globally- Database vs. Key-value Store Distributed Database [OSDI’12 best paper] “We provide a database instead of a key-value store to make it easier for programmers to write their applications.” James C. Corbett, Jeffrey Dean, Michael Epstein, Andrew Fikes, Christopher Frost, JJ Furman, Sanjay Ghemawat, Andrey Gubarev, Christopher Heiser, “We consistently received complaints from users that Bigtable Peter Hochschild, Wilson Hsieh, Sebastian Kanthak, can be difficult to use for some kinds of applications.” Eugene Kogan, Hongyi Li, Alexander Lloyd, Sergey Melnik, David Mwaura, David Nagle, Sean Quinlan, Rajesh Rao, Lindsay Rolig, Yasushi Saito, Michal Szymaniak, Christopher Taylor, Ruth Wang, Dale Woodford (Google x 26) 3 4 Spanner [Slides from OSDI’12 talk] • Worked on Spanner for 4½ years at time of OSDI’12 • Scalable, multi-version, globally-distributed, Spanner: Google’s synchronously-replicated database – Hundreds of datacenters, millions of machines, trillions of rows Globally-Distributed Database • Transaction Properties – Transactions are externally-consistent (a.k.a. Linearizable) Wilson Hsieh – Read-only transactions are lock-free representing a host of authors – (Read-write transactions use 2-phase-locking) OSDI 2012 • Flexible replication configuration 5 What is Spanner?
    [Show full text]
  • Streaming Integration for Google Cloud Spanner
    Streaming Integration for Google Cloud Spanner Spanner needs an enterprise-grade streaming data integration solution BENEFITS that complements its global scalability, transactional consistency, and • Ingest real-time data for high availability advantages. The Striim® data integration platform operational reporting and share continuously ingests business-critical data from a wide range of up-to-date Cloud Spanner data cloud or on-premises source systems and continuously moves to with other services Cloud Spanner. Striim enterprise-grade platform includes end-to-end capabilities for real-time ingestion, in-flight processing, and continuous • Comply with regulations and delivery with sub-second latency. With real-time data feeds from the requirements around data Striim platform, high-value transactional and operational workloads privacy and security via in-flight can thrive on Cloud Spanner. encryption, data masking, and filtering Available on the Google Cloud Platform as well as on premises, Striim comes with change data capture (CDC) capabilities for non-intrusive • Migrate databases with minimal data ingestion from major relational databases. It ingests real-time risk, without downtime or data loss with real-time data pipeline change data from Oracle, SQL Server, HPE NonStop, MySQL, PostgreSQL, monitoring and alerts and MariaDB without impacting the performance of the source systems. Striim also moves data from log files, message queues such as Kafka, • Easily offload operational sensors, Hadoop, and NoSQL databases in real time. workloads to the cloud by moving data in real time and Online Database Migration to Cloud Spanner in the desired format With real-time data synchronization capabilities, Striim enables Cloud Spanner customers to achieve a seamless, online database migration.
    [Show full text]
  • This Document Describes Best Practices for Using Spanner As the Primary Backend Database for Game State Storage
    1/25/2020 Best practices for using Cloud Spanner as a gaming database This document describes best practices for using Spanner as the primary backend database for game state storage. You can use Spanner (/spanner/) in place of common databases to store player authentication data and inventory data. This document is intended for game backend engineers working on long-term state storage, and game infrastructure operators and admins who support those systems and are interested in hosting their backend database on Google Cloud. Multiplayer and online games have evolved to require increasingly complex database structures for tracking player entitlements, state, and inventory data. Growing player bases and increasing game complexity have led to database solutions that are a challenge to scale and manage, frequently requiring the use of sharding (https://wikipedia.org/wiki/Shard_(database_architecture)) or clustering (https://wikipedia.org/wiki/MySQL_Cluster). Tracking valuable in-game items or critical player progress typically requires transactions and is challenging to work around in many types of distributed databases. Spanner is the rst scalable, enterprise-grade, globally distributed, and strongly consistent database service built for the cloud to combine the benets of relational database structure with non-relational horizontal scale. Many game companies have found it to be well-suited to replace both game state and authentication databases in production-scale systems. You can scale for additional performance or storage by using the Cloud Console to add nodes. Spanner can transparently handle global replication with strong consistency, eliminating your need to manage regional replicas. This best practices document discusses the following: Important Spanner concepts and differences from databases commonly used in games.
    [Show full text]