DOCSLIB.ORG

Spanner: Google's Globally- Distributed Database

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Spanner: Google's Globally- Distributed Database Today’s Reminders • Discuss Project Ideas with Phil & Kevin – Phil’s Office Hours: After class today – Sign up for a slot: 11-12:30 or 3-4:20 this Friday Spanner: Google’s Globally-Distributed Database Phil Gibbons 15-712 F15 Lecture 14 2 Spanner: Google’s Globally- Database vs. Key-value Store Distributed Database [OSDI’12 best paper] “We provide a database instead of a key-value store to make it easier for programmers to write their applications.” James C. Corbett, Jeffrey Dean, Michael Epstein, Andrew Fikes, Christopher Frost, JJ Furman, Sanjay Ghemawat, Andrey Gubarev, Christopher Heiser, “We consistently received complaints from users that Bigtable Peter Hochschild, Wilson Hsieh, Sebastian Kanthak, can be difficult to use for some kinds of applications.” Eugene Kogan, Hongyi Li, Alexander Lloyd, Sergey Melnik, David Mwaura, David Nagle, Sean Quinlan, Rajesh Rao, Lindsay Rolig, Yasushi Saito, Michal Szymaniak, Christopher Taylor, Ruth Wang, Dale Woodford (Google x 26) 3 4 Spanner [Slides from OSDI’12 talk] • Worked on Spanner for 4½ years at time of OSDI’12 • Scalable, multi-version, globally-distributed, Spanner: Google’s synchronously-replicated database – Hundreds of datacenters, millions of machines, trillions of rows Globally-Distributed Database • Transaction Properties – Transactions are externally-consistent (a.k.a. Linearizable) Wilson Hsieh – Read-only transactions are lock-free representing a host of authors – (Read-write transactions use 2-phase-locking) OSDI 2012 • Flexible replication configuration 5 What is Spanner? Example: Social Network • Distributed multiversion database Sao Paulo • x1000 Santiago General-purpose transactions (ACID) Buenos Aires • SQL query language • San Francisco Brazil Schematized tables Seattle • x1000 Arizona User posts Semi-relational data model Friend lists Moscow US London Paris x1000 Berlin Krakow • Running in production x1000 Berlin • Storage for Google’s ad data Madrid Lisbon Russia • Replaced a sharded MySQL database Spain OSDI 2012 7 OSDI 2012 8 Overview Read Transactions • Feature: Lock-free distributed read transactions • Generate a page of friends’ recent posts • Property: External consistency of distributed – Consistent view of friend list and their posts transactions – First system at global scale • Implementation: Integration of concurrency control, replication, and 2PC Why consistency matters – Correctness and performance 1. Remove untrustworthy person X as friend • Enabling technology: TrueTime 2. Post P: “My government is repressive…” – Interval-based global time OSDI 2012 9 OSDI 2012 10 Single Machine Multiple Machines Block writes Block writes Generate my page Friend1 post User posts Friend1 post Friend2 post Friend lists Friend2 post … Friend999 post User posts … Generate my page Friend lists Friend1000 post Friend999 post User posts Friend1000 post Friend lists OSDI 2012 11 OSDI 2012 12 Multiple Datacenters Version Management User posts Friend1 post x1000 • US Friend lists Transactions that write use strict 2PL – Each transaction T is assigned a timestamp s User posts – Data written by T is timestamped with s Friend2 post x1000 Friend lists Spain … Generate my page Time<8 8 15 User posts Friend999 post x1000 My friends [X] [] Friend lists Brazil My posts [P] X’s friends [me] [] User posts Friend1000 post x1000 Russia Friend lists OSDI 2012 13 OSDI 2012 14 Synchronizing Snapshots Timestamps, Global Clock Global wall-clock time • Strict two-phase locking for write transactions == • Assign timestamp while locks are held External Consistency: Commit order respects global wall-time order Acquired locks Release locks == T Timestamp order respects global wall-time order Pick s = now() given timestamp order == commit order OSDI 2012 15 OSDI 2012 16 Timestamp Invariants TrueTime • Timestamp order == commit order • “Global wall-clock time” with bounded T1 uncertainty T2 TT.now() • Timestamp order respects global wall-time order time earliest latest T3 2*ε T4 OSDI 2012 17 OSDI 2012 18 Timestamps and TrueTime Commit Wait and Replication Start consensus Achieve consensus Notify slaves Acquired locks Release locks T Acquired locks Release locks T Pick s = TT.now().latest s Wait until TT.now().earliest > s Pick s Commit wait done Commit wait average ε average ε OSDI 2012 19 OSDI 2012 20 Commit Wait and 2-Phase Commit Example Start logging Done logging Remove X from Risky post P Acquired locks Release locks my friend list TC T2 TC Committed s =6 s=8 s=15 Notify participants of s C Acquired locks Release locks Remove myself from X’s friend list TP1 TP Acquired locks Release locks sP=8 s=8 T P2 Prepared Time <8 8 15 Send s Compute s for each Commit wait done My friends [X] [] My posts [P] Compute overall s X’s friends [me] [] OSDI 2012 21 OSDI 2012 22 What Have We Covered? What Haven’t We Covered? • Lock-free read transactions across datacenters • How to read at the present time • External consistency • Atomic schema changes • Timestamp assignment – Mostly non-blocking • TrueTime – Commit in the future – Uncertainty in time can be waited out • Non-blocking reads in the past – At any sufficiently up-to-date replica OSDI 2012 23 OSDI 2012 24 TrueTime Architecture TrueTime implementation GPS GPS GPS timemaster timemaster timemaster now = reference now + local-clock offset ε = reference ε + worst-case local-clock drift GPS Atomic-clock GPS timemaster timemaster timemaster ε +6ms 200 μs/sec Client Datacenter 1Datacenter 2 … Datacenter n reference uncertainty time Compute reference [earliest, latest] = now ± ε 0sec 30sec 60sec 90sec OSDI 2012 25 OSDI 2012 26 What If a Clock Goes Rogue? Network-Induced Uncertainty • Timestamp assignment would violate external 10 6 99.9 5 consistency 8 99 90 ) • Empirically unlikely based on 1 year of data s 4 m ( 6 n o – Bad CPUs 6 times more likely than bad clocks l 3 i s p 4 E 2 2 1 Mar 29 Mar 30 Mar 31 Apr 1 6AM 8AM 10AM 12PM Date Date (April 13) OSDI 2012 27 OSDI 2012 28 What’s in the Literature Future Work • External consistency/linearizability • Improving TrueTime • Distributed databases – Lower ε < 1 ms • Concurrency control • Building out database features • Replication – Finish implementing basic features – Efficiently support rich query patterns • Time (NTP, Marzullo) OSDI 2012 29 OSDI 2012 30 Semi-relational Data Model Conclusions • Rows must have names – Every table must have one or more primary-key columns • Reify clock uncertainty in time APIs – Known unknowns are better than unknown unknowns • Applications control data locality thru their choice of keys – Rethink algorithms to make use of uncertainty • Stronger semantics are achievable – Greater scale != weaker semantics [End of slides from OSDI’12 talk] OSDI 2012 31 32 Read-Only Transactions Constraints Availability • Must declare “read-only” upfront • Must have “scope” expression – Summarize keys that will be read by the entire transaction 33 34 F1 Advertising Backend Friday Discuss Projects with Phil & Kevin 1) the problem you want to solve 2) the approach you are going to take 3) how it pertains to the course Lock Conflicts Only one DC had SSDs 35 36.
Load more

Recommended publications
  • F1 Query: Declarative Querying at Scale
    F1 Query: Declarative Querying at Scale Bart Samwel John Cieslewicz Ben Handy Jason Govig Petros Venetis Chanjun Yang Keith Peters Jeff Shute Daniel Tenedorio Himani Apte Felix Weigel David Wilhite Jiacheng Yang Jun Xu Jiexing Li Zhan Yuan Craig Chasseur Qiang Zeng Ian Rae Anurag Biyani Andrew Harn Yang Xia Andrey Gubichev Amr El-Helw Orri Erling Zhepeng Yan Mohan Yang Yiqun Wei Thanh Do Colin Zheng Goetz Graefe Somayeh Sardashti Ahmed M. Aly Divy Agrawal Ashish Gupta Shiv Venkataraman Google LLC [email protected] ABSTRACT 1. INTRODUCTION F1 Query is a stand-alone, federated query processing platform The data processing and analysis use cases in large organiza- that executes SQL queries against data stored in different file- tions like Google exhibit diverse requirements in data sizes, la- based formats as well as different storage systems at Google (e.g., tency, data sources and sinks, freshness, and the need for custom Bigtable, Spanner, Google Spreadsheets, etc.). F1 Query elimi- business logic. As a result, many data processing systems focus on nates the need to maintain the traditional distinction between dif- a particular slice of this requirements space, for instance on either ferent types of data processing workloads by simultaneously sup- transactional-style queries, medium-sized OLAP queries, or huge porting: (i) OLTP-style point queries that affect only a few records; Extract-Transform-Load (ETL) pipelines. Some systems are highly (ii) low-latency OLAP querying of large amounts of data; and (iii) extensible, while others are not. Some systems function mostly as a large ETL pipelines. F1 Query has also significantly reduced the closed silo, while others can easily pull in data from other sources.
    [Show full text]
  • Containers at Google
    Build What’s Next A Google Cloud Perspective Thomas Lichtenstein Customer Engineer, Google Cloud [email protected] 7 Cloud products with 1 billion users Google Cloud in DACH HAM BER ● New cloud region Germany Google Cloud Offices FRA Google Cloud Region (> 50% latency reduction) 3 Germany with 3 zones ● Commitment to GDPR MUC VIE compliance ZRH ● Partnership with MUC IoT platform connects nearly Manages “We found that Google Ads has the best system for 50 brands 250M+ precisely targeting customer segments in both the B2B with thousands of smart data sets per week and 3.5M and B2C spaces. It used to be hard to gain the right products searches per month via IoT platform insights to accurately measure our marketing spend and impacts. With Google Analytics, we can better connect the omnichannel customer journey.” Conrad is disrupting online retail with new Aleš Drábek, Chief Digital and Disruption Officer, Conrad Electronic services for mobility and IoT-enabled devices. Solution As Conrad transitions from a B2C retailer to an advanced B2B and Supports B2C platform for electronic products, it is using Google solutions to grow its customer base, develop on a reliable cloud infrastructure, Supports and digitize its workplaces and retail stores. Products Used 5x Mobile-First G Suite, Google Ads, Google Analytics, Google Chrome Enterprise, Google Chromebooks, Google Cloud Translation API, Google Cloud the IoT connections vs. strategy Vision API, Google Home, Apigee competitors Industry: Retail; Region: EMEA Number of Automate Everything running
    [Show full text]
  • Spanner: Becoming a SQL System
    Spanner: Becoming a SQL System David F. Bacon Nathan Bales Nico Bruno Brian F. Cooper Adam Dickinson Andrew Fikes Campbell Fraser Andrey Gubarev Milind Joshi Eugene Kogan Alexander Lloyd Sergey Melnik Rajesh Rao David Shue Christopher Taylor Marcel van der Holst Dale Woodford Google, Inc. ABSTRACT this paper, we focus on the “database system” aspects of Spanner, Spanner is a globally-distributed data management system that in particular how query execution has evolved and forced the rest backs hundreds of mission-critical services at Google. Spanner of Spanner to evolve. Most of these changes have occurred since is built on ideas from both the systems and database communi- [5] was written, and in many ways today’s Spanner is very different ties. The first Spanner paper published at OSDI’12 focused on the from what was described there. systems aspects such as scalability, automatic sharding, fault tol- A prime motivation for this evolution towards a more “database- erance, consistent replication, external consistency, and wide-area like” system was driven by the experiences of Google developers distribution. This paper highlights the database DNA of Spanner. trying to build on previous “key-value” storage systems. The pro- We describe distributed query execution in the presence of reshard- totypical example of such a key-value system is Bigtable [4], which ing, query restarts upon transient failures, range extraction that continues to see massive usage at Google for a variety of applica- drives query routing and index seeks, and the improved blockwise- tions. However, developers of many OLTP applications found it columnar storage format.
    [Show full text]
  • Economic and Social Impacts of Google Cloud September 2018 Economic and Social Impacts of Google Cloud |
    Economic and social impacts of Google Cloud September 2018 Economic and social impacts of Google Cloud | Contents Executive Summary 03 Introduction 10 Productivity impacts 15 Social and other impacts 29 Barriers to Cloud adoption and use 38 Policy actions to support Cloud adoption 42 Appendix 1. Country Sections 48 Appendix 2. Methodology 105 This final report (the “Final Report”) has been prepared by Deloitte Financial Advisory, S.L.U. (“Deloitte”) for Google in accordance with the contract with them dated 23rd February 2018 (“the Contract”) and on the basis of the scope and limitations set out below. The Final Report has been prepared solely for the purposes of assessment of the economic and social impacts of Google Cloud as set out in the Contract. It should not be used for any other purposes or in any other context, and Deloitte accepts no responsibility for its use in either regard. The Final Report is provided exclusively for Google’s use under the terms of the Contract. No party other than Google is entitled to rely on the Final Report for any purpose whatsoever and Deloitte accepts no responsibility or liability or duty of care to any party other than Google in respect of the Final Report and any of its contents. As set out in the Contract, the scope of our work has been limited by the time, information and explanations made available to us. The information contained in the Final Report has been obtained from Google and third party sources that are clearly referenced in the appropriate sections of the Final Report.
    [Show full text]
  • Spanner: Google's Globally-Distributed Database
    Spanner: Google's Globally-Distributed Database Corbett, Dean, et al. Jinliang Wei CMU CSD October 20, 2013 Jinliang Wei (CMU CSD) Spanner: Google's Globally-Distributed Database Corbett,October 20,Dean, 2013 et 1 al./ 21 What? - Key Features I Globally distributed I Versioned data I SQL transactions + key-value read/writes I External consistency I Automatic data migration across machines (even across datacenters) for load balancing and fautl tolerance. Jinliang Wei (CMU CSD) Spanner: Google's Globally-Distributed Database Corbett,October 20,Dean, 2013 et 2 al./ 21 External Consistency I Equivalent to linearizability I If a transaction T1 commits before another transaction T2 starts, then T1's commit timestamp is smaller than T 2. I Any read that sees T2 must see T1. I The strongest consistency guarantee that can be achieved in practice (Strict consistency is stronger, but not achievable in practice). Jinliang Wei (CMU CSD) Spanner: Google's Globally-Distributed Database Corbett,October 20,Dean, 2013 et 3 al./ 21 Why Spanner? I BigTable I Good performance I Does not support transaction across rows. I Hard to use. I Megastore I Support SQL transactions. I Many applications: Gmail, Calendar, AppEngine... I Poor write throughput. I Need SQL transactions + high throughput. Jinliang Wei (CMU CSD) Spanner: Google's Globally-Distributed Database Corbett,October 20,Dean, 2013 et 4 al./ 21 Spanserver Software Stack Figure: Spanner Server Software Stack Jinliang Wei (CMU CSD) Spanner: Google's Globally-Distributed Database Corbett,October 20,Dean, 2013 et 5 al./ 21 Spanserver Software Stack Cont. I Spanserver maintains data and serves client requests.
    [Show full text]
  • Scalable Crowd-Sourcing of Video from Mobile Devices
    Scalable Crowd-Sourcing of Video from Mobile Devices Pieter Simoens†, Yu Xiao‡, Padmanabhan Pillai•, Zhuo Chen, Kiryong Ha, Mahadev Satyanarayanan December 2012 CMU-CS-12-147 School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 †Carnegie Mellon University and Ghent University ‡Carnegie Mellon University and Aalto University •Intel Labs Abstract We propose a scalable Internet system for continuous collection of crowd-sourced video from devices such as Google Glass. Our hybrid cloud architecture for this system is effectively a CDN in reverse. It achieves scalability by decentralizing the cloud computing infrastructure using VM-based cloudlets. Based on time, location and content, privacy sensitive information is automatically removed from the video. This process, which we refer to as denaturing, is executed in a user-specific Virtual Machine (VM) on the cloudlet. Users can perform content-based searches on the total catalog of denatured videos. Our experiments reveal the bottlenecks for video upload, denaturing, indexing and content-based search and provide valuable insight on how parameters such as frame rate and resolution impact the system scalability. Copyright 2012 Carnegie Mellon University This research was supported by the National Science Foundation (NSF) under grant numbers CNS-0833882 and IIS-1065336, by an Intel Science and Technology Center grant, by the Department of Defense (DoD) under Contract No. FA8721-05-C-0003 for the operation of the Software Engineering Institute (SEI), a federally funded research and development center, and by the Academy of Finland under grant number 253860. Any opinions, findings, conclusions or recommendations expressed in this material are those of the authors and do not necessarily represent the views of the NSF, Intel, DoD, SEI, Academy of Finland, University of Ghent, Aalto University, or Carnegie Mellon University.
    [Show full text]
  • Develop and Deploy Apps More Easily with Cloud Spanner and Cloud Bigtable Updates | Google Cloud Blog
    8/23/2020 Develop and deploy apps more easily with Cloud Spanner and Cloud Bigtable updates | Google Cloud Blog Blog Menu D ATAB ASES Develop and deploy apps more easily with Cloud Spanner and Cloud Bigtable updates FinDde aenp tai rStricivlaes..t.ava Product Manager, Cloud Spanner Misha Brukman Latest storiPersoduct Manager, Cloud Bigtable October 11, 2018 Products Topics About One of our goals at Google Cloud is to offer all the database choices that you want as managed services in order to remove operational complexity and toil from your day. RSS Feed We’ve got a full range of managed database services to address a variety of your workload needs. We’re always working to add features to our databases to make your https://cloud.google.cwomo/rbklogb/puroildduicntsg/datapbpassese/daesveielorp-and-deploy-apps-more-easily-with-cloud-spanner-and-cloud-bigtable-updates 1/6 8/23/2020 Develop and deploy apps more easily with Cloud Spanner and Cloud Bigtable updates | Google Cloud Blog work building apps easier. Today at Next London ‘18, we’re excited to announce the general availability of two important new features for two of our cloud-native database offerings: Cloud Spanner Blog and Cloud Bigtable. Menu Cloud Spanner: We’re adding enhancements to Cloud Spanner’s SQL capabilities to make it easier to read and write data in Cloud Spanner databases using SQL, and use off-the-shelf drivers and tooling. Cloud Spanner’s API now supports INSERT, UPDATE, and DELETE SQL statements. Cloud Bigtable: We’re offering Key Visualizer so you can see key access patterns in heatmap format to optimize your Cloud Bigtable schemas for improved performance.
    [Show full text]
  • Trusting Your Data with Google Cloud Platform 2
    Google Cloud whitepaper September 2019 Trusting your data with Google Cloud Platform 2 Table of contents 1. Introduction 3 2. Managing your data lifecycle on GCP 4 2.1 Data usage 2.2 Data export/archive/backup 2.3 Data governance 2.4 Data residency 2.5 Security configuration management 2.6 Third party security solutions 2.7 Incident detection & response 3. Managing Google's access to your data 9 3.1 How does Google safeguard your data from unauthorized access? 3.2 Data export/archive/backup 3.2 Customer controls over Google access to data 3.3 Data access transparency 3.4 Google employee access authorization 3.5 What happens if we get a lawful request from a government for data? 4. Security and compliance standards 13 4.1 Independent verification of our control framework 4.2 Compliance support for customers Conclusion 14 Appendix: URLs 15 The information contained herein is intended to outline general product direction and should not be relied upon in making purchasing decisions nor shall it be used to trade in the securities of Alphabet Inc. The content is for informational purposes only and may not be incorporated into any contract. The information presented is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Any references to the development, release, and timing of any features or functionality described for these services remains at Google’s sole discretion. Product capabilities, time frames and features are subject to change and should not be viewed as Google commitments. 3 1. Introduction At Google Cloud we’ve set a high bar for what it means to host, serve, and protect customer data.
    [Show full text]
  • Spanner: Becoming a SQL System
    Spanner: Becoming a SQL System David F. Bacon Nathan Bales Nico Bruno Brian F. Cooper Adam Dickinson Andrew Fikes Campbell Fraser Andrey Gubarev Milind Joshi Eugene Kogan Alexander Lloyd Sergey Melnik Rajesh Rao David Shue Christopher Taylor Marcel van der Holst Dale Woodford Google, Inc. ABSTRACT this paper, we focus on the “database system” aspects of Spanner, Spanner is a globally-distributed data management system that in particular how query execution has evolved and forced the rest backs hundreds of mission-critical services at Google. Spanner of Spanner to evolve. Most of these changes have occurred since is built on ideas from both the systems and database communi- [5] was written, and in many ways today’s Spanner is very different ties. The first Spanner paper published at OSDI’12 focused on the from what was described there. systems aspects such as scalability, automatic sharding, fault tol- A prime motivation for this evolution towards a more “database- erance, consistent replication, external consistency, and wide-area like” system was driven by the experiences of Google developers distribution. This paper highlights the database DNA of Spanner. trying to build on previous “key-value” storage systems. The pro- We describe distributed query execution in the presence of reshard- totypical example of such a key-value system is Bigtable [4], which ing, query restarts upon transient failures, range extraction that continues to see massive usage at Google for a variety of applica- drives query routing and index seeks, and the improved blockwise- tions. However, developers of many OLTP applications found it columnar storage format.
    [Show full text]
  • How to Integrate Google Cloud Spanner with Denodo
    How to integrate Google Cloud Spanner with Denodo Revision 20210421 NOTE This document is confidential and proprietary of Denodo Technologies. No part of this document may be reproduced in any form by any means without prior written authorization of Denodo Technologies. Copyright © 2021 Denodo Technologies Proprietary and Confidential How to integrate Google Cloud Spanner with Denodo 20210421 2 of 15 CONTENTS 1 INTRODUCTION.................................................................2 1.1 SAMPLE GOOGLE CLOUD SPANNER DATABASE.................................3 2 CREATING A SERVICE ACCOUNT AND KEY IN GOOGLE CLOUD SPANNER............................................................................4 3 CONNECTING TO GOOGLE CLOUD SPANNER USING THE JDBC DRIVER...............................................................................8 3.1 DOWNLOAD THE JDBC DRIVER........................................................8 3.2 INSTALL THE JDBC DRIVER IN THE DENODO PLATFORM....................8 3.3 CONNECT TO GOOGLE CLOUD SPANNER........................................10 3.4 CREATE A JDBC DATA SOURCE.......................................................10 3.5 CREATE A BASE VIEW...................................................................13 4 REFERENCES...................................................................16 1 INTRODUCTION Spanner is a NewSQL database designed, built, and deployed at Google for OLTP systems. It is a fully managed, mission-critical, relational database service that offers transactional consistency at a global
    [Show full text]
  • Streaming Integration for Google Cloud Spanner
    Streaming Integration for Google Cloud Spanner Spanner needs an enterprise-grade streaming data integration solution BENEFITS that complements its global scalability, transactional consistency, and • Ingest real-time data for high availability advantages. The Striim® data integration platform operational reporting and share continuously ingests business-critical data from a wide range of up-to-date Cloud Spanner data cloud or on-premises source systems and continuously moves to with other services Cloud Spanner. Striim enterprise-grade platform includes end-to-end capabilities for real-time ingestion, in-flight processing, and continuous • Comply with regulations and delivery with sub-second latency. With real-time data feeds from the requirements around data Striim platform, high-value transactional and operational workloads privacy and security via in-flight can thrive on Cloud Spanner. encryption, data masking, and filtering Available on the Google Cloud Platform as well as on premises, Striim comes with change data capture (CDC) capabilities for non-intrusive • Migrate databases with minimal data ingestion from major relational databases. It ingests real-time risk, without downtime or data loss with real-time data pipeline change data from Oracle, SQL Server, HPE NonStop, MySQL, PostgreSQL, monitoring and alerts and MariaDB without impacting the performance of the source systems. Striim also moves data from log files, message queues such as Kafka, • Easily offload operational sensors, Hadoop, and NoSQL databases in real time. workloads to the cloud by moving data in real time and Online Database Migration to Cloud Spanner in the desired format With real-time data synchronization capabilities, Striim enables Cloud Spanner customers to achieve a seamless, online database migration.
    [Show full text]
  • This Document Describes Best Practices for Using Spanner As the Primary Backend Database for Game State Storage
    1/25/2020 Best practices for using Cloud Spanner as a gaming database This document describes best practices for using Spanner as the primary backend database for game state storage. You can use Spanner (/spanner/) in place of common databases to store player authentication data and inventory data. This document is intended for game backend engineers working on long-term state storage, and game infrastructure operators and admins who support those systems and are interested in hosting their backend database on Google Cloud. Multiplayer and online games have evolved to require increasingly complex database structures for tracking player entitlements, state, and inventory data. Growing player bases and increasing game complexity have led to database solutions that are a challenge to scale and manage, frequently requiring the use of sharding (https://wikipedia.org/wiki/Shard_(database_architecture)) or clustering (https://wikipedia.org/wiki/MySQL_Cluster). Tracking valuable in-game items or critical player progress typically requires transactions and is challenging to work around in many types of distributed databases. Spanner is the rst scalable, enterprise-grade, globally distributed, and strongly consistent database service built for the cloud to combine the benets of relational database structure with non-relational horizontal scale. Many game companies have found it to be well-suited to replace both game state and authentication databases in production-scale systems. You can scale for additional performance or storage by using the Cloud Console to add nodes. Spanner can transparently handle global replication with strong consistency, eliminating your need to manage regional replicas. This best practices document discusses the following: Important Spanner concepts and differences from databases commonly used in games.
    [Show full text]