Building Cloud Native GraphQL Applications on Kubernetes with YugabyteDB and Hasura
Nikhil Chandrappa Ecosystem Engineering
© 2020 All Rights Reserved Product Marketer RudderStack New Relic AT&T Tech Consultant Deloitte (MBA Intern -> Senior Consultant -> Manager) Sys Admin Gavin Johnson Developer Senior Product Marketing Manager, Yugabyte Cloud, Yugabyte
2 © 2020 All Rights Reserved Data Engineer Pivotal (Acq. VMware) Systems Engineer Infosys
Nikhil Chandrappa Lead Software Engineer, Ecosystem Engineering, Yugabyte
3 © 2020 All Rights Reserved Kubernetes is massively popular in Fortune 500s
● Walmart - Edge Computing KubeCon 2019 https://www.youtube.com/watch?v=sfPFrvDvdlk
● Target - Data @ Edge https://tech.target.com/2018/08/08/running-cassandra-in-kubern etes-across-1800-stores.html
● eBay - Platform Modernization https://www.ebayinc.com/stories/news/ebay-builds-own-servers-i ntends-to-open-source/
4 © 2020 All Rights Reserved Why stateful services on K8s?
Containerized data workloads running on Kubernetes offer several advantages over traditional VM / bare metal based data workloads including but not limited to:
● Better cluster resource utilization
● Portability between cloud and on-premises
● Frictionless multi-tenancy with versioning
● Simple and selective instant upgrades
● Robust automation framework can be embedded inside CRDs (Custom Resource Definitions) or commonly referred as ‘K8S Operator’.
5 © 2020 All Rights Reserved Getting started with GraphQL
● Provides GraphQL abstractions over your Database App
a. Query, mutate data via GraphQL constructs
b. Build your schema and evolve your domain models
● Evolve your API without versioning
● Provides out of the box pagination and filtering GraphQL ● Collate disparate Data sources Server
● Eventing Support
Database
6 © 2020 All Rights Reserved Cloud native relational database for cloud native applications
SQL PostgreSQL Resilience and Compatibility High Availability
Distributed SQL database for Horizontal Geographic transactional applications. Scalability Distribution 100% open source. Runs on any cloud.
ACID Security Transactions
7 © 2020 All Rights Reserved YugabyteDB and Hasura on K8S
8 © 2020 All Rights Reserved Design decisions for Cloud Native Databases
Cloud Native ● Scale on-demand ● API driven admin operations ● Prometheus based metrics Resiliency ● Against cloud infrastructure failures ● Automated and predictable data recovery Day 2 Operations ● Zero downtime upgrades ● Automated backup/recovery
9 © 2020 All Rights Reserved YugabyteDB deployed as StatefulSets
Admin App Clients yb-masters Clients yb-tservers Headless Service Headless Service
yb-master-0 pod yb-master-1 pod yb-master-2 pod yb-master StatefulSet yugabytedb yugabytedb yugabytedb
yb-tserver-0tablet 1’ pod yb-tserver-1tablet 1’ pod yb-tserver-2tablet 1’ pod yb-tserver-3tablet 1’ pod yb-tserver … StatefulSet yugabytedb yugabytedb yugabytedb yugabytedb
node1 node2 node3 node4
Local/Remote Local/Remote Local/Remote Local/Remote Persistent Volume Persistent Volume Persistent Volume Persistent Volume 10 © 2020 All Rights Reserved Automating day 2 operations
HANDLING FAILURES ROLLING UPGRADES BACKUP & RESTORE
Supports two upgradeStrategies: Backups and restores are a Pod failure handled by K8S onDelete (default) and database level construct automatically rollingUpgrade
YugaByteDB can perform Node failure has to be handled Pick rolling upgrade strategy for distributed snapshot and copy manually by adding a new slave DBs that support zero downtime to a target for a backup node to K8S cluster upgrades such as YugaByteDB
Restore the backup into an Local storage failure has to be New instance of the pod existing cluster or a new cluster handled manually by mounting spawned with same network id with a different number of new local volume to K8S and storage TServers
11 © 2020 All Rights Reserved Hasura connectivity with YugabyteDB on Kubernetes
Can connect to ANY node HASURA Add/remove nodes anytime
YugabyteDB YugabyteDB YugabyteDB Postgres Postgres Postgres Query Layer Query Layer Query Layer … … DocDB DocDB DocDB Storage Layer Storage Layer Storage Layer
YugabyteDB Node YugabyteDB Node YugabyteDB Node
12 © 2020 All Rights Reserved Building Real Time Poll App Using Hasura and Yugabyte on Kubernetes
13 © 2020 All Rights Reserved Scaling out GraphQL services
r YugabyteDB Cluster
HASURA HASURA HASURA HASURA …
…
App App App App
14 © 2020 All Rights Reserved Try YugabyteDB
15 © 2020 All Rights Reserved Distributed SQL database built for cloud native applications
SQL PostgreSQL Resilience and Compatibility High Availability
Distributed SQL database for Horizontal Geographic transactional applications. Scalability Distribution 100% open source. Runs on any cloud.
ACID Security Transactions
16 © 2020 All Rights Reserved Postgres reimagined for a cloud native world
Google PostgreSQL Spanner YugabyteDB
✓ ✘ ✓ SQL Ecosystem Massively Yugabyte Query Layer New SQL flavor Reuse PostgreSQL adopted YSQL YCQL ✓ ✘ ✓ RDBMS Features Advanced Basic Advanced Complex cloud-native Complex and cloud-native DocDB Distributed Document Store Highly Available ✘ ✓ ✓ Distributed Sharding & Load Raft Consensus Transaction Manager Balancing Replication Horizontal Scale ✘ ✓ ✓ & MVCC
Distributed Txns ✘ ✓ ✓ Document Storage Layer
Custom RocksDB Storage Engine Data Replication Async Sync Sync + Async
17 © 2020 All Rights Reserved Target use cases
Systems of Record Event Data and IoT Geo-Distributed and Engagement Workloads
Resilient, business critical data Handling massive scale Sync, async, and geo replication
● Identity management ● Vehicle telemetry ● Vehicle telemetry ● User/account profile ● Stock bids and asks ● Stock bids and asks ● eCommerce apps - checkout, ● Shipment information ● Shipment information shopping cart ● Credit card transactions ● Credit card transactions ● Real time payment systems
18 © 2020 All Rights Reserved Yugabyte offers flexible consumption models
Private Self-Managed DBaaS Public Fully Managed DBaaS
Infrastructure Intelligent Consolidated 24/7 Support Infrastructure Intelligent Consolidated 24/7 Support Operations Day 2 Ops Management & Advanced Operations Day 2 Ops Management Across Technology Multiple Self-Managed Distributed SQL DB Services Channels
YugabyteDB YugabyteDB YugabyteDB
100% Open Source 100% Open Source 100% Open Source
Customer Infrastructure Customer Infrastructure Yugabyte Infrastructure
vm vm vm vm vm vm vm vm vm vm vm vm
YugabyteDB Yugabyte Platform Yugabyte Cloud
● Self managed ● Self managed (or with Yugabyte Services) ● Fully managed by Yugabyte ● Manual infrastructure operations ● Automated infrastructure operations ● Consolidated management console ● Manual database day 2 operations ● Automated database day 2 operations ● Automated infrastructure operations ● Community support ● 24/7 support available ● Automated database day 2 operations ● Advanced technology services ● 24/7 support
19 © 2020 All Rights Reserved Thank You
Join us on Slack: www.yugabyte.com/slack
Star us on GitHub: github.com/yugabyte/yugabyte-db
20 © 2020 All Rights Reserved