MILLIONS of TRANSACTIONS PER SECOND on a SINGLE MACHINE CASE for a VIRTUALIZED DATABASE and SCALE-IN ROGER JOHANSSON Who Am I?
Total Page：16
File Type：pdf, Size：1020Kb
MILLIONS OF TRANSACTIONS PER SECOND ON A SINGLE MACHINE CASE FOR A VIRTUALIZED DATABASE AND SCALE-IN ROGER JOHANSSON Who am I? Roger Johansson Actor Model, Scalability, Senior Solution Architect – Starcounter Distributed Systems, C#, Go, Kotlin http://StarCounter.io Proto.Actor Founder – Ultra Fast Distributed Actors (Go, .NET, JVM) http://Proto.Actor https://github.com/AsynkronIT Twitter: @rogeralsing Github: rogeralsing Mail: [email protected] http://Github.com/rogeralsing/presentations Agenda Setting the stage - Why we need in memory computing: • Application Platform • Micro-Apps The Starcounter information operating system: • Our approach to in memory computing • The future of hardware • The Mars project Front-end Framework React, Polymer Communication Palindrom - REST, Web Sockets Application View Models, Entities, App Logic In Memory App Platform Mapping, Persistence, Queries Starcounter Front-end Framework Front-end Framework Client Side Code Network Network Services + Contracts View Models Application Code Application O/R Mapper Network Database In Memory App Platform Traditional Stack Starcounter UI A UI B UI C App A App B App C Model A Model B Model C Starcounter Client Side Blending UI B UI A UI C UI A UI B UI C App A App B App C Model A Model B Model C Starcounter Profile ObjectId 789 FirstName Stefan LastName Edqvist User ObjectId 123 FirstName Stefan LastName Edqvist Client Side Blending UI B UI A UI C UI A UI B UI C App A App B App C Model A Model B Model C ACID, Snapshot Mapper Isolation Starcounter UI A UI B UI C App A App B App C Model A Model B Model C Mapper Starcounter Demo Dataflow Application Process Object Offline copy of ObjectId 123 database data FirstName Alan LastName Turing Age 41 Map to and from Object Mapper Object / Storage model Network / IPC Transfer Data over Disk network ObjectId FirstName LastName Age Data is stored on disk, optimized for 123 Alan Turing 41 spinning disks and 124 Ada Lovelace 36 moving arms 125 Grace Hopper 86 Database Process Application Process Object Offline copy of ObjectId 123 database data FirstName Alan LastName Turing Age 41 Map to and from Object Mapper Object / Relational model Network / IPC Transfer Data over Memory network ObjectId FirstName LastName Age Data is mainly kept in memory, 123 Alan Turing 41 enabling faster 124 Ada Lovelace 36 reads and writes 125 Grace Hopper 86 Database Process Application Process Object Stateless Entity ObjectId 123 maps directly to shared memory Shared Memory ObjectId FirstName LastName Age Data is kept in memory and 123 Alan Turing 41 shared across 124 Ada Lovelace 36 processes 125 Grace Hopper 86 Database Process C# Node.JS Object Object Multiple processes ObjectId 123 ObjectId 123 can read the same data Shared Memory ObjectId FirstName LastName Age 123 Alan Turing 41 124 Ada Lovelace 36 125 Grace Hopper 86 Database Process C# Source Code public class Person { public string FirstName { get; set; } C# Compiler Output public class Person { private string _firstName; public string FirstName { get => _firstName; set value => _firstName = value; } Starcounter Weaver Output public class Person { private string _firstName; private ulong _id; public string FirstName { get => DbState.GetString(_id, FirstNameId); set value => DbState.SetString(_id, FirstNameId, value) } O/R mapper – Object Graphs Customer Order OrderDetail Product Lazy Load Ripple load, Execute SQL Query per traversed relation Hundreds/Thousands of queries per view Eager Load Hard to maintain, Carefully defining load boundaries inefficient joined queries Starcounter – Object Graphs Customer Order OrderDetail Product Chasing Pointers Traversing relations are simple pointer traversals Demo - Write 1 mil transactions Demo - Write 1 mil transactions In this demo we provided: • ACID Transactions • Snapshot Isolation • Guaranteed disk persistency Can we go even faster? • We are capped by disk IO • The Mars project Price of 1MB in USD over time 1000,00000 100,00000 10,00000 1,00000 0,10000 USD 0,01000 0,00100 0,00010 0,00001 Year 1980 1985 1990 1995 2000 2005 2010 2015 2020 Registers Registers Cache Cache Main Memory Main Memory Solid State Disk Non Volatile Memory Magnetic Disk Solid State Disk Magnetic Disk Current Future Starcounter on the CPUs of 2018 Write Volatile DRAM Delta Store Read (incremental updates) Merge CPU Main Store (batch update) Non Volatile DRAM Few Big Servers, not many small Server Server Few Big Servers, not many small Server Server .