H-STORE A High-Performance, Distributed Main Memory Transaction Processing System

Robert Kallman Andrew Pavlo Evan P. C. Jones Ryan Betts John Hugg Daniel J. Abadi Jonathan Natkins Alexander Rasin Samuel Madden Yang Zhang Bobbi Heath Hideaki Kimura Stanley Zdonik Michael McCanna

Overview System Design

Database Cluster OLLTP Application Schema Information H-Store API »»Many OLTP have common properties: »»H-Store is a new clean-slate design: Stored Sample Procedures Workload Transaction Initiator • Main-memory row storage. Node Node NodeNNoded Node • Repetitive execution of short-lived transactions. Deployment Framework Messaging Fabric NodeNNdd Node • Designed for multi-core machines. Designer Other Cluster • Entire database fits in main memory of a cluster. Query Planner/Optimizer Transactionon MManagera Execution Nodes • One execution thread per database partition. Stored Procedure Executor • Data naturally partitions on keys (e.g., customer ids, warehouse ids) Query Execution Engine Compiled Stored Proceedures System Catalogs • Individual transaction invocations only touch a small subset of data. • Distributed operation on shared-nothing clusters. Query Plans Physical Layout Main Memory Storage Manager • Applications invoke pre-defined stored Deployment Time Runtime »»The development of H-Store facilitates research into exploiting procedures consisting of structured control code System Architecture these non-trivial aspects of OLTP systems. intermixed with parameterized SQL commands. »»Replication provides data durability: »»Previous main-memory databases have Not Just Another Main • Local replication for quick fail over. focused on the migration of legacy • Remote replication for disaster recovery. features from disk-based environments. Memory RDBMS »»Uses data partitioning to distribute execution on shared-nothing, multi-core clusters »»No disk-based logging or locking OLTP Transactions »»Current prototype is 4x faster than a well- Performance »»H-Store's transaction protocol is optimized known commercial database »» Our previous work shows that a for fast single-sited transactions: specialized database engines can • Such transactions are able to execute to outperform “one-size fits all” systems: completion without retrieving intermediate data from other nodes. • A traditional row-storage RDBMS was shown to • Serial execution eliminates the need for concurrency control mechanisms. perform little useful work • An automatic database designer will aid in the deploying databases with during OLTP workload experiments. configuration that maximizes the number of single-sited transactions. »»H-Store TPC-C performance numbers: »»Complex multi-site transactions are supported, but require more • Current Prototype: 4875 txn/s heavy-handed concurrency protocols. • Commercial Database: 1207 txn/s • Tested on a 2-core Intel Xeon E5320 @ 1.86 Ghz with 10 warehouses. Main Memory Shore Performance