Benchmarking Mongodb Multi-Document Transactions in a Sharded Cluster
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Rainblock: Faster Transaction Processing in Public Blockchains
RainBlock: Faster Transaction Processing in Public Blockchains Soujanya Ponnapalli1, Aashaka Shah1, Souvik Banerjee1, Dahlia Malkhi2, Amy Tai3, Vijay Chidambaram1,3, and Michael Wei3 1University of Texas at Austin, 2Diem Association and Novi Financial, 3VMware Research Abstract Metric No state State: 10M Ratio We present RAINBLOCK, a public blockchain that achieves Time taken to mine txs (s) 1047 6340 6× " high transaction throughput without modifying the proof-of- # Txs per block 2150 833 2.5× # work consensus. The chief insight behind RAINBLOCK is that Tx throughput (txs/s) 28.6 4.7 6× # while consensus controls the rate at which new blocks are added to the blockchain, the number of transactions in each Table 1: Impact of system state on blockchain throughput. block is limited by I/O bottlenecks. Public blockchains like This table shows the throughput of Ethereum with proof-of- Ethereum keep the number of transactions in each block low work consensus when 30K txs are mined using three miners, so that all participating servers (miners) have enough time to in two scenarios: first, there are no accounts on the blockchain, process a block before the next block is created. By removing and in the second, 10M accounts have been added. Despite the I/O bottlenecks in transaction processing, RAINBLOCK al- no other difference, tx throughput is 6× lower in the second lows miners to process more transactions in the same amount scenario; we trace this to the I/O involved in processing txs. of time. RAINBLOCK makes two novel contributions: the RAIN- BLOCK architecture that removes I/O from the critical path of processing transactions (txs), and the distributed, multi- Bitcoin [1] and Ethereum, process only tens of transactions versioned DSM-TREE data structure that stores the system per second, limiting their applications [33, 65]. -
Not ACID, Not BASE, but SALT a Transaction Processing Perspective on Blockchains
Not ACID, not BASE, but SALT A Transaction Processing Perspective on Blockchains Stefan Tai, Jacob Eberhardt and Markus Klems Information Systems Engineering, Technische Universitat¨ Berlin fst, je, [email protected] Keywords: SALT, blockchain, decentralized, ACID, BASE, transaction processing Abstract: Traditional ACID transactions, typically supported by relational database management systems, emphasize database consistency. BASE provides a model that trades some consistency for availability, and is typically favored by cloud systems and NoSQL data stores. With the increasing popularity of blockchain technology, another alternative to both ACID and BASE is introduced: SALT. In this keynote paper, we present SALT as a model to explain blockchains and their use in application architecture. We take both, a transaction and a transaction processing systems perspective on the SALT model. From a transactions perspective, SALT is about Sequential, Agreed-on, Ledgered, and Tamper-resistant transaction processing. From a systems perspec- tive, SALT is about decentralized transaction processing systems being Symmetric, Admin-free, Ledgered and Time-consensual. We discuss the importance of these dual perspectives, both, when comparing SALT with ACID and BASE, and when engineering blockchain-based applications. We expect the next-generation of decentralized transactional applications to leverage combinations of all three transaction models. 1 INTRODUCTION against. Using the admittedly contrived acronym of SALT, we characterize blockchain-based transactions There is a common belief that blockchains have the – from a transactions perspective – as Sequential, potential to fundamentally disrupt entire industries. Agreed, Ledgered, and Tamper-resistant, and – from Whether we are talking about financial services, the a systems perspective – as Symmetric, Admin-free, sharing economy, the Internet of Things, or future en- Ledgered, and Time-consensual. -
What Is a Database Management System? What Is a Transaction
What is a Database? Collection of data central to some enterprise Overview of Databases and Essential to operation of enterprise Transaction Processing Contains the only record of enterprise activity An asset in its own right Chapter 1 Historical data can guide enterprise strategy Of interest to other enterprises State of database mirrors state of enterprise Database is persistent 2 What is a Database Management What is a Transaction? System? When an event in the real world changes the A Database Management System (DBMS) state of the enterprise, a transaction is is a program that manages a database: executed to cause the corresponding change in the database state Supports a high-level access language (e.g. With an on-line database, the event causes the SQL). transaction to be executed in real time Application describes database accesses using A transaction is an application program that language. with special properties - discussed later - to DBMS interprets statements of language to guarantee it maintains database correctness perform requested database access. 3 4 What is a Transaction Processing Transaction Processing System System? Transaction execution is controlled by a TP monitor s DBMS database n o i Creates the abstraction of a transaction, t c a analogous to the way an operating system s n a r creates the abstraction of a process t DBMS database TP monitor and DBMS together guarantee the special properties of transactions TP Monitor A Transaction Processing System consists of TP monitor, databases, and transactions 5 6 1 System -
Damage Management in Database Management Systems
The Pennsylvania State University The Graduate School Department of Information Sciences and Technology Damage Management in Database Management Systems A Dissertation in Information Sciences and Technology by Kun Bai °c 2010 Kun Bai Submitted in Partial Ful¯llment of the Requirements for the Degree of Doctor of Philosophy May 2010 The dissertation of Kun Bai was reviewed and approved1 by the following: Peng Liu Associate Professor of Information Sciences and Technology Dissertation Adviser Chair of Committee Chao-Hsien Chu Professor of Information Sciences and Technology Thomas La Porta Distinguished Professor of Computer Science and Engineering Sencun Zhu Assistant Professor of Computer Science and Engineering Frederico Fonseca Associate Professor of Information Sciences and Technology Associate Dean, College of Information Sciences and Technology 1Signatures on ¯le in the Graduate School. iii Abstract In the past two decades there have been many advances in the ¯eld of computer security. However, since vulnerabilities cannot be completely removed from a system, successful attacks often occur and cause damage to the system. Despite numerous tech- nological advances in both security software and hardware, there are many challenging problems that still limit e®ectiveness and practicality of existing security measures. As Web applications gain popularity in today's world, surviving Database Man- agement System (DBMS) from an attack is becoming even more crucial than before because of the increasingly critical role that DBMS is playing in business/life/mission- critical applications. Although signi¯cant progress has been achieved to protect the DBMS, such as the existing database security techniques (e.g., access control, integrity constraint and failure recovery, etc.,), the buniness/life/mission-critical applications still can be hit due to some new threats towards the back-end DBMS. -
ACID, Transactions
ECE 650 Systems Programming & Engineering Spring 2018 Database Transaction Processing Tyler Bletsch Duke University Slides are adapted from Brian Rogers (Duke) Transaction Processing Systems • Systems with large DB’s; many concurrent users – As a result, many concurrent database transactions – E.g. Reservation systems, banking, credit card processing, stock markets, supermarket checkout • Need high availability and fast response time • Concepts – Concurrency control and recovery – Transactions and transaction processing – ACID properties (desirable for transactions) – Schedules of transactions and recoverability – Serializability – Transactions in SQL 2 Single-User vs. Multi-User • DBMS can be single-user or multi-user – How many users can use the system concurrently? – Most DBMSs are multi-user (e.g. airline reservation system) • Recall our concurrency lectures (similar issues here) – Multiprogramming – Interleaved execution of multiple processes – Parallel processing (if multiple processor cores or HW threads) A A B B C CPU1 D CPU2 t1 t2 t3 t4 time Interleaved concurrency is model we will assume 3 Transactions • Transaction is logical unit of database processing – Contains ≥ 1 access operation – Operations: insertion, deletion, modification, retrieval • E.g. things that happen as part of the queries we’ve learned • Specifying database operations of a transaction: – Can be embedded in an application program – Can be specified interactively via a query language like SQL – May mark transaction boundaries by enclosing operations with: • “begin transaction” and “end transaction” • Read-only transaction: – No database update operations; only retrieval operations 4 Database Model for Transactions • Database represented as collection of named data items – Size of data item is its “granularity” – E.g. May be field of a record (row) in a database – E.g. -
Transaction Processing Monitors
Chapter 24: Advanced Transaction Processing ! Transaction-Processing Monitors ! Transactional Workflows ! High-Performance Transaction Systems ! Main memory databases ! Real-Time Transaction Systems ! Long-Duration Transactions ! Transaction management in multidatabase systems 1 Database System Concepts 24.1 ©Silberschatz, Korth and Sudarshan Transaction Processing Monitors ! TP monitors initially developed as multithreaded servers to support large numbers of terminals from a single process. ! Provide infrastructure for building and administering complex transaction processing systems with a large number of clients and multiple servers. ! Provide services such as: ! Presentation facilities to simplify creating user interfaces ! Persistent queuing of client requests and server responses ! Routing of client messages to servers ! Coordination of two-phase commit when transactions access multiple servers. ! Some commercial TP monitors: CICS from IBM, Pathway from Tandem, Top End from NCR, and Encina from Transarc 2 Database System Concepts 24.2 ©Silberschatz, Korth and Sudarshan 1 TP Monitor Architectures 3 Database System Concepts 24.3 ©Silberschatz, Korth and Sudarshan TP Monitor Architectures (Cont.) ! Process per client model - instead of individual login session per terminal, server process communicates with the terminal, handles authentication, and executes actions. ! Memory requirements are high ! Multitasking- high CPU overhead for context switching between processes ! Single process model - all remote terminals connect to a single server process. ! Used in client-server environments ! Server process is multi-threaded; low cost for thread switching ! No protection between applications ! Not suited for parallel or distributed databases 4 Database System Concepts 24.4 ©Silberschatz, Korth and Sudarshan 2 TP Monitor Architectures (Cont.) ! Many-server single-router model - multiple application server processes access a common database; clients communicate with the application through a single communication process that routes requests. -
High-Performance Transaction Processing in SAP HANA
High-Performance Transaction Processing in SAP HANA Juchang Lee1, Michael Muehle1, Norman May1, Franz Faerber1, Vishal Sikka1, Hasso Plattner2, Jens Krueger2, Martin Grund3 1SAP AG 2Hasso Plattner Insitute, Potsdam, Germany, 3eXascale Infolab, University of Fribourg, Switzerland Abstract Modern enterprise applications are currently undergoing a complete paradigm shift away from tradi- tional transactional processing to combined analytical and transactional processing. This challenge of combining two opposing query types in a single database management system results in additional re- quirements for transaction management as well. In this paper, we discuss our approach to achieve high throughput for transactional query processing while allowing concurrent analytical queries. We present our approach to distributed snapshot isolation and optimized two-phase commit protocols. 1 Introduction An efficient and holistic data management infrastructure is one of the key requirements for making the right deci- sions at an operational, tactical, and strategic level and is core to support all kinds of enterprise applications[12]. In contrast to traditional architectures of database systems, the SAP HANA database takes a different approach to provide support for a wide range of data management tasks. The system is organized in a main-memory centric fashion to reflect the shift within the memory hierarchy[2] and to consistently provide high perfor- mance without prohibitively slow disk interactions. Completely transparent for the application, data is orga- nized along its life cycle either in column or row format, providing the best performance for different workload characteristics[11, 1]. Transactional workloads with a high update rate and point queries can be routed against a row store; analytical workloads with range scans over large datasets are supported by column oriented data struc- tures. -
Cohesity Dataplatform Protecting Individual MS SQL Databases Solution Guide
Cohesity DataPlatform Protecting Individual MS SQL Databases Solution Guide Abstract This solution guide outlines the workflow for creating backups with Microsoft SQL Server databases and Cohesity Data Platform. Table of Contents About this Guide..................................................................................................................................................................2 Intended Audience..............................................................................................................................................2 Configuration Overview.....................................................................................................................................................2 Feature Overview.................................................................................................................................................................2 Installing Cohesity Windows Agent..............................................................................................................................2 Downloading Cohesity Agent.........................................................................................................................2 Select Coheisty Windows Agent Type.........................................................................................................3 Install the Cohesity Agent.................................................................................................................................3 Cohesity Agent Setup........................................................................................................................................4 -
Django-Transaction-Hooks Documentation Release 0.2.1.Dev1
django-transaction-hooks Documentation Release 0.2.1.dev1 Carl Meyer December 12, 2016 Contents 1 Prerequisites 3 2 Installation 5 3 Setup 7 3.1 Using the mixin.............................................7 4 Usage 9 4.1 Notes...................................................9 5 Contributing 13 i ii django-transaction-hooks Documentation, Release 0.2.1.dev1 A better alternative to the transaction signals Django will never have. Sometimes you need to fire off an action related to the current database transaction, but only if the transaction success- fully commits. Examples: a Celery task, an email notification, or a cache invalidation. Doing this correctly while accounting for savepoints that might be individually rolled back, closed/dropped connec- tions, and idiosyncrasies of various databases, is non-trivial. Transaction signals just make it easier to do it wrong. django-transaction-hooks does the heavy lifting so you don’t have to. Contents 1 django-transaction-hooks Documentation, Release 0.2.1.dev1 2 Contents CHAPTER 1 Prerequisites django-transaction-hooks supports Django 1.6.x through 1.8.x on Python 2.6, 2.7, 3.2, 3.3 and 3.4. django-transaction-hooks has been merged into Django 1.9 and is now a built-in feature, so this third-party library should not be used with Django 1.9+. SQLite3, PostgreSQL (+ PostGIS), and MySQL are currently the only databases with built-in support; you can exper- iment with whether it works for your favorite database backend with just a few lines of code. 3 django-transaction-hooks Documentation, Release 0.2.1.dev1 4 Chapter 1. -
Transactions.Pdf
BIT 4514: Database Technology for Business Fall 2019 Database transactions 1 1 Database transactions • A database transaction is any (possibly multi-step) action that reads from and/or writes to a database – It may consist of a single SQL statement or a collection of related SQL statements ex: Adding a new lunch to the class database – requires two related INSERT statements 2 2 Transactions (cont.) • A successful transaction is one in which all of the SQL statements are completed successfully – A consistent database state is one in which all data integrity constraints are satisfied – A successful transaction changes the database from one consistent state to another 3 3 1 Transaction management • Improper or incomplete transactions can have a devastating effect on database integrity Ex: INSERT only items into the Lunch_item table • If a DBMS supports transaction management, it will roll back an inconsistent database (i.e., the result of an unsuccessful transaction) to a previous consistent state. 4 4 Properties of a transaction • Atomicity • Consistency • Isolation • Durability • Every transaction MUST exhibit these four properties 5 5 Properties of a transaction • Atomicity – The "all or nothing" property – All transaction operations must be completed i.e. a transaction is treated as a single, indivisible, logical unit of work • Consistency – When a transaction is completed, the database must be in a consistent state 6 6 2 Properties of a transaction • Isolation – Data used during the execution of a transaction cannot be used by a second -
A Transaction Processing Method for Distributed Database
Advances in Computer Science Research, volume 87 3rd International Conference on Mechatronics Engineering and Information Technology (ICMEIT 2019) A Transaction Processing Method for Distributed Database Zhian Lin a, Chi Zhang b School of Computer and Cyberspace Security, Communication University of China, Beijing, China [email protected], [email protected] Abstract. This paper introduces the distributed transaction processing model and two-phase commit protocol, and analyses the shortcomings of the two-phase commit protocol. And then we proposed a new distributed transaction processing method which adds heartbeat mechanism into the two- phase commit protocol. Using the method can improve reliability and reduce blocking in distributed transaction processing. Keywords: distributed transaction, two-phase commit protocol, heartbeat mechanism. 1. Introduction Most database services of application systems will be distributed on several servers, especially in some large-scale systems. Distributed transaction processing will be involved in the execution of business logic. At present, two-phase commit protocol is one of the methods to distributed transaction processing in distributed database systems. The two-phase commit protocol includes coordinator (transaction manager) and several participants (databases). In the process of communication between the coordinator and the participants, if the participants without reply for fail, the coordinator can only wait all the time, which can easily cause system blocking. In this paper, heartbeat mechanism is introduced to monitor participants, which avoid the risk of blocking of two-phase commit protocol, and improve the reliability and efficiency of distributed database system. 2. Distributed Transactions 2.1 Distributed Transaction Processing Model In a distributed system, each node is physically independent and they communicates and coordinates each other through the network. -
Database Systems 09 Transaction Processing
1 SCIENCE PASSION TECHNOLOGY Database Systems 09 Transaction Processing Matthias Boehm Graz University of Technology, Austria Computer Science and Biomedical Engineering Institute of Interactive Systems and Data Science BMVIT endowed chair for Data Management Last update: May 13, 2019 2 Announcements/Org . #1 Video Recording . Since lecture 03, video/audio recording . Link in TeachCenter & TUbe . #2 Exercises . Exercise 1 graded, feedback in TC in next days 77.4% . Exercise 2 still open until May 14 11.50pm (incl. 7 late days, no submission is a mistake) 53.7% . Exercise 3 published and introduced today . #3 CS Talks x4 (Jun 17 2019, 5pm, Aula Alte Technik) . Claudia Wagner (University Koblenz‐Landau, Leibnitz Institute for the Social Sciences) . Title: Minorities in Social and Information Networks . Dinner opportunity for interested female students! INF.01014UF Databases / 706.004 Databases 1 – 09 Transaction Processing Matthias Boehm, Graz University of Technology, SS 2019 3 Announcements/Org, cont. #4 Infineon Summer School 2019 Sensor Systems . Where: Infineon Technologies Austria, Villach Carinthia, Austria . Who: BSc, MSc, PhD students from different fields including business informatics, computer science, and electrical engineering . When: Aug 26 through 30, 2019 . Application deadline: Jun 16, 2019 . #5 Poll: Date of Final Exam . We’ll move Exercise 4 to Jun 25 . Current date: Jun 24, 6pm . Alternatives: Jun 27, 4pm / 7.30pm, or week starting Jul 8 (Erasmus?) INF.01014UF Databases / 706.004 Databases 1 – 09 Transaction Processing Matthias Boehm, Graz University of Technology, SS 2019 4 Transaction (TX) Processing User 2 User 1 User 3 read/write TXs #1 Multiple users Correctness? DBS DBMS #2 Various failures Deadlocks (TX, system, media) Constraint Reliability? violations DBs Network Crash/power failure Disk failure failure .