DOCSLIB.ORG

Design and Implementation of a Blockchain Relational Database∗

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Design and Implementation of a Blockchain Relational Database∗ Blockchain Meets Database: Design and Implementation of a Blockchain Relational Database∗ Senthil Nathan1, Chander Govindarajan1, Adarsh Saraf1, Manish Sethi2, and Praveen Jayachandran1 1IBM Research India, 2IBM Industry Platforms USA 1(snatara7, chandg12, adasaraf, praveen.j)@in.ibm.com, [email protected] ABSTRACT transactional processing capabilities already built into rela- In this paper, we design and implement the first-ever de- tional databases over decades of research and development, centralized replicated relational database with blockchain we demonstrate that we can build a blockchain relational properties that we term blockchain relational database. We database with all features provided by existing blockchain highlight several similarities between features provided by platforms and with better support for complex data types, blockchain platforms and a replicated relational database, constraints, triggers, complex queries, and provenance queries. although they are conceptually different, primarily in their Furthermore, our approach makes building blockchain appli- trust model. Motivated by this, we leverage the rich fea- cations as easy as building database applications; developers tures, decades of research and optimization, and available can leverage the rich tooling available for backup, analytics, tooling in relational databases to build a blockchain rela- and integration with existing enterprise systems. Applica- tional database. We consider a permissioned blockchain tions that have a strong compliance and audit requirements model of known, but mutually distrustful organizations each and need for rich analytical queries such as in financial ser- operating their own database instance that are replicas of vices that are already built atop relational databases, or ones one another. The replicas execute transactions indepen- that are reliant on rich provenance information such as in dently and engage in decentralized consensus to determine supply chains are likely to most benefit from the blockchain the commit order for transactions. We design two approaches, relational database proposed in this paper. the first where the commit order for transactions is agreed With a focus on enterprise blockchain applications, we upon prior to executing them, and the second where trans- target a permissioned setup of known but untrusted orga- actions are executed without prior knowledge of the commit nizations each operating their own independent database order while the ordering happens in parallel. We leverage nodes but connected together to form a blockchain network. serializable snapshot isolation (SSI) to guarantee that the These database nodes need to be replicas of each other but replicas across nodes remain consistent and respect the or- distrust one another. The replication of transaction logs dering determined by consensus, and devise a new variant and state among multiple database nodes is possible us- of SSI based on block height for the latter approach. We ing master-slave [16, 25, 45, 50] and master-master [25, implement our system on PostgreSQL and present detailed 48, 50] protocols. In these protocols, the master node is performance experiments analyzing both approaches. trusted as it is the only node that executes transactions and propagates the final updates to other masters and slaves ei- 1. INTRODUCTION ther synchronously or asynchronously [28]. For synchronous replication in the Master-Master architecture, the serializ- Blockchain has gained immense popularity over recent able order is decided using techniques such as 2-phase com- years, with its application being actively explored in several mit [38], 3-phase commit [20], a quorum protocol [44] to industries. At its core, it is an immutable append-only log ensure consistency. Asynchronous replication can result in arXiv:1903.01919v2 [cs.DC] 31 May 2019 of cryptographically signed transactions, that is replicated an intermediate inconsistent state between masters. Various and managed in a decentralized fashion through distributed solutions [42, 46] exist to reconcile inconsistency. However, consensus among a set of untrusted parties. Opinion on in a blockchain network, one node cannot trust the execu- the technology has varied widely from being hailed as the tion results of another node. Hence, all transactions need biggest innovation since the Internet to being considered as to be independently executed on all nodes. Further, such another database in disguise. For the first time, we under- an execution and subsequent commits across multiple un- take the challenge of explaining blockchain technology from trusted nodes must result in the same serializable order to the perspective of concepts well known in databases, and ensure consistency. As a result, existing replication proto- highlight the similarities and differences between the two. cols cannot be used for blockchain networks and hence, we Existing blockchain platforms [17, 51, 39, 14], in an at- need a novel protocol with a notion of decentralized trust. tempt to build something radically new and transforma- Although crash fault-tolerant synchronous commit proto- tive, rebuild a lot of features provided by databases, and cols [29, 49] exist, it involves multiple rounds of commu- treat it as just a store of information. We take a contrar- nications between database nodes per transaction or a cen- ian view in this paper. By leveraging the rich features and tralized trusted controller. However, applying synchronous ∗Copyright held by the owner/author(s) commit per transaction on a blockchain network would be 1 very costly as nodes may be globally distributed. Further, (1) Smart contract & transaction: Many blockchain a trusted controller cannot be used. platforms such as Hyperledger Fabric [17], Ethereum [51] In this paper, we address this key challenge of ensuring and Corda [30] support smart contracts which are functions that all the untrusted database nodes execute transactions that manage state on the blockchain ledger. Transactions independently and commit them in the same serializable or- are invocations of smart contract functions. Independent der asynchronously. Towards achieving this, we make two execution of the smart contract functions by different mutu- key design choices. First, we modify the database to sep- ally distrustful peers need to be deterministic. This is sim- arate the concerns of concurrent transaction execution and ilar to stored PL/SQL procedures in relational databases. decentralized ordering of blocks of transactions. We leverage Both accept input arguments, retrieve data by executing ordering through consensus only to order blocks of transac- queries, apply the predefined logic, and write data back to tions, and not for the serializable ordering of transactions the database. However, the execution of a PL/SQL proce- within a single block. Second, independently at each node, dure may not be deterministic when executed independently we leverage serializable snapshot isolation (SSI) [23, 43] to on multiple nodes if utility functions such as random and execute transactions concurrently and then serially validate timestamp are used. Hence, we need to constrain PL/SQL & commit each transaction in a block. to obtain a serializ- procedures to ensure that the execution is deterministic. able order that will be the same across all untrusted nodes (2) User authenticity & non-repudiability: Permis- and respect the ordering of blocks obtained from consensus. sioned blockchain systems employ public key infrastructure Using these ideas, we present two novel approaches, the first for user management and ensuring authenticity. Participat- where block ordering is performed before transaction execu- ing users belong to organizations are permissioned to par- tion, and the second where transaction execution happens ticipate in the blockchain network. Transactions are digi- parallelly without prior knowledge of block ordering, and tally signed by the invoker and recorded on the blockchain, discuss their trade-offs. We leverage and modify serializable making them non-repudiable. Relational databases have so- snapshot isolation [43] to create a novel variant which guar- phisticated user management capabilities including support antees the same serializable ordering across all nodes. We for users, roles, groups, and various user authentication op- implement our ideas on PostgreSQL and discuss several sub- tions. However, submitted and recorded transactions are tle challenges we had to overcome. We make the following not signed by the invoker making them repudiable. Hence, contributions in this paper: we need invoker’s signature on transaction. (3) Confidentiality & access control: Some permis- 1. We devise two novel approaches of building a blockchain sioned blockchains support the notion of confidentiality, where platform starting from a database, ensuring that a net- smart contracts, transactions and data are only accessible work of untrusted database nodes can remain consis- by authorized users. In addition, access to invoke functions tent. and to modify data is restricted to specific users. Relational 2. We devise a new variant of SSI based on block height databases have comprehensive support for access control on to permit transaction execution to happen parallelly tables, rows, columns and PL/SQL procedures. Some re- with ordering based on consensus. lational databases even provide features like content-based access control, encryption and isolation of data. Advanced 3. We implement the system in
Load more

Recommended publications
  • SQL Server Protection Whitepaper
    SQL Server Protection Whitepaper Contents 1. Introduction ..................................................................................................................................... 2 Documentation .................................................................................................................................................................. 2 Licensing ............................................................................................................................................................................... 2 The benefits of using the SQL Server Add-on ....................................................................................................... 2 Requirements ...................................................................................................................................................................... 2 2. SQL Protection overview ................................................................................................................ 3 User databases ................................................................................................................................................................... 3 System databases .............................................................................................................................................................. 4 Transaction logs ................................................................................................................................................................
    [Show full text]
  • How to Conduct Transaction Log Analysis for Web Searching And
    Search Log Analysis: What is it; what’s been done; how to do it Bernard J. Jansen School of Information Sciences and Technology The Pennsylvania State University 329F IST Building University Park, Pennsylvania 16802 Email: [email protected] Abstract The use of data stored in transaction logs of Web search engines, Intranets, and Web sites can provide valuable insight into understanding the information-searching process of online searchers. This understanding can enlighten information system design, interface development, and devising the information architecture for content collections. This article presents a review and foundation for conducting Web search transaction log analysis. A methodology is outlined consisting of three stages, which are collection, preparation, and analysis. The three stages of the methodology are presented in detail with discussions of goals, metrics, and processes at each stage. Critical terms in transaction log analysis for Web searching are defined. The strengths and limitations of transaction log analysis as a research method are presented. An application to log client-side interactions that supplements transaction logs is reported on, and the application is made available for use by the research community. Suggestions are provided on ways to leverage the strengths of, while addressing the limitations of, transaction log analysis for Web searching research. Finally, a complete flat text transaction log from a commercial search engine is available as supplementary material with this manuscript. Introduction Researchers have used transaction logs for analyzing a variety of Web systems (Croft, Cook, & Wilder, 1995; Jansen, Spink, & Saracevic, 2000; Jones, Cunningham, & McNab, 1998; Wang, 1 of 42 Berry, & Yang, 2003). Web search engine companies use transaction logs (also referred to as search logs) to research searching trends and effects of system improvements (c.f., Google at http://www.google.com/press/zeitgeist.html or Yahoo! at http://buzz.yahoo.com/buzz_log/?fr=fp- buzz-morebuzz).
    [Show full text]
  • Destiny® System Backups
    Destiny® system backups Establishing a backup and restore plan for Destiny Overview It is important to establish a backup and restore plan for your Destiny installation. The plan must be validated and monitored to ensure that your data is sufficiently backed up and can be recovered in the event of hardware failure or other disaster. IMPORTANT Follett recommends deploying a comprehensive backup solution and testing and monitoring all backup processes. There are tradeoff decisions to be made in the backup strategy that will be shaped by your organization’s risk tolerance, technology constraints, and ability to absorb data loss or downtime. This document provides an overview of standard backup and restore for the Destiny system. IMPORTANT This content does not cover high-availability configurations such as clustered servers or log shipping. Please contact Follett School Solutions, Inc. Technical Support should you have any questions about backing up your Destiny installation. For details, see Destiny® system backups. Backing up Destiny: an overview SQL database The heart of the Destiny data resides in the SQL database. These are the main components of SQL data to be backed up: The Destiny SQL database. This database contains the main Destiny data. Proper SQL backup configuration of this database is essential. NOTE If your installation is a consortium, you will have to ensure a proper backup of multiple SQL databases (one per member). The Destiny SQL transaction log. The Master SQL database. This system database is useful when restoring to a replacement server. It is not necessary to back up the tempdb database. This is a SQL Server work area and is recreated automatically.
    [Show full text]
  • Infosphere MDM Collaboration Server: Installation Guide
    IBM InfoSphere Master Data Management Collaboration Server Version 11.6 Fix Pack 15 Installation Guide IBM Note Before using this information and the product that it supports, read the information in “Notices” on page 159. Edition Notice This edition applies to version 11.6 of IBM® InfoSphere® Master Data Management and to all subsequent releases and modifications until otherwise indicated in new editions. © Copyright International Business Machines Corporation 2000, 2020. US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Chapter 1. Planning to install.................................................................................1 Installation scenarios.................................................................................................................................. 1 Installation and configuration worksheets................................................................................................. 3 Installation directory worksheet............................................................................................................3 IBM Db2 data source worksheet............................................................................................................4 Oracle data source worksheet............................................................................................................... 5 WebSphere Application Server installation worksheet.........................................................................6
    [Show full text]
  • Database Management Systems Introduction Transaction ACID
    Introduction What is Concurrent Process (CP)? • Multiple users access databases and use computer Database Management systems simultaneously. • Example: Airline reservation system. Systems œ An airline reservation system is used by hundreds of travel agents and reservation clerks concurrently. Transaction, Concurrency and Why Concurrent Process? Recovery • Better transaction throughput and response time • Better utilization of resource Adapted from Lecture notes by Goldberg @ Berkeley Transaction ACID Properties of transaction • What is Transaction? • Atomicity: Transaction is either performed in its entirety or not performed at all, this should be DBMS‘ • A sequence of many actions which are responsibility considered to be one atomic unit of work. • Consistency: Transaction must take the database • Basic operations a transaction can include from one consistent state to another if it is executed in —actions“: isolation. It is user‘s responsibility to insure consistency œ Reads, writes • Isolation: Transaction should appear as though it is œ Special actions: commit, abort being executed in isolation from other transactions • Durability: Changes applied to the database by a committed transaction must persist, even if the system fail before all changes reflected on disk Concurrent Transactions Schedules • What is Schedules œ A schedule S of n transactions T1,T2,…Tn is an ordering of the B B operations of the transactions subject to the constraint that, for each transaction Ti that participates in S, the operations of Ti in Smust CPU2 A appear in the same order in which they occur in Ti. CPU A 1 œ Example: Sa: r1(A),r2(A),w1(A),w2(A), a1,c2; CPU1 T1 T2 time Read(A) Read(A) t1 t2 t1 t2 Write(A) interleaved processing parallel processing Write(A) Abort T1 Commit T2 1 Oops, something‘s wrong Another example • Reserving a seat for a flight • Problems can occur when concurrent transactions execute in an uncontrolled manner.
    [Show full text]
  • Data Definition Language (Ddl)
    DATA DEFINITION LANGUAGE (DDL) CREATE CREATE SCHEMA AUTHORISATION Authentication: process the DBMS uses to verify that only registered users access the database - If using an enterprise RDBMS, you must be authenticated by the RDBMS - To be authenticated, you must log on to the RDBMS using an ID and password created by the database administrator - Every user ID is associated with a database schema Schema: a logical group of database objects that are related to each other - A schema belongs to a single user or application - A single database can hold multiple schemas that belong to different users or applications - Enforce a level of security by allowing each user to only see the tables that belong to them Syntax: CREATE SCHEMA AUTHORIZATION {creator}; - Command must be issued by the user who owns the schema o Eg. If you log on as JONES, you can only use CREATE SCHEMA AUTHORIZATION JONES; CREATE TABLE Syntax: CREATE TABLE table_name ( column1 data type [constraint], column2 data type [constraint], PRIMARY KEY(column1, column2), FOREIGN KEY(column2) REFERENCES table_name2; ); CREATE TABLE AS You can create a new table based on selected columns and rows of an existing table. The new table will copy the attribute names, data characteristics and rows of the original table. Example of creating a new table from components of another table: CREATE TABLE project AS SELECT emp_proj_code AS proj_code emp_proj_name AS proj_name emp_proj_desc AS proj_description emp_proj_date AS proj_start_date emp_proj_man AS proj_manager FROM employee; 3 CONSTRAINTS There are 2 types of constraints: - Column constraint – created with the column definition o Applies to a single column o Syntactically clearer and more meaningful o Can be expressed as a table constraint - Table constraint – created when you use the CONTRAINT keyword o Can apply to multiple columns in a table o Can be given a meaningful name and therefore modified by referencing its name o Cannot be expressed as a column constraint NOT NULL This constraint can only be a column constraint and cannot be named.
    [Show full text]
  • Transaction Processing System
    Transaction processing system From Wikipedia, the free encyclopedia (Redirected from Transaction processing systems) It has been suggested that this article or section be merged into Transaction processing. (Discuss) Proposed since November 2012. Transaction processing is a style of computing that divides work into individual, indivisible operations, called transactions.[1] Atransaction processing system (TPS) or transaction server is a software system, or software/hardware combination, that supports transaction processing. Contents [hide] 1 History 2 List of transaction processing systems 3 Processing types o 3.1 Batch processing o 3.2 Real-time processing o 3.3 Time-sharing o 3.4 Transaction processing 4 Transaction processing system features o 4.1 Performance o 4.2 Continuous availability o 4.3 Data integrity o 4.4 Ease of use o 4.5 Modular growth 5 Databases and files o 5.1 Data warehouse o 5.2 Backup procedures . 5.2.1 Recovery process . 5.2.2 Types of back-up procedures . 5.2.2.1 Grandfather- father-son . 5.2.2.2 Partial backups . 5.2.3 Updating in a batch . 5.2.4 Updating in real-time 6 See also 7 References 8 Further reading [edit]History One of the first transaction processing systems was American Airline SABRE system, which became operational in 1960. Designed to process up to 83,000 transactions a day, the system ran on two IBM 7090 computers. SABRE was migrated to IBM System/360computers in 1972, and became an IBM product first as Airline control Program (ACP) and later as Transaction Processing Facility (TPF). In addition to airlines TPF is used by large banks, credit card companies, and hotel chains.
    [Show full text]
  • System Administration Guide: Volume 1
    System Administration Guide: Volume 1 SAP® Adaptive Server® Enterprise 16.0 DOCUMENT ID: DC31654-01-1600-01 LAST REVISED: May 2014 Copyright © 2014 by SAP AG or an SAP affiliate company. All rights reserved. No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice. Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors. National product specifications may vary. These materials are provided by SAP AG and its affiliated companies ("SAP Group") for informational purposes only, without representation or warranty of any kind, and SAP Group shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP Group products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty. SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and other countries. Please see http://www.sap.com/corporate-en/legal/copyright/ index.epx#trademark for additional trademark information and notices. Contents CHAPTER 1: Overview of System Administration .......1 Roles Required for System Administration Tasks ..............1 Database Owner .............................................................3
    [Show full text]
  • Transactions – Recovery ACID Properties and Recovery
    Transactions – Recovery CS 317/387 1 ACID Properties and Recovery The Recovery Manager is responsible for ensuring Atomicity and Durability . Atomicity is guaranteed by undoing the actions of the transactions that did not commit (aborted). Durability is guaranteed by making sure that all actions of committed transactions survive crashes and failures. 2 1 Types of Failures System crashes due to hardware or software errors main memory content is lost Transaction failures overflow, interrupt, data not available, explicit rollback, concurrency enforcement, programming errors no memory loss. Media failures problems with disk head, unreadable media surface (parts of ) information on secondary storage may be lost Natural disasters fire, flood, earthquakes, theft, etc. physical loss of all information on all media 3 Strategy If a transaction Ti is aborted (e.g., for concurrency control reasons), all its actions have to be undone . Active transactions at the time of the crash have to be aborted, i.e., their effects have to be undone when the system comes back. DBMS has to maintain enough information to undo actions of transactions (the LOG File) 4 2 Desired Behavior after system restarts: T1, T2 & T3should be durable.– T4 & T5 should be rolled back, i.e., effects undone. 5 Log Sequence of records (sequential file) Modified by appending (no updating) Contains information from which database can be restored Log and database stored on different mass storage devices Often replicated to survive single media failure Contains
    [Show full text]
  • Microsoft SQL Server DBA's Guide to Actifio Copy Data Management
    An SQL Server DBA’s Guide to Actifio Copy Data Management Actifio VDP 10.0 Copyright, Trademarks, and other Legal Matter Copyright © 2021 Google LLC. All rights reserved. Actifio™, OnVault™, and VDP™ are trademarks of Google LLC. All other brands, product names, goods and/or services mentioned herein are trademarks or property of their respective owners. ii An SQL Server DBA’s Guide to Actifio Copy Data Management | actifio.com | | actifio.com | An SQL Server DBA’s Guide to Actifio Copy Data Management iii iv An SQL Server DBA’s Guide to Actifio Copy Data Management | actifio.com | Contents Preface ........................................................................................................................................................................................................................................................................ vii The ActifioNOW Customer Portal...........................................................................................................................................................................................................................vii Actifio Support Centers.....................................................................................................................................................................................................................................................vii Chapter 1 - Introduction ...............................................................................................................................................................................................................................1
    [Show full text]
  • Zz-Cryptoprevent-Do
    REPLAY 4 ADMINISTRATOR’S GUIDE Copyright 2009/2010 by AppAssure Software , Inc. ALL RIGHTS RESERVED. No part of this work covered by the copyright hereon may be reproduced or used in any form or by any means — graphic, electronic, or mechanical, including photocopying, recording, taping, Web distribution or information storage and retrieval systems — without the written permission of the publisher. For permission to use material from this publication, contact AppAssure Software, Inc. 1925 Isaac Newton Square, Suite 440 Reston, VA 20190 +1 703-547-8686 [email protected] Replay 4 Administrator’s Guide ii Contents Chapter One: Introduction ............................................................. 1 REPLAY 4 .......................................................................................................................................................... 1 Chapter One: Introduction ........................................................................................................................ 1 What Is In This Guide ......................................................... 1 About Replay 4 ................................................................... 1 What you get with Replay 4 .............................................. 2 What's New in Replay 4? ................................................... 2 Replication for Off-site Backup and Disaster Recovery .............................................. 2 New Enterprise Console ...........................................................................................................
    [Show full text]
  • Finding Sensitive Data in and Around Microsoft SQL Server
    Technical White Paper Finding Sensitive Data In and Around the Microsoft SQL Server Database Finding Sensitive Data In and Around the Microsoft SQL Server Database Vormetric, Inc. 888.267.3732 408.433.6000 [email protected] www.vormetric.com Technical White Paper Page | 1 Finding Sensitive Data In and Around the Microsoft SQL Server Database Audience This technical document is intended to provide insight into where sensitive data resides in and around a Microsoft® SQL Server® database for the Microsoft® Windows® operating system. The information and concepts in this document apply to all versions of SQL Server (2000, 2005, 2008, 2008 R2, 2012). It also assumes some knowledge of Vormetric’s data encryp- tion and key management technology and its associated vocabulary. Overview At first blush, simply encrypting the database itself would be sufficient to secure data at rest in inside of the Microsoft SQL Server (SQL Server) database software running on Windows server platforms. However, enterprises storing sensitive data in a SQL Server database need to consider the locations around the database where sensitive data relating to SQL Server might reside, even outside the direct control of the Database Administrators (DBAs). For example, it is well within the realm of possibility that a SQL Server database could encounter an error that would cause it to send information containing sensi- tive data into a trace file or the alert log. What follows is a description of the SQL Server database along with a table that includes a list of all types of files and sub- types, along with the function these files provide and why it makes sense to consider protecting these files.
    [Show full text]