IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 11, NO. 1, JANUARY/FEBRUARY 1999 3
Concurrency Control in Database Systems
Bharat Bhargava, Fellow, IEEE
Abstract—Ideas that are used in the design, development, and performance of concurrency control mechanisms have been summarized. The locking, time-stamp, optimistic-based mechanisms are included. The ideas of validation in optimistic approach are presented in some detail. The degree of concurrency and classes of serializability for various algorithms have been presented. Questions that relate arrival rate of transactions with degree of concurrency and performance have been briefly presented. Finally, several useful ideas for increasing concurrency have been summarized. They include flexible transactions, adaptability, prewrites, multidimensional timestamps, and relaxation of two-phase locking.
Index Terms—Degree of concurrency, adaptability, time-stamp, optimistic, classes of serializability, performance, flexible transactions.
——————————F——————————
1INTRODUCTION ATABASE systems are essential for many applications, 2CONCURRENCY CONTROL APPROACHES D ranging from space station operations to automatic AND ALGORITHMS teller machines. A database state represents the values of Our main concern in designing a concurrency control algo- the database objects that represent some real-world entity. rithm is to correctly process transactions that are in conflict. The database state is changed by the execution of a user Each transaction has a read set and a write set. Two trans- transaction. Individual transactions running in isolation are actions conflict if the read set of one transaction intersects assumed to be correct. When multiple users access multiple with the write set of the other transaction and/or the write database objects residing on multiple sites in a distributed set of one transaction conflicts with the write set of the database system, the problem of concurrency control arises. other transaction. We illustrate this further in Fig. 1. The database system through a scheduler must monitor, If read set S(R ) and write set S(W ) have some database examine, and control the concurrent accesses so that the 1 2 overall correctness of the database is maintained. There entities (or items) in common, we say that the read set of T1 are two criteria for defining the correctness of a database: conflicts with the write set of T2. This is represented by the database integrity and serializability [5]. The database in- diagonal edge in the figure. Similarly if S(R2) and S(W1) tegrity is satisfied by assigning a set of constraints (predi- have some database items in common, we draw the other cates or rules) that must be satisfied for a database to be diagonal edge. correct. The serializability ensures that database transitions If S(W1) and S(W2) have some database items in com- from one state to the other are based on a serial execution of mon, we say that the write set of T1 conflicts with the write all transactions. For formal definitions, we refer the reader set of T2. This situation is represented by the horizontal to Appendix A. edge at the bottom. Concurrency control in database system has been We do not need to worry about the conflict between the the focus of research in the past 20 years. Concurrency read sets of the two transactions because read actions do control problems and solutions have been formalized in not change the values of the database entities. [24] and implemented and used in a variety of real world It must be noted that transactions T1 and T2 can conflict applications [17]. only if both are executing at the same time. If, for example, In this paper, we present several classes of concur- T1 has finished before T2 was submitted to the system, even rency control approaches and present a short survey of if their read and write sets intersect, they are not considered ideas that have been used for designing concurrency con- to be in conflict. trol algorithms. We have presented ideas that give an insight in the performance of these algorithms. Finally we present a few ideas that are useful in increasing the degree of concurrency.
²²²²²²²²²²²²²²²²
• B. Bhargava is with the Department of Computer Sciences, Purdue University, West Lafayette, IN 47907. E-mail: [email protected].
Manuscript received 7 July 1997; revised 26 Jan. 1999. For information on obtaining reprints of this article, please send e-mail to: [email protected], and reference IEEECS Log Number 109082. Fig. 1. Types of conflicts for two transactions.
1041-4347/99/$10.00 © 1999 IEEE 4 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 11, NO. 1, JANUARY/FEBRUARY 1999
Generic Approaches to Synchronization Locking an entity gives rise to two new problems: There are basically three generic approaches that can be livelock and deadlock. Livelock occurs when a transaction used to design concurrency control algorithms. The syn- repeatedly fails to obtain a lock. Deadlock occurs when chronization can be accomplished by utilizing: various transactions attempt locks on several entities si- multaneously; each transaction gets a lock on a different • Wait: If two transactions conflict, conflicting actions of entity and waits for the other transactions to release the one transaction must wait until the actions of the lock on the entities that they have succeeded in securing. other transactions are completed. The problem of deadlock can be resolved by the follow- • Timestamp: The order in which transactions are exe- ing approaches, among others: cuted is selected based on a time stamp. Each trans- • action is assigned a unique timestamp by the system Each transaction locks all entities at once. If some and conflicting actions of two transactions are proc- locks are held by some other transaction, then the transaction releases any locks that it was able to essed in timestamp order. The time stamp may be as- obtain. signed in the beginning, middle or end of the execu- • Assign an arbitrary linear ordering to the items, and tion. Version-based approaches assign time stamps to require all transactions to request locks in this order. database objects. • Rollback: If two transactions conflict, some actions of Gray and Reuter [17] has described experiments in a transaction are undone or rolled back or else one of which it was observed that deadlocks in database systems the transactions is restarted. This approach is also are very rare and it may be cheaper to detect and resolve called optimistic because it is expected that conflicts them rather than to avoid them. Since the correctness criterion for concurrently process- are such that only a few transactions would rollback. ing several transactions is serializability, locking must be In the following section, we give further details of each done correctly to assure the above property. One simple of these approaches and describe the concurrency control protocol that all transactions can obey to ensure seri- algorithms that are based on them. alizability is called Two-phase Locking (2PL). The protocol simply requires that in any transaction, all locks must pre- 2.1 Algorithms Based on Wait Mechanism cede all unlocks. A transaction operates in two phases: The When two transactions conflict, one solution is to make one first phase is the locking phase, and the second phase is the transaction wait until the other transaction has released the unlocking phase. The first phase can also be considered as entities common to both. To implement this, the system can the growing phase, in which a transaction obtains more and provide locks on the database entities. Transactions can get a more locks without releasing any. By releasing a lock, the lock on an entity from the system, keep it as long as the transaction is considered to have entered the shrinking particular entity is begin operated upon, and then give the phase. During the shrinking phase the transaction releases lock back. If a transaction requests the system for a lock on more and more locks and is prohibited from obtaining ad- an entity, and the lock has been given to some other trans- ditional locks. When the transaction terminates, all re- action, the requesting transaction must wait. To reduce the maining locks are automatically released. The instance just waiting time when a transaction wants to read, there are before the release of the first lock is called lockpoint. The two two types of locks that can be employed, based on whether phases and lockpoint are illustrated in Fig. 2. the transaction wants to do a read operation or a write op- We now present a simple centralized algorithm that eration on an entity: utilizes locking in a distributed database system. For the sake of simplicity, we may assume that all transactions 1) Readlock: The transaction locks the entity in a shared write into the database and the database is fully replicated. mode. Any other transaction waiting to read the same In real systems it might be very inefficient to have a fully entity can also obtain a readlock. replicated database. Moreover, the majority of the transac- 2) Writelock: The transaction locks the entity in an exclu- tions usually only read from the database. But since multi- sive mode. If one transaction wants to write on an en- ple copies of a given entity and the write operations of tity, no other transaction may get either a readlock or transactions are the major reason for studying concurrency a writelock. control algorithms, we focus on these issues. When we say lock, it means any of the above types 2.1.1 A Sample Centralized Locking Concurrency of lock. After a transaction has finished operations on an Control Algorithm entity, the transaction can do an unlock operation. After an A brief outline of a simple centralized locking algorithm is unlock operation, either type of lock is released, and the given below. When a transaction T arrives at node X, the entity is made available to other transactions that may i following steps are performed: be waiting. It is important to note that lock and unlock operations 1) Node X requests from the central node the locks for can be embedded in a transaction by the user or be trans- all the entities referenced by the transaction. parent to the transaction. In the later case, the system takes 2) The central node checks all the requested locks. If the responsibility of correctly granting and enforcing lock some entity is already locked by another transaction, and unlock operations for each transaction. then the request is queued. There is a queue for each entity and the request waits in one queue at a time. BHARGAVA: CONCURRENCY CONTROL IN DATABASE SYSTEMS 5
Fig. 2. Two-phase locking and lockpoint: Obtain lock; Release lock.
3) When the transaction gets all its locks, it is executed node. But it is not necessary for the central node at the central node (the execution can also take place to wait for this to happen; it is sufficient for the at node X, but that may require more messages). The central node to guarantee that the write set values values of read set are read from the database, neces- are written at every node in the same order as they sary computations are carried out, and the values were performed at the central node. To achieve this of the write set are written in the database at the cen- the central node can assign a monotonically increas- tral node. ing sequence number to each transaction. The sequence 4) The values of the write set are transmitted by the number is appended to the write set of the transaction central node to all other nodes (if the database is fully and is used to order the update of the new values into replicated). the database at each node. 5) Each node receives the new write set and updates the database; then an acknowledgment is sent back to the Now the central node does not have to wait for any ac- central node. knowledgments, but the equivalent effect is achieved. This 6) When the central node receives acknowledgments can make the centralized control algorithm more efficient. from all other nodes in the system, it knows that Sequence numbers may cause additional problems. Sup- pose two transactions T5 and T6 are assigned sequence transaction Ti has been completed at all nodes. The central node releases the locks and starts processing numbers 5 and 6, respectively, by the central node. Let us the next transaction. further suppose that T5 and T6 have no entities in common and so do not conflict. If transaction T5 is very long, trans- Some interesting variations of the centralized locking al- action T6, which arrived at the central node after T5, may gorithm are as follows: be ready to write the values of its write set, but this opera- • Locking at Central Node, Execution at all Nodes. Instead tion for T6 must wait at all nodes for T5. A simple solution of executing the transaction at the central node, we to this problem is to attach the sequence numbers of all can only assign the locks at the central node and send lower-numbered transactions for which a given transaction the transaction back to node X. The transaction Ti is must wait before writing in the database. This list is called executed at node X. The values of the read set are a wait-for list. In such a case, a transaction waits only for read, and the values of the write set are obtained at the transactions in its wait-for list. The wait-for list is at- node X. Node X sends the values of the write set and tached to the write set of each transaction. In some cases the obtains acknowledgments from all other nodes. It size of the wait-for list can grow very large, but transitivity then knows that transaction Ti has been completed. among sequence numbers in wait-for lists can be used to The node X sends a message to unlock entities refer- reduce it. Moreover a complement of this wait-for list is enced by Ti. The central node after receiving this mes- called do not-wait-for list can also be used. Many such ideas sage releases the locks and starts assigning locks to are discussed in [14]. The notion of wait-for list is similar to waiting transactions. causal ordering as discussed in [27]. In causal ordering, a • Avoid Acknowledgments, Assign Sequence Numbers. In message carries information about its transitive causal the centralized control algorithm, the acknowledg- predecessors and the overheads to achieve this can be re- ments are needed by the central node (or node X in duced by requiring that each message carries information the above extension) to find out if the values of the about its direct predecessor only. Causal ordering has also write set have been written in the database at every been discussed in [7]. 6 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 11, NO. 1, JANUARY/FEBRUARY 1999
• Global Two-phase Locking. This is a simple variation order, each read-write conflict relation and write-write of the centralized locking mechanisms. Instead of conflict relation is resolved by timestamp order. Conse- a transaction getting all locks in the beginning and quently all paths in the relation are in timestamp order and, releasing all locks in the end, the policy of two- since all transactions have unique timestamps, it follows phase locking is employed. Each transaction obtains that no cycles are possible in a graph representing transac- the necessary locks as they are needed, computes, tion histories. and then releases locks on entities that are no long- 2.2.1 Timestamp Ordering with Transaction Classes er needed. A transaction cannot get a lock after it has released any lock. So if more locks are needed In this approach, it is assumed that the read set and the in the future, it should hold on to all the present locks. write set of every transaction is known in advance. This The other parts of the algorithm remain the same information is used to group transactions into predefined as before. classes. A transaction class is defined by a read set and a • Primary Copy Locking. In this variation, instead of se- write set. A transaction T is a member of a class C if the lecting a node as the central controller, a copy of each read set of T is a subset of the read set of class C and the entity on any node is designated as the primary copy write set of T is a subset of the write set of class C. Class of the entity. A transaction must obtain the lock on the definitions are used to provide concurrency control. This primary copy of all entities referenced by it. At any mechanism was used in the development of a prototype given time the primary copy contains the most up-to- distributed database management system called SDD-1, date value for that entity. developed by the Computer Corporation of America [3]. It is important to point out that locking approaches are 2.2.2 Distributed Voting Algorithm in general pessimistic. For example, two-phase locking is a This algorithm uses distributed control to decide which sufficient condition rather than the necessary condition for transaction can be accepted and executed. The nodes of the serializability. As an example, if an entity is only used by a distributed database system communicate among them- single transaction, it can be locked and unlocked freely. The selves and vote on each transaction. If a transaction gets a question is, “How can we know this?” Since this informa- majority of OK votes, it is accepted for execution and com- tion is not known to the individual transaction, it is usually pletion. A transaction may also receive a reject vote, in not utilized. Thus locking that is based on prevention of which case it must be restarted. In addition to voting OK access does not fully benefit from actual favorable condi- and reject, nodes can also defer or postpone voting on a tions that may exist. particular transaction. This approach is a result of the work of Thomas [29]. The 2.2 Algorithms Based on Time-Stamp Mechanism timestamps are maintained for the database entities. A Timestamp is a mechanism in which the serialization order timestamp on an entity represents the time when this entity is selected a priori; the transaction execution is obliged to was last updated. obey this order. In timestamp ordering, each transaction is assigned a unique timestamp by the scheduler or concur- 2.3 Algorithms Based on Rollback Mechanisms rency controller. Obviously, to achieve unique timestamps As we have seen in the last two sections, timestamp algo- for transactions arriving at different nodes of a distributed rithms are a major departure from the locking or the wait system, all clocks at all nodes must be synchronized or else mechanisms. In this section, a family of nonlocking or two identical timestamps must be resolved. optimistic concurrency control algorithms [19] are presented. Lamport [20] has described an algorithm to synchronize In this approach, the idea is to validate a transaction against distributed clocks via message passing. If a message arrives a set of previously committed transactions. If the validation at a local node from a remote node with a higher time- fails, the read set of the transaction is updated and the stamp, it is assumed that the local clock is slow or behind. transaction repeats its computation and again tries for vali- The local clock is incremented to the timestamp of the re- dation. The validation phase will use conflicts among the cently received message. In this way all clocks are advanced read sets and the write sets along with certain timestamp until they are synchronized. In the other scheme where two information. The validation procedure starts when a trans- identical timestamps must not be assigned to two transac- action has completed its execution under the optimistic tions, each node assigns a timestamp to only one transac- assumption that other transactions would not conflict with tion at each tick of the clock. In addition the local clock time it. The optimistic approach maximizes the utilization of is stored in higher-order bits and the node identifiers are syntactic information and attempts to make use of some stored in the lower-order bits. Because node identifiers are semantic information about each transaction. If no a priori different, this procedure will ensure unique timestamps. information about an incoming transaction is available to When the operations of two transactions conflict, they the concurrency controller, it cannot preanalyze the trans- are required to be processed in timestamp order. It is easy action and try to guess potential effects on database entities. to prove that timestamp ordering (TSO) produces serializ- On the other hand, maximum information is available able histories. Thomas [29] has studied the correctness and im- when a transaction has completed its processing. A concur- plementation of this approach and described it. Essentially rency controller can make decisions about which transac- each node processes conflicting operations in timestamp tion must abort while other transactions may proceed. This BHARGAVA: CONCURRENCY CONTROL IN DATABASE SYSTEMS 7 decision can be made at the time of arrival of a transaction, Kung and Papadimitriou [18] have shown that when during the execution of a transaction, or the decision can be only the syntactic information is available to the concurren- made at the end of processing. Decisions made at arrival cy controller, serializability is the best achievable correct- time will tend to be pessimistic and decisions made at the ness criterion. We now describe the validation phase. end may invalidate the transaction processing and require A transaction enters the validation phase only after rollback. If the transactions’ effects are kept in a private completing its computation phase. The transaction that en- space and are not made known to other transactions until ters the validation phase before any other transaction is the concurrency controller ensures their correctness, one automatically validated and committed. This is because can design concurrency control mechanisms that employ initially the set of committed transactions is empty. This maximum information at the cost of restarting some trans- transaction writes updated values in the database. Since actions. The extent of this restart will be proportional to the this transaction may be required to validate against future degree of conflict among concurrent transactions. A similar transactions, a copy of its read and write sets is kept by the approach was suggested in [19] for a centralized hierarchi- system. Any transaction that enters the validation phase cal database system and was studied further in [4], [8]. validates against the set of committed transactions that There are four phases in the execution of a transaction in were concurrent with it. As an extension the validation pro- the optimistic concurrency control approach: cedure could include validation against other transactions 1) Read: Since reading a value of an entity cannot cause a currently in the validation phase.
loss of integrity, reads are completely unrestricted. A Consider two transactions Ti and Tj. Let S(Ri) and S(Rj) transaction reads the values of a set of entities (called be the read sets and S(Wi) and S(Wj) be the write sets of Ti P P read set) and assigns them to a set of local variables. and Tj, respectively. Let (Ri) and (Rj) denote the time The names of local variables have one-to-one corre- when the last item of the read set S(Ri) and S(Rj) were read P P spondence to the names of entities in the databases, from the database and let (Wi) and (Wj) denote the time but the values of local variables are an instance of a when the first item of the write set S(Wi) and S(Wj) will be past state of the database and are only known to the or were written in the database.
transaction. Of course since a value read by a transac- Assume Ti arrives in the system before Tj. Let Tj be a tion could be changed by a write of another transac- committed transaction when the transaction Ti arrives for tion, making the read value incorrect, the read set is validation. Now there are four possibilities: subject to validation. The read set is assigned a time- 1) If T and T do not conflict, T is successful in the vali- P i j i stamp denoted by (Ri). dation phase and can either proceed or follow Tj. 2) Compute: The transaction computes a set of values for ʝ ¡ ʝ ¡ 2) If S(Ri) S(Wj) Ø and S(Rj) S(Wi) Ø, Ti fails in data entities called the write set. These values are as- the validation phase and restarts. signed to a set of corresponding local variables. Thus ʝ ¡ ʝ 3) If S(Ri) S(Wj) Ø and S(Rj) S(Wi) = Ø, Ti is suc- all writes after computation take place on a transac- cessful in validation. T must proceed T in any serial tion’s copy of the entities of the database. i j history since P(R ) < P(W ). This possibility is illus- 3) Validate: The transaction’s local read set and write set i j trated as follows: are validated against a set of committed transactions. Details of this phase constitute a main part of this al- gorithm and are given in the next section. 4) Commit and Write (called write for short): If the trans- action succeeds in validation, it is considered com- mitted in the system and is assigned a timestamp de- P noted by (Wi). Otherwise the transaction is rolled back or restarted at either the compute phase or the read phase. If a transaction succeeds in the validation The edge between S(Wi) and S(Wj) does not matter phase, its write set is made global and the values of because if S(Wi) intersects with S(Wj), then S(Wi) the write set become values of entities in the database - ʝ can be replaced by S(Wi) [S(Wi) S(Wj)]. In other at each node. words, Ti will write values for only those entities All four phases of concurrently processing transactions that are not common with the write set of Tj. If we can be interleaved, but the read phase should precede the do so, we get the equivalent effect as if Ti were writ- computation and validation phase. ten before Tj. S R ʝ S W S R ʝ S W ¡ T 2.3.1 The Validation Phase 4) If ( i) ( j) = Ø and ( j) ( i) Ø, i is suc- cessful in validation. Ti must follow Tj in any serial The concurrency controller can utilize syntactic infor- history since P(W ) > P(R ). This possibility is illus- mation, semantic information, or a combination of the two. i j trated as follows: Here we discuss the use of syntactic information in the context of a validation at one node only. For use of semantic information, we refer the reader to [5]. 8 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 11, NO. 1, JANUARY/FEBRUARY 1999
This obviously requires updating a given entity of the read set many times and thus is inefficient. But one nice property of this procedure is that the transaction does not have to restart from the beginning and does not have to read the database on secondary storage. A practical question is whether the read sets and write sets can be stored in memory. The transactions Tj that must For a set of concurrent transactions we proceed as fol- be stored in memory must satisfy the following condition: lows: For each transaction that is validated and enters the If Tj is a committed transaction, store Tj for future valida- · list of committed transactions, we draw a directed edge tion if Ti set of committed transaction such that: according to the following rules: P P ʝ ¡ f { (Ri) < (Wj)} AND {S(Ri) S(Wj) } • If Ti and Tj do not conflict, do not draw any edge. It has been shown that the set of transactions to be • If Ti must precede Tj, draw an edge from Ti to Tj, stored for future validation will usually be small [4]. In Ti Tj. general, a maximum size of the number of committed • If Ti must follow Tj, draw an edge from Tj to Ti, transactions that can be stored in the memory can be de- Ti Tj. termined at design time. In case the number of committed Thus, a directed graph is created for all committed transactions exceed this limit, the earliest committed trans- transactions with transactions as nodes and edges as ex- action Tj can be deleted from this list. But care should be taken to restart (or invalidate) all active transactions Ti for plained above. P P which (Ri) < (Wj) before Tj is deleted. When a new transaction Ti arrives for validation, it is checked against each committed transaction to check if Ti A DETAILED EXAMPLE. This example illustrates a variety of should precede or follow, or if the order does not matter. ideas of optimistic approach and its advantages over Condition for Validation: There is never a cycle in the locking. We assume that a history is presented to a scheduler (concurrency controller). The scheduler ei- graph of committed transactions because they are serializ- ther accepts or rejects the history. It does so by trying able. If the validating transaction creates a cycle in the graph, conflict preserving exchanges on the input history to it must restart or rollback. Otherwise, it is included in the check if it can be serializable. set of committed transactions. We assume the validation of a transaction to be in the critical section so that the set of In this example, we use Ri and Wi to represent the read committed transactions does not change while a transaction action (as well as the read set) and the write action (as well is actively validating. as the write set) of a transaction Ti. h n In case a transaction fails in validation, the concurrency Let be an input history of transactions to the sched- uler as follows: controller can restart the transaction from the beginning LLL of the read phase. This is because the failure in the valida- h = R1R2W2R3W3 RnWnW1 tion makes the read set of the failed transaction incorrect. Here transaction T executes the read actions, followed by The read set of such a transaction becomes incorrect be- 1 the read/write action of T , T , ¤, T , followed by the write cause of some write sets of the committed transactions. 2 3 n actions of T . Since the write sets of the committed transactions meet the 1 Suppose R1 and W2 conflict as represented by an edge read set of the failed transaction (during validation), it may as follows: be possible to update the values of the read set of the trans- action at the time of validation. If this is possible, the failed transaction can start at the beginning of the compute phase rather than at the beginning of the read phase. This will save the I/O access required to update the read set of the failed transaction.
2.3.2 Implementation Issues The history h is not allowed in the locking protocols be- Let Ti be the validating transaction and let Tj be a member cause W2 is blocked by R1. If T1 is a long transaction and T2 of a set of committed transactions. is a small transaction, the response time for T2 will suffer. In The read sets and write sets of the committed transac- general T1 can block T2, T2 can block T3 (if T2 and T3 have a tions are kept in the system. The transaction is validated conflict) and so on. against the committed transactions. The committed trans- Let us consider several cases in optimistic approach. actions are selected in the order of their commitment. The Case 1: For the history h, in the optimistic approach of read set is updated by the conflicting write set at the time of ¤ Kung and Robinson [19], Ti (i = 2, , n) can commit. Write each validation. If none of the transactions conflict with the sets (Wi s) of committed transactions are saved to validate validating transaction, it is considered to have succeeded in against the read set of T1. Basically the conflict preserving the validation and hence to have committed. BHARGAVA: CONCURRENCY CONTROL IN DATABASE SYSTEMS 9
exchange (switch) as follows is attempted so that R1 can be brought next to W1.
We can get a history as follows. LLL R2W2RKWKR1W1R3W3 RnWn which is serializable. Case 2: An extension of this idea is to try the either ex- 2.3.3 Implementation of Validations change (switch) as follows: Let us now illustrate how these ideas for optimistic can be
implemented. Consider n transactions. Assume T1 starts L before T2, but finishes after Tn. Let T2T3 Tn finish in order. Since T2 is the first transaction to validate, it is commit- ted automatically. So we have a conflict graph with a node
for T2 as follows:
T2 The resulting histories can be either When T3 arrives for validation, the read set of T3 is LLL R1W1R2W2 RnWn validated against write set of T2. If they conflict, the edge T3 T2 is drawn. Next, the write set of T3 is validated or against the read set of T2 leading to the edge T2 T3. Since LLL R2W2 RnWnR1W1. this causes a cycle, T3 is aborted. Otherwise, T3 is serialized with T2. So we have a conflict graph say as follows: For switching W1, we would need to save not only the write sets of committed transactions, but also the read sets T2 T3 of committed transactions. This will allow more histories to For a transaction T4 the edges are checked as follows: be acceptable to the scheduler. Check the edge T4 T2. If it exists, check the edge Case 3: A further extension of this idea is to try switching T2 T4. Abort if both edges exist. If only T4 T2 exists, do R1 toward W1 and W1 toward R1 if conflict preserving ex- not check the edge T4 T3, but check the edge T3 T4 changes are possible. only. This requires checking only three edges as follows: Consider the conflict as follows:
L L Consider the history h = R1R2W2 RK-1Wk-1RKWK R W W . Because of a conflict edge between R and W , R n n 1 1 K 1 If edge T T does not exist, there is no need to check can be scheduled only before W . Similarly due to the con- 4 2 K T T . In this case check the edge T T and T T . flict edge R - and W , W can be scheduled only after R - . 2 4 4 3 3 4 K 1 1 1 K 1 Once again only three edges are checked as follows: Switching R1 and W1, the scheduler can get a serializable L L history R2W2 RK-1WK-1R1W1RKWK RnWn. Using the switching of R1 or W1 alone would not have allowed this history to be acceptable to a scheduler.
Finally we consider the case where both R1 and W1 are stuck due to conflicts. Consider the history:
So, in general, for every new transaction that comes for validation, only n edges are checked if there are n - 1 com- mitted transactions. This may be more efficient implemen- tation than checking for a cycle for the conflict graph of n R1 can switch up to T3 and W1 can switch up to TK due to transactions for each validation. conflicts (say R1W3 and W1WK). Now we present a theorem that relates the rollback of The scheduler can try to move the subhistory R1R3W3 to optimistic with deadlock of locking approach: the right and RKWKW1 to the left as shown next: 10 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 11, NO. 1, JANUARY/FEBRUARY 1999
THEOREM 1 [8]. In a two step transaction model (all reads for a In contrast, the class DSS (distributed strict serializabil- transaction precede all writes) whenever there is a transac- ity) the histories retain the completion order of transactions tion rollback in the optimistic approach due to a failure in based on the event w. The class DSTO is contained in class the validation, there will be a deadlock in the locking ap- DSS. Finally, the class DCP (distributed conflict preserving) proach (unless deadlocks are not allowed to occur) and will is based on the notion the a read or write action is freely cause a transaction rollback. rearranged as long as the order of conflicting accesses is preserved. The serializability is guaranteed by maintaining PROOF. For deadlock detection, the system can produce a wait-for digraph in which the vertices represent the a acyclic conflict graph that is constructed for each history. transactions active in the system. An edge between 3.1.1 The Hierarchy two transactions in the wait-for graph is drawn if and All the classes G2PL, L2PL, DCP, DSTO, and DSS are seri- only if one transaction holds a read-lock or a write- alizable and form a hierarchy based on the degree of lock and the other transaction is requesting a write- concurrency. Fig. 3 depicts the hierarchy, where SR is the set lock on the same item. This will happen when the of all serializable histories. read-set or the write-set of the first transaction con- In Fig. 3, each possible intersection of these classes is flicts (intersects) with the write-set of the second marked by ‘.i’ where i is from 1 to 11, and the exemplary transaction. An edge in the dynamic conflict graph exists in exactly the same case. history for area ‘.1’ is denoted as ‘h.i’. Some of the histories Thus a wait-for graph has the same vertices (i.e., are composite (formed by concatenating two histories). The the set of all active transactions) as the dynamic con- transaction set and conflict information are given below. flict graph and the edges in the wait-for graph corre- Let there be two nodes represented by N = {1, 2}, and spond one to one with the edges in the dynamic con- seven transactions denoted by T = {a, b, c, d, e, f, g}. flict graph. Hence the wait-for graph is identical to The atomic operations for each transaction are as shown the dynamic conflict graph and a cycle in the wait-for below: graph occurs whenever there is a cycle in the dynamic 112 Trans. ‘a’ = >CRxaa,, Wy W a y ; conflict graph. A deadlock occurs when there is a cy- 112 cle in the wait-for graph and to resolve the deadlock, Trans. ‘b’ = >CRwbbb,, Wy W y ; some transaction must be rolled back. Since validation Trans. ‘c’ = >CRuWv212,, Wv ; of a transaction fails and a rollback happens when ccc there is a cycle in the dynamic conflict graph, the as- 212 Trans. ‘d’ = >CRvWvWvddd,, ; sertion of the theorem is concluded. o 112 Trans. ‘e’ = >CRweee,, Wv W v ; 21 2 3PERFORMANCE EVALUATION OF CONCURRENCY Trans. ‘f’ = JLRtff,, Ww Ww f ; CONTROL ALGORITHM 212 Trans. ‘g’ = JLRwWyggg,, Wy ; There are two main criteria for evaluating the performance of core control algorithms. We discuss them in some detail Basically, each transaction reads an entity and broadcasts as follows: the update to both nodes. The hierarchical relation among the classes DCP, DSS, 3.1 Degree of Concurrency and G2PL is similar to that in [24]. However, the classes This is the set of histories that are acceptable to a sched- L2PL and DSTO and their relationships with other classes is uler. For example a serial history has the lowest degree of different. Note that, unlike the class 2PL in the centralized concurrency. 2PL and optimistic approaches provide a database system, the class L2PL which also uses two-phase higher degree of concurrency. The concurrency control al- locking but with local freedom of choosing the lock points gorithms have been classified in various classes based on is not contained in DSS. the degree of concurrency provided by them in [24]. The 122121 concurrency control algorithms for distributed database h.1: RwWyRuWvWvWybb[ ] []ccc [] [] []b [] processing have been classified in [7]. We specifically point h.2: RwRxWyWyWyWy11[ ] [] 2 [] 1 [] 1 [] 2 [] out the classes of global two-phase locking (G2PL) and local b aa a bb 11 1 1 2 2 two-phase locking (L2PL). All histories in class G2PL are h.3: RxRwWyWyWyWya[]b [ ]a []b []a []b [] characterized by global lock points. Since each node is ca- h.4: h.2 + h.3 pable of independent processing, the global history can be h.5: RxRwRtWwWyWwWy11[] [] 2 [] 1 [] 1 [] 2 [] 2 [] serializable if each node maintains the same order of lock a b ff a f a 21 points for all conflicting transactions locally. The class L2PL Wbb[]yW [] y contains the class G2PL and provides a higher degree of h.6: h.2 + h.5 concurrency [7]. In a history for the class DSTO (distributed 121 22 21 serializable in the time stamp order), the transactions are h.7: RwRtWwWwRvWvWve [ ]ff [] [ ] f [ ]dd [] [] d [] guaranteed to follow in the final equivalent serial history, 21 W []v W []v the same order as the transaction’s initial access or event a. ee a, w events are discussed in Appendix A. BHARGAVA: CONCURRENCY CONTROL IN DATABASE SYSTEMS 11
Fig. 3. The hierarchy of the classes SR, DCP, L2PL, G2PL, DSTO, and DSS.
21 1 22 1 2 The probability that two transactions do not share a data h.8: RuRwWvWvRvWvWvce[] [ ] e [] e []d []cc [] [] object is given by the following term: W 12[]W [] ddv v M ()MB− 22 2 21 1 2 ∗ CCB B h.9: RtWwRwWyRwWyWvff[] []ggege [] [] [] [] [] CCM ∗ M W 1[]W 1 [ ] B B e v f w This term is equal to h.10: h.6 + h.8 MB− MB−−1 MB−+21 h.11: h.7 + h.10 ∗ ∗∗L M M − 1 MB−+1 3.2 System Behavior One can evaluate the performance of a concurrency con- Lower bound on this term trol algorithms by studying the response time for transac- B MB−+21 = tions, throughput of transactions per second, rollback or −+ . blocking of transactions, etc. Such measures are system de- MB1 pendent and change as technology changes. Several re- Maximum probability that two transactions will share a search papers have done simulation, analytical, and ex- data object is given by perimental study of a wide range of algorithms. These B studies tend to identify the conditions under which a spe- MB−+21 1 − . cific approach will perform better. For example, in [4], we MB−+1 have shown after detailed simulations that the optimistic approach performs better than locking when there is a mix By plugging some sample values for B and M, we get of large and small transactions. This is contrary to the wis- the following: dom that optimistic performs better when there are few conflicts. We found that in the case of low conflicts, in op- B M Probability of Conflict P(C) timistic approach there are fewer aborts, but in locking 5 100 0.0576 there is less blocking. Similarly, if a lot of conflicts occur, 10 500 0.0025 both locking and optimistic algorithms suffer. Thus, one 20 1,000 0.1130 could conclude that if the cost of rollback and validation is 2 The probability of a cyclic conflict is order (P(C)) which not considerably high, in both locking and optimistic, the transactions will either suffer or succeed. is quite small. In many applications, it has been found that conflicts are We have conducted a simulation study [4] that illustrates rare [12], [16], [17]. We present another strawman analysis. the issues of arrival rate, relates multiprogramming level, Assume that the database size is M and the read set and frequency of cycles in a database environment. In Fig. 4, we M show that the degree of multiprogramming is low for a va- write set size is B. C represents the number of combina- B riety of transaction arrival rates in a sample database. tions for choosing B objects from a set of M objects. 12 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 11, NO. 1, JANUARY/FEBRUARY 1999
Fig. 4. Database size = 100, I/O cost = 0.025, CPU cost = 0.0001.
In Fig. 5, we show that the probability of a cycle is quite can use the approach of dynamic timestamp vector genera- low for low degrees of multiprogramming. tions for each transaction and dynamic validation of con- In Fig. 6, we found that optimistic performs better than flicting transactions to increase the degree of concurrency. locking for very low arrival rates. Details of this study can The class of multidimensional time stamp vectors intersects be found in [4]. with the class SSR and 2PL and is contained in the class DSR. Classes 2PL, SSR, and DSR are defined as in [24]. 4MORE IDEAS FOR INCREASING CONCURRENCY 4.2 Relaxations of Two-Phase Locking 4.1 Multidimensional Time Stamps In [22], we have provided a clarification of the definition of There are several variations of timestamp ordering. For ex- two-phase blocking. A restricted non two-phase locking ample, multiple versions [25] of item values have been used (RN2PL) class that contains the class of 2PL has been for- to increase the degree of concurrency. The conventional mally defined. An interesting interpretation of the RN2PL is time stamp ordering tends to prematurely determine the given as follows. serializability order, which may not fit in with the subse- A transaction (a leaser) may release a lock (rent out a quent history, forcing some transactions to abort. The mul- lock token) before it may still request some more locks. If a tidimensional time stamp protocol [21] provides a higher later transaction (a leasee) subsequently obtains such a re- degree of concurrency than single time stamp algorithms. leased lock (rents the lock token), it cannot release this lock This protocol allows the transaction to have a time stamp (sublease the lock token) until ALL its leasers will not re- vector of up to k elements. The maximum value of k is lim- quest any more locks. (Now the leasers are ready to transfer ited by twice the maximum number of operations in a sin- all their lock tokens to leasees. So, each of their leasees can gle transaction. Each operation may set up a new depend- be a new leaser.) ency relationship between two transactions. The relation- This scenario enforces acyclic leaser-leasee relation- ship (or order) is encoded by making one vector less than ships, and thus produces only serializable histories. Fur- another. A single time stamp element is used to bear this ther, the locking sequence may not be two-phased. It is not information. Earlier assigned elements are more significant appropriate to claim that either protocol is superior to the in the sense that subsequent dependency relationships can- other because many conditions need to be considered for not conflict with previously encoded relationships. Thus such a comparison. Since two-phase locking is a special the scheduler can decide to accept or abort an operation case of restricted-non-two-phase locking, it gives the flexi- bility for some transactions to be non-two-phase locked. In based on the dependency information derived from all pro- some cases, it would be desirable to allow long-lived trans- ceeding operations. In other words, the scheduler can use actions to be non-two-phase locked to increase the avail- the approach of dynamic timestamp vector generations for ability of data items. each transaction and dynamic validation of conflicting one BHARGAVA: CONCURRENCY CONTROL IN DATABASE SYSTEMS 13
Fig. 5. Database size = 500, transaction size = 5, 10, 15, 20, 25.
Fig. 6. Database size = 200, no. of nodes = 3, communication delay = 0.10, I/O cost = 0.025, transaction size = 5 (time in seconds).
4.3 System Defined Prewrites precommit operation. After the precommit, another read In [23], we have introduced a prewrite operation before an transaction is permitted to read the announced prewrite actual write operation is performed on database files. A values even before the other transaction has finally updated prewrite operation announces the value that a transaction the data objects and committed. The eventual updating on intends to write in future. A prewrite operation does not stable storage may take a long time. This allows nonstrict change the state of the data object. Once all the prewrites executions and increases the potential concurrency as com- of a transaction are announced, the transaction executes a pared to the algorithms that permit only read and write operations on the database files. A user does not explicitly 14 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 11, NO. 1, JANUARY/FEBRUARY 1999 mention a prewrite operation but the system introduces a method requires the development of a common data prewrite operation before every write. structure for all the ways to implement a particular concur- Short duration transactions can read the value of a data rency controller (called sequencer). The converting state item produced but not yet released by a long transaction method works by invoking a conversion routine to change before its commit. Therefore, using prewrites, one can bal- the state information as required by a different method. The ance a system consisting of short and long transactions suffix sufficient method requires switching from one without causing delay for short duration transactions. method to another by overlapping the execution of both methods until certain termination conditions are satisfied. 4.4 Flexible Transactions The flexible transaction model [32] supports flexible exe- 5CONCLUSIONS cution control flow by specifying two types of depen- dencies among the subtransactions of a global distrib- Concurrency Control is a problem that arises when multi- uted transaction: ple processes are involved in any part of the system. Earlier ideas of notions of serializability and the concept of two- 1) execution ordering dependencies between two subtrans- phase locking were discussed in [13]. The ideas of time actions, and stamps were introduced by [29]. The optimistic approach 2) alternative dependencies between two subsets of sub- was proposed by [19]. The classes of serializability and the transactions. formalism for concurrency control was presented in [24]. A flexible transaction allows for the specification of multi- Several books that detail these subjects have been published ple alternate subsets of subtransactions to be executed and [6], [25], [2], in addition to survey papers [1], [5]. The per- results in the successful execution and commitment of the formance evaluation was studied in [14]. In most commer- subtransactions in one of those alternate subsets, the execu- cial systems, the most popular mechanism for concurrency tion of a flexible transaction can proceed in several different control is two-phase locking [17]. The ideas of adaptable ways. The subtransaction in different alternate subsets may concurrency control were published in [11] and were im- be attempted simultaneously, as long as any attempted plemented in the RAID system [10]. It has been commented subtransactions not in the committed subset of subtransac- by system experts that concurrency control only contributes tions can either be aborted or have their effects undone. The 5 percent to the response time of a transaction and so even a flexible transaction model increases the failure resilience of simple two-phase locking protocol should suffice. How- global transactions. In [32], we have defined a weaker form ever, due to the many interesting ideas that came into play of atomicity, termed semiatomicity, that is applicable to in distributed database systems in the context of replication flexible transactions. Semiatomicity allows a flexible trans- and reliability, research in concurrency control is continu- action to commit as long as a subset of its subtransactions ing. Some studies are being done for object-oriented sys- that can represent the execution of the entire flexible trans- tems while others are dealing with semantics of transac- action commit. Semiatomicity enlarges the class of executa- tions and weaker form of consistency. Over a hundred ble global transactions in a heterogeneous distributed data- Ph.D. thesis that study some aspect of concurrency control base system. have been produced. We continue to learn of new ideas such as flexible 4.5 Adaptable Concurrency Control transactions, value-dates, prewrites, degrees of commit- Existing database systems can be interconnected resulting ment and view serializability [9]. In large scale systems, it is in a heterogeneous distributed database system. Each site in difficult to block access to database objects for transactions. such a system could use a different strategy for concurrency If a system has to perform 10,000 transactions per second, control. For example, one site could be using the two-phase the locking as we know today will not be a solution. We locking concurrency control method while another could be suggest readers to learn from variety of books that are running the optimistic method. Since it may not be possible available on both theory and implementation of concur- to convert such different systems and algorithms to a ho- rency control mechanisms. mogeneous system, solutions must be found to deal with such heterogeneity. Already research has been done toward the designing of algorithms for performing concurrent up- APPENDIX A dates in a heterogeneous environment [31]. The issues of BASIC TERMINOLOGY global serializability and deadlock resolution have been A distributed database management system (DDBMS) is a data- solved. The approach in [11] is a variation of the optimistic base system distributed among a set of nodes N connected concurrency control for global transactions while allowing by communication links. Each node has its own independ- individual sites to maintain their autonomy. ent computing resources. Another concept that has been studied in the Reliable, The database is modeled by a set of logical database enti- Adaptable, Interoperable Distributed (RAID) database sys- ties which may have one or more physical copies of data value. tem [10] involves facilities to switch concurrency control The database entities are accessed by unique names; how methods. A formal model for an adaptable concurrency con- this naming is maintained is insignificant to this paper. The trol [11] suggested three approaches for dealing with vari- database may be either completely or partially replicated, ous system and transaction’s states: generic state, convert- or it may be partitioned on different nodes. ing state, and suffix sufficient state. The generic state BHARGAVA: CONCURRENCY CONTROL IN DATABASE SYSTEMS 15
A distributed database is consistent if it satisfies some 12233121 RxWyWzWyWzWxRzRxii[] [] i [] i [] i [] i []ii [] [] predefined assertions about the intrinsic characteristics of 1 the data values. For a replicated distributed database, it is is Wi [x ] since it is the latest higher level access to any entity necessary for the physical copies of the same database en- (x in this case). tity on different nodes to remain identical. The concurrent activities of a distributed database sys- The user actions on a distributed database consists of a tem can be modeled as a sequence of all atomic opera- sequence of atomic operations. An atomic operation is repre- tions. This sequence is called the history of the system, and σ = j S p sented by iiAx[ ], where i is a unique identification for a is represented by a quadruple h = < D, T, , >, where D transaction, j is a unique identification for a node, A is ei- is a distributed database, T is the transaction set, S is ther R or W representing read or write operation, and x is the atomic operation set, and p is a permutation function one or more logical database entities. As far as the DDBMS which gives the permutation indices for atomic opera- is concerned, these read/write operations constitute indi- tions s in h(s ¶ S). For example, if a history h is the fol- visible (or atomic) operations to the database. The atomic lowing sequence operations are grouped into logical units called transactions abg L w that will preserve the database consistency if executed p a p b ¤ p w S alone. A transaction can be viewed as a quantum change for then ( ) = 1, ( ) = 2, , ( ) = | |. A serial history is one the database from one consistent state to another; however, in which each transaction runs to completion before the the consistency assertions may be temporarily violated next one starts. In other words in a serial history the atomic during the execution of a transaction but must be satisfied operations of different transactions are not interleaved. when there are no incomplete transactions or the system is Although the system’s activities can be modeled as a quiescent. The purpose of the concurrency control is to string of atomic operations, the activity at one node is guarantee that the concurrent execution of a set of transac- potentially independent of those at other nodes. Each node records its own history. To capture this notion of local tions does not result in an inconsistent database state. j j j activities, the node projection h =