Crash Recovery Chapter 18 Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 1 Review: The ACID properties A tomicity: All actions in the Xact happen, or none happen. C onsistency: If each Xact is consistent, and the DB starts consistent, it ends up consistent. I solation: Execution of one Xact is isolated from that of other Xacts. D urability: If a Xact commits, its effects persist. The Recovery Manager guarantees Atomicity & Durability. Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 2 Motivation Atomicity: . Transactions may abort (“Rollback”). Durability: . What if DBMS stops running? (System crashes or media failures) Desired Behavior after crash! system restarts: T1 – T1, T2 & T3 should be T2 durable. T3 – T4 & T5 should be T4 aborted (effects not seen). T5 Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 3 Assumptions Concurrency control is in effect. Strict 2PL, in particular. Updates are happening “in place”. i.e. data is overwritten on (deleted from) the disk. A simple scheme to guarantee Atomicity & Durability? Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 4 ARIES When the recovery manager is invoked after a crash, restart proceeds in three phrases: . Analysis: identifies dirty pages and active transactions . Redo: Repeats some actions necessary . Undo: Undoes actions which did not commit. Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 5 Principles Write-Ahead Logging . Changes are written ahead in a stable storage. Repeating History During Redo . When restart, bring system back to the state when crash happened. Logging Changes During Undo . Changes during undo is also logged. Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 6 Basic Idea: Logging The log is a history of actions executed by the DBMS. Physically, the log is stored in stable storage to survive crashes. The most recent portion of the log, the log tail, is kept in memory and is periodically forced to stable storage. Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 7 Basic Idea: Logging Record REDO and UNDO information, for every update, in a log. Sequential writes to log (put it on a separate disk). Minimal info (diff) written to log, so multiple updates fit in a single log page. Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 8 Log Record Every log record has a unique id – Log Sequence Number (LSN) and LSN always increases. It could be the memory address of the log record. For recovery, every page in the database has the LSN of the most recent log record that describes a change to this page – pageLSN. Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 9 Log Records A linked list of one transaction LogRecord fields: Possible log record types: prevLSN Update XID Commit type Abort End Additional Fields Compensation Log Records (CLRs) . for UNDO actions Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 10 Additional Fields of Log Record Commit (Commit record) . Transaction id is included and it force writes the log tail to a stable storage. Abort (Abort record) . Transaction id is included. Undo is initiated for the transaction. End (End record) . After additional actions after Abort/Commit. The transaction id is included. Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 11 Update Log Records LSN: <prevLSN, transID, type, pageID, length, offset, before-image, after-image> pageID: modified page id. length: length of the change in bytes offset: offset of the change before-image: the value of the changed bytes before the change (for undo) after-image: the value after the change (for redo) Redo-only update vs. undo-only update Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 12 Transaction Table One entry for each active transaction. Each entry includes . transaction id, . the status (in progress, committed, or aborted), . lastLSN (the LSN of the most recent log record for this transaction). transID Status lastLSN T100 In progress 4 T200 In progress 3 Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 13 pageID recLSN Dirty Page Table P500 1 P600 2 P505 4 Dirty page: pages with changes not reflected on disk. One entry for each dirty page in the system’s buffer pool. Each entry includes . recLSN (the LSN of the first log record that caused the page dirty, which is also the earliest log record to be redone) Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 14 pageID recLSN P500 1 Dirty Page Table P600 2 P505 4 LSN Prev transID Type pageID Length Offset Before- After- LSN image image 1 null T100 Update P500 3 21 ABC DEF 2 null T200 Update P600 3 41 HIJ KLM 3 2 T200 Update P500 3 20 GDE QRS 4 1 T100 Update P505 3 21 TUV WXY LOG transID Status lastLSN T100 In progress 4 Transaction Table T200 In progress 3 Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 15 Write-Ahead Logging (WAL) Protocol The Write-Ahead Logging Protocol: 1. Must write the log record for an update before the corresponding data page gets to disk. (Guarantee Atomicity) 2. Must write all log records to stable storage for a Xact before commit. (Guarantee Durability) WAL is the fundamental rule to ensure a record of every change to the database is available while recovering a crash. Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 16 The Big Picture: What’s Stored Where LOG RAM DB LogRecords prevLSN Xact Table XID Data pages lastLSN type each status pageID with a length pageLSN Dirty Page Table offset recLSN before-image Master Record after-image flushedLSN . System keeps track of flushedLSN, The max LSN flushed so far. Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 17 Transaction DB RAM Commits LSNs pageLSNs flushedLSN System keeps track of flushedLSN. Log records WAL: Before a page is written, flushed to disk . pageLSN flushedLSN Write commit record to log. All log records up to Xact’s lastLSN are flushed. Guarantees that flushedLSN lastLSN. Note that log flushes are sequential, synchronouspageLSN writes to disk. “Log tail” in RAM . Many log records per log page. Write end record to log. Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 18 Simple Transaction Abort For now, consider an explicit abort of a Xact. No crash involved. We want to “play back” the log in reverse order, UNDOing updates. Get lastLSN of Xact from Xact table. Can follow chain of log records backward via the prevLSN field. Before starting UNDO, write an Abort log record. • For recovering from crash during UNDO! Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 19 Abort, cont. To perform UNDO, must have a lock on data! Before restoring old value of a page, write a CLR: . You continue logging while you UNDO!! . CLR has one extra field: undoNextLSN • Points to the next LSN to undo (i.e. the prevLSN of the record we’re currently undoing). CLRs never Undone (but they might be Redone when repeating history: guarantees Atomicity!) At end of UNDO, write an “end” log record. Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 20 Example of CLR LSN Prev transID Type pageI Length Offset Before- After- LSN D image image 1 Null T100 Update P500 3 21 ABC DEF 2 Null T200 Update P600 3 41 HIJ KLM 3 2 T200 Update P500 3 20 GDE QRS 4 1 T100 Update P505 3 21 TUV WXY To undo the last Update, a CLR is written as LSN: <prevLSN, transID, type, pageID, length, 5 : < 4 , T100 , CLR , P505 , 3 , offset, before-image, after-image, undoNextLSN> 21 , WXY , TUV , 1 > Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 21 Checkpointing Periodically, the DBMS creates a checkpoint, in order to minimize the time taken to recover in the event of a system crash. Write to log: . begin_checkpoint record: Indicates when chkpt began. end_checkpoint record: Contains current Xact table and dirty page table. This is a `fuzzy checkpoint’: • Other Xacts continue to run; so these tables accurate only as of the time of the begin_checkpoint record. • No attempt to force dirty pages to disk; effectiveness of checkpoint limited by oldest unwritten change to a dirty page. (So it’s a good idea to periodically flush dirty pages to disk!) . Store LSN of chkpt record in a safe place (master record). Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 22 Crash Recovery: Big Picture Oldest log rec. of Xact Start from a checkpoint (found active at crash via master record). Smallest Three phases. Need to: recLSN in dirty page – Figure out which Xacts table after committed since checkpoint, Analysis which failed (Analysis). – REDO all actions. Last chkpt • (repeat history) – UNDO effects of failed Xacts. CRASH A R U Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 23 Recovery: The Analysis Phase Analysis phase identifies . Where to start the Redo pass (the point in the log) . What are dirty (pages in the buffer pool at the time of crash). Who are active ( transactions should be undone). Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 24 Recovery: The Analysis Phase Reconstruct state at checkpoint. Initializing D.P.T. and Xact Table via end_checkpoint record. Scan log forward from checkpoint. End record: Remove T from Xact table. Other records: Add T to Xact table, set lastLSN=LSN, change T status to Last commit/To be Undone. chkpt . Update record: If P not in Dirty Page CRASH Table, add P to D.P.T., set its A R U recLSN=LSN. Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke 25 Recovery: The Analysis Phase At the end, transaction table has a list of all transactions with status U (they were active while crash).