PhysicalPhysical SchemaSchema DesignDesign Storage structures Indexing techniques in DBS 1 PhysicalPhysical Design:Design: GoalGoal andand influenceinfluence factorsfactors Physical schema design goals — Effective data organization Dr A. Hinze, of University Waikato, NewZealand,2006, COMP329 :Database Administration — Effective access to disc — Effectiveness = performance Quality Measures — Throughput: # transactions / sec — Turn-around-time: time for answering an individual query (e.g. average) Parameters — Application dependent factors: size of database, typical operations, frequency of operations, isolation level — System related factors: storage of data, access path, index structures, optimization, … 2 PhysicalPhysical Design:Design: StorageStorage DevicesDevices Memory Hierarchy: DBMS Dr A. Hinze, of University Waikato, NewZealand,2006, COMP329 :Database Administration Archive storage Tertiary storage Secondary storage Disk Main memory Primary storage Cache 3 PhysicalPhysical Design:Design: StorageStorage DevicesDevices DBMS Tertiary storage Disk Cache Main memory Dr A. Hinze, of University Waikato, NewZealand,2006, COMP329 :Database Administration — Fastest, most costly form of storage Cache — Size very small — Usage managed by OS/Hardware Main Memory — Data available to be operated on — Too small for entire DB — Data lost after power failure or a system crash 4 PhysicalPhysical Design:Design: StorageStorage DevicesDevices DBMS Tertiary storage Disk Disk Main memory Dr A. Hinze, of University Waikato, NewZealand,2006, COMP329 :Database Administration — Where entire DB is stored Cache — Direct access (not sequential) — Data operations: disk -> main memory -> disk — (Usually) survives power failures/system crashes Tape storage (tertiary storage) —Cheaper than disk — slower access: tape read sequentially from the beginning (sequential-access storage). — Backups and archival data — Used for recovery from disk failures — Less complex than disk => more reliable 5 PhysicalPhysical Design:Design: StorageStorage DevicesDevices Access time vs capacity: Dr A. Hinze, of University Waikato, NewZealand,2006, COMP329 :Database Administration Bytes Y 13 Tertiary storage 12 Capacity in 10 11 Secondary storage 10 9 Main memory 8 Zip disk 7 6 Floppy disk Cache 5 2 1 0 -1 -2 -3 -4 -5 -6 -7 -8 -9 Access time in 10X sec Source: Garcia-Molina, Ullman, Widom “Database systems”, 2002 6 PhysicalPhysical Design:Design: SecondarySecondary storagestorage Mechanics Disk heads Cylinder Dr A. Hinze, of University Waikato, NewZealand,2006, COMP329 :Database Administration Platter = 2 surfaces track gap sector 7 PhysicalPhysical Design:Design: SecondarySecondary storagestorage “Typical” Characteristics — Rotation: 5400 –10,000 RPM Dr A. Hinze, of University Waikato, NewZealand,2006, COMP329 :Database Administration — Capacity: 360 KB – 100+ GB — Price (per GB): $2 - $2.5 — Platter diameter: 2.5 to 25 cm — Platters: 1 - 15 — Tracks (per surface): 40 – 20,000 — Sector size: 512 B – 50 KB — Sectors (per track): 32-500 Holds: Virtual memory (handled by OS) File system (access via OS) DBMS stuff (bypass OS) 8 PhysicalPhysical Design:Design: SecondarySecondary storagestorage Block: — Neighboring byte sequence on single track of one platter Dr A. Hinze, of University Waikato, NewZealand,2006, COMP329 :Database Administration — Smallest data unit moved between disk and main memory — Databases work with blocks — Fixed size — Size determined by physical properties of disk + OS — Size: from 512 to 4096 bytes track gap sector blocks 9 PhysicalPhysical Design:Design: I/OI/O costcost Data transfer disk - main memory Dr A. Hinze, of University Waikato, NewZealand,2006, COMP329 :Database Administration —In blocks — Bytes transferred at constant speed — Information flow (if): between 120 Kb/s and 5 Mb/s Seek time: Time for repositioning the arm to right cylinder Move disk heads to the right cylinder: Start (constant), Move (variable), Stop (constant) 0 if arm there, otherwise long (between 8 to 10 ms) Track-to-track seek time: 0.5ms –2ms 10 PhysicalPhysical Design:Design: I/OI/O costcost Rotate time (disk latency): — Time until interesting sector positioned under the head Dr A. Hinze, of University Waikato, NewZealand,2006, COMP329 :Database Administration — Access to all data within a cylinder within rotate time — 12 to 6 ms per rotation — Average: 6 to 3 ms rotational latency. — => store related information close by Transfer time (read time): Depends on # bytes to be transferred Total time to transfer T bytes: Seek time + Rotational time + T/if 11 PhysicalPhysical Design:Design: I/OI/O costcost Typical access time: Disk access time = SeekTime 6 ms Dr A. Hinze, of University Waikato, NewZealand,2006, COMP329 :Database Administration + RotateTime 3 ms + TransferTime 1 ms Seek time dominates ! Compare: RAM 3-10 nsec Random Disk / RAM: ~10 * 10-3 / 10 * 10-9 = 106 ~ Distance Hamilton - moon vs. distance Hamilton – Raglan Sequential disk read ("scan") much faster 12 PhysicalPhysical Design:Design: I/OI/O costcost Disk characteristics: year Capacity $/GB Scan Scan Dr A. Hinze, of University Waikato, NewZealand,2006, COMP329 :Database Administration GB Sequential Random 1988 0.25 20,000 2 min 20 min 1998 18 50 20 min 5 hrs 2003 200 5 2 hrs 1.2 days Source: The Rebirth of Database Machines; talk by Bitton/Gray at VLDB'98 Random access favorable only if small percentage of the data accessed frequently Rule of thumb: less than 15 % on a large table 13 PhysicalPhysical Design:Design: I/OI/O costcost The Myth: seek time dominates The Reality: Wait Dr A. Hinze, of University Waikato, NewZealand,2006, COMP329 :Database Administration 1. Queuing dominates (Queuing) 2. Transfer dominates BLOBs 3. Disk seeks often short Implication: many cheap servers better than one fast expensive server Transfer — Shorter queues — Parallel transfer Rotate — Lower cost/access and cost/byte Seek Source: “Storage: Alternate Futures”;Futures” talk by J.Gray at NetStore99, Seattle 14 PhysicalPhysical Design:Design: I/OI/O costcost Goal: Accelerate access to secondary storage Dr A. Hinze, of University Waikato, NewZealand,2006, COMP329 :Database Administration Strategies 1. Place blocks that are accessed together on same cylinder (avoids seek time) 2. Divide data between smaller disks (independent heads increase # block accesses) 3. Mirror a disk: simultaneous access to several blocks 4. Disk-scheduling algorithm: selects order of block access 5. Prefetch blocks in main memory Strategy depends on situation/application # processes, parallel access, shared disks, (un)predictable queries, budget,… 15 PhysicalPhysical Design:Design: RAIDRAID RAID Technology — Redundant array of independent disks; originally redundant Dr A. Hinze, of University Waikato, NewZealand,2006, COMP329 :Database Administration array of inexpensive disks — Reduces access time and queue length — Fault tolerance by "Parity disks" — Principle technique: striping A B C D E Large disk: F G H Long queue, Long transfer RAID 0: A E B F C G D H Striping (interleaving) 16 PhysicalPhysical Design:Design: RAIDRAID RAID levels: Dr A. Hinze, of University Waikato, NewZealand,2006, COMP329 :Database Administration RAID 1 Mirroring and Duplexing: mirror without stripping A B ====A B C D C D E F E F G H G H RAID 0+1 High Data Transfer Performance A E ====A E B F B F C G C G D H D H 17 PhysicalPhysical Design:Design: RAIDRAID RAID levels: Dr A. Hinze, of University Waikato, NewZealand,2006, COMP329 :Database Administration RAID 2 Byte (Bit) level striping + error correcting disks (A0..A3) = word A (B0..B3) = word B A0 B0 A1 B1 A2 B2 A3 B3 EA EB RAID 3 Block stripping with parity Byte level A0 B0 A1 B1 A2 B2 A3 B3 PA PB stripe 0 stripe 1 stripe 2 stripe 3 18 PhysicalPhysical Design:Design: RAIDRAID RAID levels: Dr A. Hinze, of University Waikato, NewZealand,2006, COMP329 :Database Administration RAID 4 Independent Data disks with shared Parity disk Block level A0 B0 A1 B1 A2 B2 A3 B3 PA PB block 0 block 1 block 2 block 3 RAID 5 Independent Data disks with distrib. parity blocks A0 A1 P2 B0 P1 B2 P0 C1 C2 A block B block C block 19 PhysicalPhysical Design:Design: RAIDRAID Levels 0, 1, 2, 3, 4, 5, 6, 7, 10, 1+0, 53 Dr A. Hinze, of University Waikato, NewZealand,2006, COMP329 :Database Administration RAID controller provides OS / DBS with standard disk interface Performance: — gains for read operations — Writes need recomputation of parity: Main cause of parity disk bottleneck in RAID-4 architecture Further info: http://www.raid.com 20 PhysicalPhysical Design:Design: StorageStorage ofof DBDB datadata Database data — Stored in disk files Dr A. Hinze, of University Waikato, NewZealand,2006, COMP329 :Database Administration — Files provided by the underlying OS File: — Logical sequence of records — Records mapped onto disk blocks Records — Composed of fields (byte sequences for attributes) —Fixed or variable length — BLOBs stored separately in pool of disk blocks — Record points to BLOB 21 PhysicalPhysical Design:Design: StorageStorage ofof DBDB datadata Goal: Mapping the database to files Dr A. Hinze, of University Waikato, NewZealand,2006, COMP329 :Database Administration Problem: — Blocks with fixed size but varying record sizes (tuples of distinct relations of different size) Approaches: 1. Use several files and store records of only one fixed length in any given file 2. Structure the files such that they can deal with multiple lengths for records (more difficult than fixed length). 22 PhysicalPhysical Design:Design: StorageStorage ofof DBDB