Internal Report 09-13 August 2009 Universiteit Leiden Opleiding Informatica An Implementation of Intelligent Disk Block Replication in Minix Kristian Rietveld MASTER’S THESIS Leiden Institute of Advanced Computer Science (LIACS) Leiden University Niels Bohrweg 1 2333 CA Leiden The Netherlands i Abstract With ever increasing speeds of central processing units and main memory, the speed of disks lags behind. Reading from disks such as hard disks, floppy disks and DVDs is orders of magnitude slower than the computer’s main memory. Since the capacity of the main memory is limited and we cannot do without disks for large sets of data, there is a real need to optimize reading from disks as much as possible. One way to do this is by optimizing according to the usage of the computer system by the user. We describe a way to optimize accessing disks by monitoring operations that are done on the disk thereby discovering patterns in these operations. By exploiting the fact that reading blocks that are stored on the disk in a contiguous sequence is faster than reading blocks that are scattered over the disk, we will store the blocks that make up these patterns in a contiguous sequence at another place on the disk. We will show that such replication of related disk blocks indeed leads to improvements in disk read performance in certain situations. We have modified the Minix operating system kernel to be able to discover such patterns and support the replication of blocks on its native file system. A concise overview of the full Minix kernel and our modifications to it are given. iii Voorwoord Al sinds het begin van mijn studie Informatica ben ik uitgebreid ge¨ınteresseerd geweest in besturingssystemen. Ook na het vak “Operating Systems” bleef de hebzucht bestaan om meer te leren over deze fundamenten van elk computersysteem. Onder begeleiding van Andr´eDeutz ben ik initieel begonnen met het bestuderen van de Linux broncode. Dat bleek te groot en complex te zijn om ook maar gedeeltelijk te doorgronden. Daarop zijn we verder gegaan met het compleet doorgronden van de Minix broncode. Van het begrijpen van een compleet besturingssysteem heb ik vele nieuwe inzichten in deze systemen gekregen. Als eerste zou ik Andr´eDeutz willen bedanken voor zijn begeleiding tijdens dit project. Met hem heb ik vele interessante discussies gevoerd. Het idee om met Minix aan de slag te gaan en het daarop volgende project zijn van zijn hand. Daarnaast wil ik Harry Wijshoff bedanken voor het overwegen van deze scriptie als afstudeerscriptie. Achter dit werk zit een proces van meerdere jaren. Zonder steun van vele andere mensen was dit werk waarschijnlijk nooit afgekomen. Ik zou de volgende mensen graag willen danken voor hun steun, motivatie, interesse en afleiding in de laatste jaren. Thijs, Wouter en Pieter voor de vele, interessante, vaak technisch getinte discussies in de kroeg tijdens de weekenden. Vian, Ron en anderen uit de FooBar, voor interesse en vooral mentale steun. Dankzij hun kreeg ik vaak weer extra motivatie om door te gaan met mijn studie en deze scriptie af te ronden. Mijn huidige bazen, Mikael Hallendal en Richard Hult, voor het geven van een kans om voor hun te werken. Daarbij voor alle hulp en flexibiliteit om naast mijn werk mijn studie af te kunnen maken. Mijn ouders, vanzelfsprekend, voor alle steun tijdens het studeren. Op hun kan ik altijd rekenen en terugvallen. Mijn vriendin Ginny, zij heeft mij als geen ander weten te inspireren en motiveren om mijn studie en in het bijzonder deze scriptie tot een goed einde te brengen. Een groot gedeelte van deze scriptie is geschreven bij haar in Barcelona, weg uit het drukke en soms chaotische Nederland. Haar liefde, steun en interesse zijn onge¨evenaard. Zonder haar inspiratie had het maanden langer geduurd om dit werk tot een einde te brengen. Kristian Rietveld September, 2008 1 Contents 1 Introduction 7 I Detailed study of Minix 8 2 Basics of Operating Systems 8 2.1 TheKernel ...................................... 8 2.1.1 Monolithickernels .............................. 8 2.1.2 Microkernels................................. 9 2.1.3 A mix between micro and monolithic kernels . 9 2.2 Interfacing with the hardware . 10 2.2.1 Booting up . 10 2.2.2 Handling interrupts and exceptions . 10 2.2.3 Protection . 11 2.2.4 Contextswitches ............................... 11 2.3 Process management . 12 2.3.1 Scheduling .................................. 12 2.3.2 Managing the process’ memory . 13 2.3.3 Systemcalls.................................. 13 2.3.4 Signal handling . 14 2.3.5 Passing messages . 15 2.4 Managing files . 15 2.4.1 General file system structure . 15 2.4.2 Inodes and file descriptors . 15 2.4.3 Caching . 16 3 Minix in detail 17 3.1 Origins......................................... 17 3.2 TheMinixkernel................................... 17 3.2.1 Structure ................................... 17 3.2.2 Message passing . 18 3.2.3 PIDsversusendpoints ............................ 20 3.2.4 Scheduling .................................. 21 3.2.5 Interrupt and exception handling . 22 3.2.6 Theclocktask ................................ 23 3.2.7 Thesystemtask ............................... 23 3.2.8 System call invocation . 24 3.2.9 Locking . 24 3.3 Bootprocess ..................................... 25 3.4 The process manager . 26 3.4.1 Initialization . 26 3.4.2 Managing memory . 27 3.4.3 Creating new processes . 28 3.4.4 Signal handling . 30 3.4.5 Swapping . 33 2 3.5 Thefilesystemserver ............................... 33 3.5.1 Inodes..................................... 34 3.5.2 Thesuperblock ............................... 35 3.5.3 Accessingdiskblocks ............................ 35 3.5.4 General model of operation . 35 3.5.5 Mapping between processes and file descriptors . 36 3.5.6 Disk block management . 37 3.5.7 Examples of operations . 37 3.6 Device driver interface . 41 3.6.1 Overview of the interface . 41 3.6.2 Driver mappings . 42 3.7 Improving reliability . 42 3.7.1 Message passing and system call masks . 42 3.7.2 The reincarnation server . 43 3.7.3 The data store server . 43 3.8 Conclusions...................................... 44 3.8.1 Arealmicrokernel? ............................. 44 3.8.2 Drawbacks of the micro kernel approach . 44 3.8.3 Areas which can be improved . 45 3.8.4 An option for resource-limited computers? . 46 II Implementing disk block replication in Minix 47 4 Intelligent disk block replication 47 4.1 Current performance measures . 48 4.1.1 Zones . 48 4.1.2 Read ahead . 48 4.2 Ouridea........................................ 49 4.2.1 Contiguous versus scattered disk reads . 49 4.2.2 Logging disk accesses . 50 4.2.3 Finding related blocks . 52 4.3 Howtodomeasurements?.............................. 55 4.4 Results of measurements with the proof of concept implementation . 56 5 An implementation of disk block replication 63 5.1 General overview of the implementation . 63 5.1.1 The optimization server . 63 5.1.2 Modifications to the file system server . 66 5.1.3 Communication between both servers . 71 5.1.4 Problems during implementation . 72 5.2 Performance measurements . 72 5.2.1 Testing conditions . 72 5.2.2 Testresults .................................. 73 5.2.3 Othertests .................................. 75 5.3 Conclusions and further research . 77 3 6 Conclusions 79 Glossary 80 Bibliography 82 A Source code listing of read-blocks.c 83 B Source code listing of pattern-generator.pl 86 C Source code listing of discover-pairs.pl 90 D Source code listing of simulate-minix-cache.pl 92 E Source code listing of the kernel adaptions 98 5 List of Figures 1 Simplified overview of a process’ memory layout . 13 2 General structure of the Minix system . 18 3 Thekernel’smemorymap .............................. 27 4 The different signal handling code paths . 30 5 Overview of the stack during signal handler invocation . 32 6 Overview of inodes and indirect zones . 37 7 Seconds needed to read various amounts of blocks from a floppy. 50 8 Sample sequence demonstrating neighborhoods . 52 9 Overview of the “accesses” data structure. Note that not for each subscript a hashtablehasbeendrawn. ............................. 53 10 Area with 5 points chosen and blocks in their radii in gray. Here a radius of 1 isused. ........................................ 55 11 Results of testing with the “contiguous blocks” pattern with different pattern densities........................................ 58 12 Results of testing with the “common blocks” pattern with different numbers of candidates....................................... 59 13 Results of testing with the “contiguous blocks” pattern with different pattern densities using the fixed cache simulator . 60 14 Comparison between running the tests with and without write-back enabled . 61 15 A sample sliding window, only the numbers inside the rectangle are in memory 65 16 General layout of the file system including replicated blocks . 68 17 General layout of the info block . 69 18 Results of measurements using the “contiguous blocks” pattern. 74 19 Results of measurements using the “common blocks” pattern. 75 Introduction 7 1 Introduction In computer systems an operating system is at least as important as the Central Processing Unit (CPU). Of course, you are able to run simple programs on a computer system without making use of an operating system, but you will quickly become frustrated with writing tedious code for being able to use the computer’s keyboard, showing output on the display, etc. One of the main tasks of the operating system is abstracting the hardware away from programmers, who want to get the next pressed key in the keyboard buffer with one library call instead of doing the Input/Output (I/O) operations with the keyboard themselves. Microprocessors these days are quite ubiquitous and so are operating systems. Operating systems are being developed, or derived from an existing operating system, for various kinds of electronic equipment: DVD players, televisions, cell phones, video game consoles, etc. When designing new electronic devices with a CPU, or embedded systems, thinking about your operating system design is a very important.