CS 4310: Operating Systems Lecture Notes - Student Version∗ Kyle Burke January 10, 2018 Contents -1.0 Using Chapel . 2 0 OS Basics 2 0.1 Interrupts . 2 1 Parallel Programming (using Chapel) 2 2 Hardware Threads 2 3 Concurrency 3 4 Semaphores 4 5 The Producer-Consumer Problem 4 5.1 Circular Queues . 4 6 Memory Management 6 7 Stack vs. Heap 8 7.1 Stack Management . 9 7.2 Heap Management . 9 8 Scheduling 9 8.1 Shortest-Job First . 10 8.2 First Come, First Serve . 13 8.3 Earliest-Deadline First . 14 8.4 Round Robin . 15 8.5 Hybrid Schedulers . 15 8.6 Examples . 15 9 Interrupts 20 10 File Systems 24 10.1 Fragmentation . 27 11 OS Security 31 12 History of OSes by Candace 31 ∗Created with lectureNotes.sty, which is available at: http://turing.plymouth.edu/~kgb1013/lectureNotesLatexStyle.php (or, GitHub: https://github.com/paithan/LaTeX-LectureNotes). Many or most of the answers to questions are hidden so that some of class will still be a challenge for students. 1 -1.0 Using Chapel 2 HARDWARE THREADS -1.0 Using Chapel This course was last taught with programming assignments given in Chapel1 using compiler version 1.14. This is a High-Performance Computing language designed to make parallel programming easier for computational scientists. Here are some comments about this language. • It makes launching threads and handling synchronization very easy. • It is missing many of the basic libraries that exist for common languages (e.g. Java). • Goal: focus on the 0 OS Basics 〈 Go over syllabus! 〉 Q: What are the responsibilities of an Operating System? What do they do? A: • TODO 0.1 Interrupts This material is currently in 9. Should we move it? 1 Parallel Programming (using Chapel) 2 Hardware Threads Q: How many hardware threads do your computers have? How many CPUs? (Might be different numbers.) Q: How can a CPU have multiple threads? A: TODO: add stuff here? Picture, maybe? 1http://chapel.cray.com © 2018 Kyle Burke 3 CONCURRENCY Q: What if you have more threads than that? A: Q: What are some of the parts of a PCB? ID Unique identification number for this process. State "New", "Ready", "Running", "Blocking", etc. Program Counter : Pointer to the address of the next line of code to execute. Registers Data that will be loaded into the registers when this executes. Process Scheduling State "Ready" or "Suspended". A: Could include info about the priority. Process Structuring Information IDs of any chil- dren processes. Interprocess communication info Links to any other processes or variables outside of this process that are relevant. Privileges What are the privileges of this process? (User/superuser, etc) And more! Q: Do PCBs go on the stack or the heap? A: 3 Concurrency 〈 Do a Dining Philosophers live-action example. Show a deadlock. (Tell everyone to pick up their right-hand fork first, then tell them all that they’re hungry.) 〉 〈 Talk about the following things: © 2018 Kyle Burke 5 THE PRODUCER-CONSUMER PROBLEM • Concurrent Threads/Processes • Race conditions • Critical Section code - code where race conditions can occur. • Mutual Exclusion (mutex) - property of concurrent programming where only one thread can execute its critical section at a time. • Lock: Actual mechanism to enforce mutual exclusion. We will use semaphores. 〉 Q: What are the four steps for a thread executing a critical section? 1. Non-critical: Code is not executing a critical section. 2. Try: Before entering a critical section, the thread requests access. A: 3. Critical section: The thread executes the critical section. No other threads are simultaneously exe- cuting their critical sections. 4. Exit: The thread exits the critical section, freeing up the lock. 4 Semaphores 〈 How can we use semaphores as locks? 〉 〈 Do dining philosophers again. 〉 Q: How many semaphores are we using? 5 The Producer-Consumer Problem 〈 Quickly describe the Producer-Consumer problem. 〉 Common solution: give them a buffer of items produced, but not yet consumed. 5.1 Circular Queues 〈 Explain need for a bounded-size buffer. 〉 〈 Buffer should be a queue. 〉 © 2018 Kyle Burke 5.1 Circular Queues 5 THE PRODUCER-CONSUMER PROBLEM 〈 Two common options: Linked List or Circular array. Quickly describe each. 〉 Q: What’s the pro for each? A: Q: How can we speed the circular array up? A: Q: When should I not be able to add to the circular array? A: When it is full. Q: What should happen to the thread that’s trying to add to it? A: Block! Q: Until when? A: Q: When should I not be able to remove from the circular array? A: © 2018 Kyle Burke 6 MEMORY MANAGEMENT Q: What should happen to a thread trying to remove something? A: Q: How can we enforce these blocking behaviors? A: Q: How many semaphores does each blocking queue need? A: 〈 Talk about whether 2 is enough! 〉 6 Memory Management OS Takes up some memory space. ("Linux Kernel") Remainder goes to the applications that will run on top of the OS. Q: How big are modern OS kernels? A: Memory Management Techniques • Single contiguous Allocation: all the memory is available to one application. (MSDOS, Single-purpose hardware.) © 2018 Kyle Burke 6 MEMORY MANAGEMENT • Partitioned Allocation: Memory is divided up into separate partitions (maybe by appli- cation or process). Need to allocate space when jobs/applications start and deallocate when they stop. • Paged Allocation: Memory is divided up into fixed size page frames. These pages do not need to be contiguous in RAM. Addresses: (page, offset). Hardware Memory Management Unit (MMU) is needed to convert page indices to actual RAM addresses. • Memory Segmentation: divided into variable-size segments. Addresses: (segment, off- set). MMU translates these pairs into physical addresses. (Implements Segment Table.) Segmentation Fault: Tried to access an offset too big for the given segment. Q: Pros and Cons. Which has most overhead? Which is most flexible? Which is most used? Q: What is Virtual Memory? A: 2 2VM Image source: https://commons.wikimedia.org/wiki/File:Virtual_memory.svg © 2018 Kyle Burke 7 STACK VS. HEAP Q: What is the distinction between page, page frame, and paging? • A page is a fixed-length piece of VM. A: • A page frame is a fixed-length piece of RAM. • Paging is process of exchanging pages from hard drive to RAM as needed. AKA swapping. Q: Can we have VM without paging? A: Q: Pros and Cons? Q: Can we use Segmentation technique with paging? A: Nice bonus of VM: each application/process could get it’s own contiguous space, a la Partitioned Allocation. Thus, each could have it’s own Stack and Heap. Note: may have to share some heap parts with other processes. 7 Stack vs. Heap "The Stack", also known as call stack, stores data about currently-running code. What does top stack frame look like? • Process Control Block up top (small) • Lines of code from module below • Data below that. • Other stack frames below that. © 2018 Kyle Burke 7.1 Stack Management 8 SCHEDULING 7.1 Stack Management 7.2 Heap Management 〈 Just go over the notes from CS 232. I should probably copy them over here, but I haven’t done that yet... 〉 8 Scheduling 〈 Talk about traffic lights, train tunnels, etc. 〉 Q: What are different goals for scheduling? A: Q: Okay, so what properties of a "Job" might a scheduler want to know? A: © 2018 Kyle Burke 8.1 Shortest-Job First 8 SCHEDULING Q: What are some possible ways you would choose which process gets to run next? What do you think are common scheduling algorithms? A: Q: What is preemptive? A: 8.1 Shortest-Job First SJF minimizes total waiting time and latency. Q: What’s the benefit of shortest job first? A: © 2018 Kyle Burke 8.1 Shortest-Job First 8 SCHEDULING Let’s say I have four processes: A, B, C, D. Each has the following running times: • A: 5 ms Q: • B: 10 ms • C: 7 ms • D: 3 ms How long does it take to run all the processes using SJF? (Assume negligible context switching.) A: Q: Is that any faster if I rearrange them? A: Q: So how does the order change the latency? I’ve still got to wait 25 ms to complete them all! A: Q: What’s the latency for each of the four processes using SJF? A: © 2018 Kyle Burke 8.1 Shortest-Job First 8 SCHEDULING Q: So average? A: Q: What if we aren’t using SJF and instead do them in the order D, A, B, C? What is the average latency now? A: Q: How long to add something to the scheduler? A: Q: How long to remove the next process to run? A: Q: What’s a big issue with SJF? What’s a problem with this algorithm? A: Starvation: One (or more) processes never get executed. © 2018 Kyle Burke 8.2 First Come, First Serve 8 SCHEDULING Q: How could this happen? A: Q: Could there be a preemptive version of SJF? Q: How does that work? 8.2 First Come, First Serve Q: Which property does FCFS optimize for? A: Q: How long to add something to the scheduler? A: Q: How long to remove the next process to run? A: Q: Could there be a preemptive version of this? A: © 2018 Kyle Burke 8.3 Earliest-Deadline First 8 SCHEDULING 8.3 Earliest-Deadline First Q: Which property does EDF optimize for? A: Q: What should you do if you can’t make a deadline? A: Q: How long to add something to the scheduler? A: Q: How long to remove the next process to run? A: Q: Could there be a preemptive version of this? A: © 2018 Kyle Burke 8.4 Round Robin 8 SCHEDULING 8.4 Round Robin Q: Which property does Round Robin optimize for? A: Q: How long to add something to the scheduler? A: Q: How long to remove the next process to run? A: Q: Could there be a preemptive version of this? A: 8.5 Hybrid Schedulers 〈 Talk about them.