Course Outline • Prerequisites – COMP2011 Data Organisation • Stacks, queues, hash tables, lists, trees, heaps,…. Introduction – COMP2021 Digital Systems Structure • Assembly programming • Mapping of high-level procedural language to assembly COMP3231/9201/3891/9283 language – or the postgraduate equivalent (Extended) Operating Systems – You are expected to be competent Kevin Elphinstone programmers!!!! • We will be using the C programming language – The dominant language for OS implementation.

– Need to understand pointers, pointer arithmetic, explicit 2 memory allocation.

Why does this fail? Lectures • Common for all courses (3231/3891/9201/9283) void func(int *x, int *y) • Wednesday, 2-4pm { • Thursday, 5-6pm *x = 1; *y = 2; – All lectures are here (EE LG03) – The lecture notes will be available on the course web site } • Available prior to lectures, when possible. void main() – The lecture notes and textbook are NOT a substitute for { attending lectures. int *a, *b; func(a,b); }

3 4

Tutorials Assignments • Assignments form a substantial component of • Start in week 2 your assessment. • A tutorial participation mark will • They are challenging!!!! – Because operating systems are challenging contribute to your final assessment. • We will be using OS/161, – Participation means participation, NOT – an educational attendance. – developed by the Systems Group At Harvard – Comp9201/3891/9283 students excluded – It contains roughly 20,000 lines of code and comments • You will only get participation marks in your enrolled tutorial.

5 6

1 Assignments Assignments • Assignments are in pairs • Don’t under estimate the time needed to do the – Info on how to pair up available soon assignments. • We usually offer advanced versions of the – ProfQuotes: [About the midterm] "We can't keep you working assignments on it all night, it's not OS.“ Ragde, CS341 – Available bonus marks are small compared to amount of • If you start a couple days before they are due, you will be late. effort required. • To encourage you to start early, – Student should do it for the challenge, not the marks. – Bonus 10% of max mark of the assignment for finishing a – Attempting the advanced component is not a valid excuse week early for failure to complete the normal component of the – To iron out any potential problems with the spec, 5% bonus assignment for finishing within 48 hours of assignment release. • Extended OS students (COMP3891/9283) are – See course handout for exact details • Read the fine print!!!! expected to attempt the advanced assignments

7 8

Assignments Assignments

• Four assignments • Late penalty – due roughly week 3, 6, 9,13 – 4% of total assignment value per day • The first one is trivial • Assignment is worth 20% • You get 18, and are 2 days late – It’s a warm up to have you familiarize • Final mark = 18 – (20*0.04*2) = 16 (16.4) yourself with the environment and easy marks. • Assignments are only accepted up to – Do not use it as a gauge for judging the one week late. 8+ days = 0 difficulty of the following assignments.

9 10

Assignments Plagiarism

• To help you with the assignments • We take cheating seriously!!! – We dedicate a tutorial per-assignment to • Penalties include discuss issues related to the assignment – Copying of code: 0 FL – Prepare for them!!!!! – Help with coding: negative half the assignment’s max marks – Originator of a plagiarised solution: 0 for the particular assignment – Team work outside group: 0 for the particular assignments

11 12

2 Cheating Statistics Exams

Session 1998/S1 1999/S1 2000/S1 2001/S1 2001/S2 2002/S1 2002/S2 2003/S1 2003/S2 enrolment 178 410 320 300 107 298 156 333 133 • There is NO mid-session suspected cheaters 10(6%) 26(6%) 22(7%) 26(9%) 20(19%) 15(5%) ???(?%) 13 (4%) ???(?%) • The final written exam is 2 hours full penalties * * * * 2 6 9 14 109521 reduced penalties 7157754229 • Supplementary exams are oral. cheaters failed 4 101616101254 ? cheaters – Supplementaries are available according to suspended 001 001 000 school policy, not as a second chance.

*Note: Full penalty 0 FL not applied prior to 2001/S1 13 14

Assessment 3891/9201/9283

• Exam Mark • Class Mark • No tutorial participation component Component Component • Assignment marks scaled to 100 – Max mark of 100 – Max mark of 100 • Based solely on the • 10% tutorial final exam participation • 90% Assignments

15 16

Undergrad Assessment Postgrads (9201/9283) • Maximum of a 50/50 weighted harmonic • The final assessment is the harmonic mean and a 20/80 harmonic mean mean of the exam and class – Can weight final mark heavily on exam if you can’t commit the time to the assignments component. – You are rewarded for seriously attempting the • If >= 40, assignments 2EC • if E >= 40, 2EC 5EC M = M = max E+C, E+4C E + C ³ ´ 17 18

3 Harmonic Mean (Class Mark = 100 - Exam Mark)

Assessment 100

90 • If E < 40 80 70

60 Harm 50/50 Arith 50/50 50 Arith 20/80  2EC  Final Mark 40 Harm 20/80 M = min44,  30 20  E + C  10 0 0 102030405060708090100 Exam Mark

19 20

Final Mark = 50

100 Assessment

90 80 • You need to perform reasonably 70 consistently in both exam and class 60 Harm 50/50 components. 50 Harm 20/80 40 • Harmonic mean only has significant 30 effect with significant variation. 20 Exam Mark Required to Pass to Required Exam Mark • Reserve the right to scale, and scale 10

0 courses individually if required. 0 102030405060708090100 – Warning: We have not scaled in the past. Class Mark

21 25

Textbook References

• Andrew • A. Silberschatz and P.B. Galvin, Operating System Concepts, 6th ed., Addison Wesley (2001) Tanenbaum, • William Stallings, Operating Systems: Internals and Design Modern Operating Principles, 4th edition, 2001, Prentice Hall. nd • A. Tannenbaum, A. Woodhull, Operating Systems--Design and Systems, 2 Implementation, Prentice Hall, ??? Edition, Prentice • John O'Gorman, Operating Systems, MacMillan, 2000 Hall • Uresh Vahalla, UNIX Internals: The New Frontiers, Prentice Hall, 1996 • McKusick et al., The Design and Implementation of the 4.4 BSD Operating System, Addison Wesley, 1996

26 27

4 Consultations/Questions Course Outline

• Questions should be directed to the forum. • Processes and threads • Admin related queries to Charles Gray [email protected] • Concurrency control • Personal queries can be directed to me [email protected] • Memory Management • We reserve the right to ignore email sent directly to • File Systems us (including tutors) if it should have been directed to the forum. • I/O and Devices • Consultation Times • Security – TBA • Scheduling

28 29

What is an Operating System? Introduction to Operating Systems

Chapter 1 – 1.3

31

Viewing the Operating System as an Abstract Machine • Extends the basic hardware with added functionality • Provides high-level abstractions – More programmer friendly – Common core for all applications • It hides the details of the hardware – Makes application code portable

32 33

5 Disk Users Viewing the Operating System as a Resource Manager

Memory • Responsible for allocating resources to users and processes • Must ensure CPU – No Starvation – Progress Network – Allocation is according to some desired policy Bandwidth • First-come, first-served; Fair share; Weighted fair share; limits (quotas), etc… – Overall, that the system is efficiently used

34 35

Dated View: the Operating The Operating System is System as the Privileged Privileged Privileged Mode Component • Applications should not be able to interfere or bypass the operating system Operating System – OS can enforce the “extended machine” – OS can enforce its resource allocation policies – Prevent applications from interfering with each other Requests Applications • Note: Some Embedded OSs have no privileged (System Calls) component, e.g. PalmOS User Mode – Can implement OS functionality, but cannot enforce it. Applications Applications • Note: Some operating systems implement significant OS functionality in user-mode, e.g. User-mode 36 37

Why Study Operating (A brief) Operating System History Systems? • Largely parallels hardware • There are many interesting problems in development operating systems. • First Generation machines • For a complete, top-to-bottom view of a – Vacuum tubes system. – Plug boards • Programming via wiring • Understand performance implications of • Users were simultaneously application behaviour. designers, engineers, and programmers • Understanding and programming large, • “single user” complex, software systems is a good skill to • difficult to debug (hardware) acquire. – No Operating System

38 39

6 Second Generation Machines Batch System Batch Systems Operating Systems • IBM 7094 • Sometimes called “resident job monitor” – 0.35 MIPS, 32K x 36-bit memory • Managed the Hardware – 3.5 million dollars • Simple Job Control Language (JCL) • Batching used to more efficiently use the – Load compiler hardware – Compile job – Share machine amongst many users – Run job – One at a time – End job – Debugging a pain • Drink coffee until jobs • No resource allocation issues finished 40 – “one user” 41

Problem: Keeping Batch Third Generation Systems - Systems Busy Multiprogramming • Divided memory among several • Reading tapes or punch cards was time Memory consuming loaded jobs • Expensive CPU was idle waiting for input • While one job is loading, CPU works on another Job 1 • With enough jobs, CPU 100% busy Job 2 • Needs special hardware to isolate memory partitions from each other Job 3 – This hardware was notably absent on early 2nd gen. batch systems OS 42 43

Multiprogramming Example Job turn-around time was still an issue. • Batch systems were well suited to – Scientific calculations – Data processing • For programmers, debugging was much easier on older first gen. machines as the programmer had the machine to himself. • Word processing on a batch system? 44 45

7 Time sharing An then…

• Each user had his/her own terminal • Further developments (hardware and connected to the machine software) resulted in improved techniques, • All user’s jobs were multiprogrammed concepts, and operating systems….. – Regularly switch between each job – CAP, , , UNIX V6, BSD UNIX, THE, , Sprite, Accent, UNIX SysV, Linux, EROS, – Do it fast KeyKOS, OS/360, VMS, HPUX, Apollo Domain, • Gives the illusion that the programmer Nemesis, L3, L4, CP/M, DOS, Exo-kernel, Angel, has the machine to himself Mungi, BE OS, Cache Kernel, Choices, , Inferno, Grasshopper, MOSIX, Opal, SPIN, VINO, OS9, • Early examples: Compatible Time Plan/9, QNX, Synthetix, Tornado, x-kernel, Sharing System (CTSS), MULTICS VxWorks, Solaris………. 46 47

Operating System Time Line The Advent of the PC

• Large Scale Integration (LSI) made small, fast(-ish), cheap computers possible • OSs followed a similar path as with the mainframes – Simple “single-user” systems (DOS) – Multiprogramming without protection, (80286 era, Window 3.1, 95, 98, ME, etc…, MacOS <= 9) – “Real” operating systems (UNIX, WinNT, MacOS X etc..)

48 49

Operating Systems

• Exploit the hardware available Computer Hardware • Provide a set of high-level services that Review represent or are implemented by the hardware. Chapter 1.4 • Manages the hardware reliably and efficiently • Understanding operating systems requires a basic understanding of the underlying hardware 51

8 Basic Computer Elements Basic Computer Elements • CPU – Performs computations – Load data to/from memory via system bus • Device controllers – Control operation of their particular device – Operate in parallel with CPU – Can also load/store to memory (Direct Memory Access, DMA) – Control register appear as memory locations to CPU • Or I/O ports – Signal the CPU with “interrupts” • Memory Controller – Responsible for refreshing dynamic RAM – Arbitrating access between different devices and CPU

52 53

The real world is logically similar, A Simple Model of CPU but a little more complex Computation • The fetch-execute cycle

54 55

A Simple Model of CPU A Simple Model of CPU Computation Computation • Stack Pointer • The fetch-execute cycle CPU Registers CPU Registers • Status Register – Load memory contents from – Condition codes address in program counter • Positive result PC: 0x0300 (PC) PC: 0x0300 • Zero result SP: 0xcbf3 • The instruction SP: 0xcbf3 • Negative result Status – Execute the instruction Status • General Purpose Registers R1 – Increment PC R1  – Holds operands of most – Repeat  instructions Rn Rn – Enables programmers to minimise memory references. 56 57

9 ‘Safe’ registers and Privileged-mode Operation CPU Registers instructions • To protect operating system Interrupt Mask • Registers and instructions are safe if execution, two or more CPU Exception Type – Only affect the state of the application itself modes of operation exist MMU regs – Privileged mode (system-, Others – They cannot be used to uncontrollably kernel-mode) PC: 0x0300 interfere with • All instructions and registers are SP: 0xcbf3 • The operating system available Status • Other applications – User-mode R1 • Uses ‘safe’ subset of the – They cannot be used to violate a correctly instruction set  – E.g. no disable interrupts Rn implemented operating system policy. instruction

• Only ‘safe’ registers are 58 59 accessible

Privileged-mode Operation I/O and Interrupts Address Space • I/O events (keyboard, mouse, incoming network • The accessibility of 0xFFFFFFFF packets) happen at unpredictable times Accessible only • How does the CPU know when to service an I/O addresses within an to event? address space Kernel-mode changes depending 0x80000000 on operating mode – To protect kernel code Accessible to and data User- and Kernel-mode

0x00000000 60 61

Interrupts Interrupt Cycle • An interruption of the normal sequence of execution • Processor checks for interrupts • A suspension of processing caused by an event • If no interrupts, fetch the next instruction external to that processing, and performed in • If an interrupt is pending, divert to the such a way that the processing can be resumed. interrupt handler • Improves processing efficiency – Allows the processor to execute other instructions while an I/O operation is in progress – Avoids unnecessary completion checking (polling)

62 63

10 Classes of Interrupts Interrupt Handler

• Program exceptions • A program that determines the nature of (also called synchronous interrupts) the interrupt and performs whatever – Arithmetic overflow actions are needed. – Division by zero – Executing an illegal/privileged instruction • Control is transferred to the handler by – Reference outside user’s memory space. hardware. • Asynchronous (external) events • The handler is generally part of the – Timer operating system. – I/O

– Hardware or power failure 64 65

Simple Interrupt Processing Simple Interrupt Device controller issues an interrupt User SP & PC User state is restored Application loaded from User Mode Processor finishes kernel stack current instruction and changes to kernel mode Interrupt is processed including t R p et acknowledging ur Processor masks rru n the interrupt at the Processor unmasks te fro interrupts and In m transfers flow of device level interrupts and returns In t control (new PC) to user mode.

Kernel Mode Remainder of user Kernel SP loaded state is saved user SP & PC pushed on kernel stack Interrupt Software Hardware Handler Hardware 66 67

Multiple Multiple Interrupts Interrupts • Prioritised (nested) interrupts • Sequential interrupts – Processor ignores any new lower-priority – Processor ignores interrupt signals any new interrupt – New higher-priority signals interrupts interrupt the – Interrupts remain current interrupt handler pending until current – Example: when input interrupt completes arrive from a communication line, it – Upon completion, needs to be absorbed processor checks for quickly to make room for additional interrupts more input

68 69

11 Programmed I/O Interrupt-Driven I/O

• Also called polling, or busy • Processor is interrupted when waiting I/O module (controller) ready to • I/O module (controller) exchange data performs the action, not the • Processor is free to do other processor work • Sets appropriate bits in the I/O • No needless waiting status register • No interrupts occur • Consumes a lot of processor time because every word read • Processor checks status until operation is complete or written passes through the processor – Wastes CPU cycles 70 71

Multiprogramming Direct memory access (DMA) • I/O exchanges occur directly (Multitasking) with memory • Processor has more that one program • Processor directs I/O controller to read/write to memory to execute. • Relieves the processor of the – Some tasks waiting for I/O to complete responsibility for data transfer – Some tasks ready to run, but not running • Processor free to do other • Interrupt handler can switch to other things tasks when they become runnable • An interrupt is sent when the task is complete • Regular timer interrupts can be used for timesharing 72 73

Memory Hierarchy • Going down the Memory Hierarchy hierarchy • Rough approximation of memory hierarchy – Decreasing cost per bit – Increasing capacity – Increasing access time – Decreasing frequency of access to the memory by the processor • Hopefully • Principle of locality!!!!! 74 75

12 Cache Processor-DRAM Gap Word Transfer Block Transfer (latency) CPU Cache Main Memory Registers µProc 1000 CPU 60%/yr. “Moore’s Law” • Cache is fast memory placed between the CPU and main memory – 1 to a few cycles access time compared to RAM access time of tens – hundreds of cycles 100 Processor-Memory • Holds recently used data or instructions to save memory accesses. Performance Gap: • Matches slow RAM access time to CPU speed if high hit rate (grows 50% / year) • Is hardware maintained and (mostly) transparent to software 10 • Sizes range from few kB to several MB. DRAM • Usually a hierarchy of caches (2–5 levels), on- and off-chip. Performance DRAM 7%/yr. • Block transfers can achieve higher transfer bandwidth than single 1 words. 1987 1997 1981 1983 1988 1989 1990 1991 1992 1993 1994 1995 1996 1998 1999 1980 1982 1984 1985 1986 – Also assumes probability of using newly fetch data is higher than the 2000 probability of reuse ejected data. Time 76 77

Slide originally from Dave Patterson, Parcon 98

Cache size affect on performance Moving-Head Disk Mechanism

78 79

A Strategy: Avoid Waiting for Example Disk Access Times Disk Access • Disk can read/write data relatively fast • Keep a subset of the disk’s data in – 15,000 rpm drive - 80 MB/sec memory – 1 KB block is read in 12 microseconds • Access time dominated by time to locate the ⇒ Main memory acts as a cache of disk head over data contents – Rotational latency • Half one rotation is 2 milliseconds – Seek time • Full inside to outside is 8 milliseconds • Track to track .5 milliseconds • 2 milliseconds is 164KB in “lost bandwidth” 80 81

13 Two-level Memories and Hit Effective Access Time Rates • Given a two-level memory,

– cache memory and main memory (RAM) Teff = H T1 +(1 H) (T1 + T2) × − × – main memory and disk T1 = access time of memory 1 what is the effective access time? T2 = access time of memory 2 H = hit rate in memory 1

Teff = effective access time of system • Answer: It depends on the hit rate in the first level.

82 83

Example • Cache memory access time 1ns • Main memory access time 10ns • Hit rate of 95%

T =0.95 1 10 9 + eff × × − 0.05 (1 10 9 +10 10 9) × × − × − =1.5 10 9 × −

84

14