Advanced Operating Systems

COMP9242 Introduction Staff

• Lecturer in Charge – Gernot Heiser • Lecturer – Kevin Elphinstone • Various Support Staff –TBA

COMP9242 03s2 2 Why are you here?

• You’ve done comp3231 – Did well (minimum credit) – You would like delve deeper into issues of modern OS construction – You’d like more challenging projects to really get your hands dirty • Curious about where research is heading in the field of operating systems. • Thinking about doing a thesis project in operating systems

COMP9242 03s2 3 What can you expect?

• Challenging Project • Lectures in general: – Background required for project – Exposure to local research projects – An in-depth look at OS issues • Building upon the background in COMP3231 – Exposure to recent and seminal research papers – Guest lectures by active researchers (PhD students and local researchers)

COMP9242 03s2 4 Project Goal

Provide students with a deeper understanding of Operating Systems through practical experience.

• Approach: Participate in the design and implementation of a simple operating system (SOS).

COMP9242 03s2 5 Project Aims • Provide experience in OS design and development, including: – Microkernel based systems (L4::Pistachio). • User-level OS servers. • User-level page fault handlers. – Device drivers – Performance evaluation – Implications of cache architectures – Exposure to alternative OS Designs • Demonstrate the importance of design • Provide experience of being a team member in a large software project.

COMP9242 03s2 6 Project Aims

• Expose students to a mostly realistic OS development environment. – Similar to professional OS and or embedded systems developer. • Give an understanding of what’s involved in constructing an entire OS on bare hardware. • Give an understanding of the interaction between low-level software and hardware. • Encourage you to undertake a thesis, or do research within Distributed Systems Group.

COMP9242 03s2 7 Prerequisites

• Students are expected to be very competent C programmers. • Students are expected to be familiar with – basic computer architecture concepts. – Assembly language (read-only) – Basic RISC processor characteristics (we’ll use a MIPS R4600 for the project)

COMP9242 03s2 8 Lectures (subject to change) • Introduction and Overview • Introduction to the L4 Microkernel – L4 system calls and usage (to get you started on the project) • A close look at selected OS issues – Protection, capabilities – Caching, and its implications for OS – Page tables for wide address spaces – SMP issues: locking, cache coherence, scheduling – File systems

COMP9242 03s2 9 Lectures (subject to change) • Microkernels and User-level Servers – History and motivation for microkernel systems, Hydra, Mach, discussion, experiences; second- generation microkernel systems, L4, Exokernel, Spin; design and implementation of microkernel-based systems, including user-level page fault handling and device drivers • Microkernel Implementation – A detailed look at a real microkernel (L4Ka::Pistachio). • Persistent systems and Single-address-space operating systems – Concepts and examples; UNSW Mungi project COMP9242 03s2 10 Project/Lab Work • Build a simple operating system (SOS) from the ground up • Major component of the course • Use L4Ka::Pistachio – ported to MIPS here by Carl van Schaik • Develop and test on U4600 computers – R4600 based machines design and built by Kevin Elphinstone and Dave Johnson – “Clean” machine to get your hands dirty • Can also use CPU simulator Simular – Developed locally – Demos must be on real hardware COMP9242 03s2 11 Project

• Some warm-up experiments • Students will work in groups of two • End goal: – To produce a small efficient operating system • Project will have a series of due milestones – Demo to pass the milestone and be awarded marks – Help you manage your time – Avoid major problems

COMP9242 03s2 12 Milestones

• Details released RSN (week 2) • Late milestones – Less than one week late, 25% of the milestone mark is lost – More than one week late, but still less than two weeks late, 50% of the milestone mark lost

COMP9242 03s2 13 Alternative Projects

• Special arrangements might be made for particular student to do alternative projects – Must be at least as challenging as the original project – Must convince us that you can actually do it.

COMP9242 03s2 14 Assessment

• 65% for project work • 35% for final exam – A minimum of 14 (40%) required in final exam to pass

• Final Exam – 24hr take-home exam – Read and analyse two recent research papers and submit a critical report

COMP9242 03s2 15 Textbook

• No particular textbook for course – See course web page for useful reference books • Selected research papers referred to in the course

COMP9242 03s2 16 L4 and Microkernels

Background documents windows threads symbols coroutines stacks & heaps Application modules arrays & procedures structures statements variables

Monolithic Kernel Semaphore Priority Pipe File Address Space IPC Event Segment ACL Page Socket Process Monitor Task Mutex Thread Schedule

Bit Byte Word Register Instructions COMP9242HW 03s2 18 Monolithic Kernels - Advantages

• Kernel has access to everything, potentially: – All optimizations are possible – All techniques/mechanisms/concepts are implementable • Can be extended by simply adding more code to the kernel

COMP9242 03s2 19 Linux Kernel Evolution

Linux Kernel Size (.tar.gz) For reference: Linux 2.4.18 = 2.7 35 million lines of code

15 Size (Mb)Size 10

0 31-Jan-93 28-Oct-95 24-Jul-98 19-Apr-01 14-Jan-04 Date COMP9242 03s2 20 Approaches to tackling complexity • Monolithic approaches – Layered Kernels – Modular Kernels – Object Oriented Kernels • Alternatives – Extensible Kernels – Microkernels

COMP9242 03s2 21 Server File

documents windows threads symbols coroutines stacks & heaps Application modules arrays & procedures structures statements variables

Address Space Thread Bit Byte Word Register Instructions µ-kernel HW

COMP9242 03s2 22 History

• monolithic kernels

COMP9242 03s2 23 History

• monolithic kernels

•1st- generation µ-kernels – Mach CMU, OSF External Pager

– Chorus Inria, Chorus – Amoeba Vrije Universiteit – (L3) User-Level Driver GMD User-Level Driver

COMP9242 03s2 24 Brief History

• monolithic kernels

•1st- generation µ-kernels – Mach CMU, OSF External Pager

– Chorus Inria, Chorus – Amoeba Vrije Universiteit – (L3) GMD User-Level Driver •2nd- generation µ-kernels

– (Spin) U Washington

– Exokernel MIT – L4 GMD / IBM / UKa

COMP9242 03s2 User-Level Address Space25 classic + thin specialized classic Security em- native OS bedded highly-specialized Java RT app component MM

L4 L4 L4

HW HW HW

COMP9242 03s2 Project

sosh sosh Security

SOS

COMP9242 03s2 The Great Promise

• coexistence of different • APIs • file systems • OS personalities • flexibility • extensibility • simplicity • maintainability • security • safety

COMP9242 03s2 28 TheThe Big Great Disaster Promise

• coexistence of different • APIs • file systems • OS personalities • flexibility • SLOW • UNFLEXIBLE • extensibility • LARGE • simplicity • maintainability • security • safety

COMP9242 03s2 29 Macro µ Macro µ

PerformancePerformance FlexibilityFlexibilty

Simplicity TheThe 100100--µµss DisasterDisaster TheThe 100100--µµss DisasterDisaster

25 MHz 386 50 MHz 486 90 MHz Pentium 133 MHz Alpha IPC Costsµs (486, / ipc 50 MHz)

400

300

200

100 raw move

0 0 1000 2000msg len 3000 4000 5000

COMP9242 03s2 IPC Costs (486, 50 MHz)

Mac 400

100

300

200 50

100 raw move

0 0 0 50msg len 100 150 0 1000 2000msg len 3000 4000 5000

COMP9242 03s2 L4 IPC 0.04 µs (P III 500 MHz) (hopefully)

250 0.47 µs (P III 500 MHz)

200 0.36 µs (P III 500 MHz)

[cycles]

150 140 0.73 µs (Pentium 166 MHz) 120 236 23 100 0.91 µs (R4600 100 MHz) 100 16 180 80

60 55 0.10 µs (21164 433 MHz) 50 40 82 38 20 36 20 7 0 0 Pent III P3 Sysops P3 Lipc ? COMP9242Pentium 03s2 R4600 Alpha 36 Thesis:

• A µ-Kernel does the Job • if Properly Designed • if Carefully Implemented

Minimality Efficiency Elegance Flexibility Architectural Integration

COMP9242 03s2 37 When analyzing IPC performance,

Cycles are not the only the to consider!!

COMP9242 03s2 38 Processor-DRAM Gap (latency)

µProc 1000 CPU 60%/yr. “Moore’s Law” 100 Processor-Memory Performance Gap: (grows 50% / year) 10 DRAM

Performance DRAM 7%/yr. 1 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1999 2000 1992 1993 1994 1995 1996 1997 1998 Time Slide originally from Dave COMP9242 03s2 39 Patterson, Parcon 98 Today’s Situation: Microprocessor

• Microprocessor-DRAM performance gap – time of a full cache miss in instructions executed 1st Alpha (7000): 340 ns/5.0 ns = 68 clks x 2 or 136 2nd Alpha (8400): 266 ns/3.3 ns = 80 clks x 4 or 320 3rd Alpha (t.b.d.): 180 ns/1.7 ns =108 clks x 6 or 648 – 1/2X latency x 3X clock rate x 3X Instr/clock ⇒ ≈5X

Slide originally from Dave COMP9242 03s2 40 Patterson, Parcon 98 Cache Working Sets

L1 cache 98.83% • 1024 cache lines (16K + 16K) 1.17% • 12 lines used for IPC

L2 cache • 8192 cache lines (256K) • 12 lines used for IPC

99.85%

0.15%

COMP9242 03s2 41 Other Complications

• P4 trace cache – A cache of recently translated µ-ops – Flushed on every page-table switch • Virtual Caches – Need to be flushed on address space switch

COMP9242 03s2 42 50% Mach 486

45% Mach Alpha Chorus 486 40% Spin Alpha L4 Pentium 35%

30%

25%

20% overhead due to IPC

15%

10%

0% 100 200 400 800 1600 3200 6400 12800 25600 51200 102400 204800

average cycles between successive IPCs

COMP9242 03s2 Α µ-kernel does no real work. µ-Kernel services are only required to overcome µ-kernel constraints.

Therefore, µ-kernels have to be infinitely fast!

Minimality is the key!

• Threads IPC • Address Spaces Mapping

COMP9242 03s2 44 Address Threads, Communication Spaces

Threads

COMP9242 03s2 45 Drivers at User Level

INTR User

Driver Device

= ipc • IO ports: part of the user address space • interrupts: messages from hardware

COMP9242 03s2 46 Address Spaces

COMP9242 03s2 47 Address Spaces

• map • unmap • grant

COMP9242 03s2 48 Address Spaces

• map • unmap • grant

COMP9242 03s2 49 Address Spaces

• map • unmap • grant

COMP9242 03s2 50 Page Fault Handling

"PF" msg

Application Pager

map msg

COMP9242 03s2 51 Page Fault Handling

PF IPC "PF" msg

Application Pager

res IPC map msg

COMP9242 03s2 52 Address Spaces

Physical Memory COMP9242 03s2 53 Address Spaces

Initial AS

Physical Memory COMP9242 03s2 54 Address Spaces

Pager 1 Pager 2

Initial AS

Physical Memory COMP9242 03s2 55 Address Spaces

Pager 4

Pager 3

Pager 1 Pager 2

Initial AS

Physical Memory COMP9242 03s2 56 Address Spaces Application

Application Application

Application Pager 4

Pager 3

Pager 1 Pager 2

Initial AS

Physical Memory COMP9242 03s2 57 Address Spaces Application

Application Application

Application Pager 4

Driver

Pager 3 Driver

Pager 1 Pager 2

Initial AS

Physical Memory COMP9242 03s2 58 Mach Virtual Memory In comparison Application

Application Application

External Pager

Physical Memory Mach Paging Policy COMP9242 03s2 59 Size Comparison

Linux (all platforms): 2.7 Million lines

L4Ka::Pistachio/ia32 10,000 lines Mach 4 x86: 90,000 lines

COMP9242 03s2 60 classic + thin specialized classic Security em- native OS bedded highly-specialized Java RT app component MM

LN LN LN

HW HW HW

COMP9242 03s2