SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 2 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.2 4 Operating System Components
• Process management Chapter 12 • Main memory management Summary • File management • I/O management • Secondary storage management • Networking • Security • Command Interpreter
SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.1 3 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.3 5 Concurrency UNIX Flavours
A concurrent system consists of a set (with at least two elements) of threads of control or processes that UNIX
• execute (essentially) independently BSD System V • may access common resources OpenBSD Ultrix • may communicate with each other SunOS 4 NetBSD Linux HPUX = Solaris 1.4 Solaris 2 OSF1 FreeBSD IRIX AIX MacOS X True64 Unix POSIX SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.4 6 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.6 8 Bash Summary man bash (ctd.)
• Complex language • Redirection: command args < infile > outfile 2>> errfile • Context-sensitive lexing • Aliases: “The rules […] are somewhat confusing”. • Complete imperative control structures “For almost every purpose, aliases are superseded by shell • Mostly dynamic binding (static binding with local) functions.” • Iterated expansion mechanisms — functional flavour • Functions: local variables need to be declared local • Arithmetic evaluation, conditional expressions • Concise syntax for command-line interaction • Simple command expansion, command execution • Shell scripts need documentation! environment • Shell scripts need robustness! • Signals, job control • Shell scripts need security awareness! • Prompting, readline, history, history expansion • Shell builtin commands
SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.5 7 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.7 9 man bash Expansion
• Options: shell mode, and set options • Brace expansion: a{d,c,b}e → ade ace abe • Invocation: shell modes, initialisation, sh emulation • Tilde expansion: Home directories ~user and ~ • Shell Grammar: simple commands, pipelines, lists, • Parameter and variable expansion: $varname compound commands, function definitions with default: $varname:-default • Quoting: special characters in different contexts various string manipulations possible • Parameters: • Command substitution: $(command) or `command` • Arithmetic expansion: $((expression)) – positional parameters (arguments) • Process substitution: (obtaining FIFOs for processes) – special parameters $[*@#?-$0_] • Word splitting: results of parameter expansion, command – shell variables $BASH*, $HOSTNAME, $PWD, $HOME, $PS1 substitution, and arithmetic expansion are split into words – user-defined variables (outside "...") – array variables • Pathname expansion (globbing): dir*/file?.[coh] • Expansion: “seven kinds”! • Quote Removal SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.8 10 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.10 12 Disk Organization File Tables in UNIX In-memory Inode Table • One boot control block per disk — information for booting • For each open file a single copy of its file control blocks an OS from that disk. • Acts as cache for file control blocks • Several partitions: System File Table – partition control block: partition size, block size, block • one entry per active open management data structures (free block count, free bolck • contains reference to in-memory inode pointer, free FCB count, free FCB pointer) • current position, access rights, access mode – directory structure File Descriptor Table – File Control Blocks (FCBs) — in UNIX: inode: • one per process — copied by fork – Ownership, premission information • contains reference to system file table entry • In user space, references to file descriptors are int indexes – Location of data blocks into this table. • file locks — not copied by fork
SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.9 11 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.11 13 inodes in Detail Building Software Packages From Source
– mode (permissions) • source-code only – number of hard links — look for main module, config.h; read; compile… – owner, group • README-based – timestamps: modification mtime, access atime, change — read README, INSTALL; follow instructions ctime • Makefile-based – size (in bytes) – edit configuration part of Makefile – number of blocks allocated – make all – Pointers to allocated blocks: – make install (as root) – 12 direct block pointers to the first data blocks of the file • configure-based (GNU auto-tools) – one single indirect pointer, points to a block containing – ./configure --help — notice enable* and with* options pointers to the next (blocksize / pointersize) data blocks – ./configure --prefix=/usr/local/packages/XYZ – one double indirect pointer – make — sometimes separately: make doc html ps – one triple indirect pointer – make install (as root) SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.12 14 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.14 16 Process Management Overview Context Switch Steps
•A process is a program in execution (Assuming a user process is interrupted) • The CPU is shared among the processes ready to run – CPU senses interrupt – Only one process can run at a time – CPU starts the interrupt handler in privileged mode – (Interrupt handler may disable all interrupts) – The CPU is rapidly switched between processes — “context switch” – Interrupt handler stores state of interrupted process – Interrupt handler executes (kernel) code for the • Processes communicate with the OS using system calls interrupt • A process can be interrupted at any time by a device – Interrupt handler ends by calling the CPU scheduler interrupt or a system call – CPU scheduler loads the state of the process it selects • A process is represented by a data structure called a process – (Interrupt handler would re-enable interrupts) control block (PCB) or a process descriptor – CPU switches back to user mode and executes selected process
SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.13 15 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.15 17 Context Switching Process Scheduling
A context switch saves the state of the current running • All processes are on the job queue / process table process and then loads the state of the next process to – Some ready processes may be swapped to disk. be executed. – Processes in memory that are ready to execute are also on the ready queue It needs to save enough information about the current – Processes waiting for an I/O device are also on its device running process so that it can be resumed later as if queue nothing had happened. This determines important • The job scheduler selects processes from the job queue to components of the PCB. load into and out of memory A context switch is initiated by an interrupt: • When the CPU is free, the CPU scheduler selects one process from the ready queue to be executed by the CPU – software interrupt (system call) – A process can run until a timer interrupt or some other – device interrupt interrupt occurs – timer interrupt (quantum expired) – The CPU scheduler is called after the processing of each interrupt SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.16 18 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.18 20 Unix-Style Process Creation Signal Lifecycle, Long Version
Process creation in Unix involves four system calls: 1. A signal is generated and directed to a process 2. If the process ignores this signal, the signal is discarded • fork: Creates an exact copy of the calling process — KILL, STOP cannot be ignored – Returns process id of child to parent 3. If the process blocks this signal, it is kept pending until the process unblocks it – Returns 0 to child — KILL, STOP cannot be blocked • execve: Overwrites the calling process’s program 4. Otherwise, the signal is delivered to the process 5. Once delivered, the signal must be handled • exit: Causes the calling process to terminate – The default action is to terminate the process; • the process is stopped by STOP (^Z), TTIN, TTOU, – Returns process id to waiting processes TSTP (^S — type ^Q to continue) • CONT continues the process; CHLD is ignored • wait: Causes the calling process to wait until one of its – The default action is not taken if a signal handler has been children exits installed for catching the signal – Returns process id of exited child to process — KILL, STOP cannot be caught
SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.17 19 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.19 21 Interruption by Signals Pipes
• Signals are a special kind of IPC “messages” • Pipes are kernel data structures for inter-process • A process can install signal handlers communication • (Most) unhandled signals terminate the process • int pipe(int filedes[2]); • Signals can arrive at any time • Linux: pipe creates a pair of file descriptors, pointing to a • Signals can interrupt certain system calls — EINTR pipe inode, and places them in the array pointed to by filedes. #include
prw------1 kahl users 0 Jan 30 00:08 /tmp/fifo1– a set L of action labels δ : ( × × ) • Data is passed though the FIFO by the kernel without – a transition relation IP S L S . writing it to the file system. Example: • Normally, opening the FIFO blocks until the other end is = ( { , }, ,{ , }, opened also. LightSwitch1 dark light dark on off • When a process tries to write to a FIFO that is not opened for {(dark, on, light), (light, off, dark)}) read on the other side, the process is sent a SIGPIPE signal. off darkon light onoff
SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.21 23 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.23 25 Avoiding Suspension on Individual read and write Calls Concurrent Composition
Problem: • A system composed of several processes has a state that is • Normal read and write block until I/O possible composed from the states of the individual processes. think talk • Program may need to do other things while I/O impossible Converse = 0 1 2 thinktalk • Program may need perform I/O where it first becomes scratch possible Itch = a b scr Different solutions: 0 think 1 talk 2 • Open with O_NONBLOCK and “poll manually” a a a • Use select scratch scratch scratch Converse || Itch = • Use poll think talk 0b 1b 2b • Open with O_ASYNC and perform I/O in signal handlers scrthinktalk While Converse and Itch have only one trace each, their • Use multiple threads (carefully …) composition has three, representing arbitrary interleaving. SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.24 26 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.26 28 Liveness and Safety Properties Non-Deterministic Choice, Traces, and Composition
Coin2 and Coin3 have the same trace set! A safety property asserts: But, Coin2 || Bet and Coin3 || Bet have different trace sets! ⇒ “something bad will never happen” Two LTSs P1 and P2 are equivalent iff for every LTS Q, the compositions P1 || Q and P2 || Q have the same trace set. This is a black-box view: “No context enables distinction.” A liveness property asserts: tails “something good will eventually happen” heads = 2 = toss Bet HTtails Coin 0 1 headstails tossheadstails heads heads tails Coin3 = 1 0 2 tossheadstails toss toss
SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.25 27 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.27 29 Branching Transitions Fairness Fairness assumption: A state of a process from which several transitions exist Q0 usually models one of the following: open stair stair If a choice is arrived at – In this state, the process is prepared to react to different Q4 infinitely often, then all environmental stimuli enter close Q1 of its branches are taken – In this state, the process acts by making a Q7 Q5 infinitely often. (non-deterministic) choice close updown Assuming fairness, • non-determinism could be intended up Q8 Q6 up down down up down additional lifeness • non-determinism could be the result of abstraction Q9 properties hold, e.g.: LTSs do not differentiate between action and reaction! exit “After an enter, there Q2 will eventually be an
stairstair exit.”
Q3 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.28 30 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.30 32 Threads Multithreading Models
• Threads are “light-weight processes” • Many-to-one: Several user threads to one kernel thread – Threads of a process managed as a unit • Threads are “processes inside a process” – Used for multithreading in absence of kernel threads • Threads share process data structures • One-to-one: One user thread to one kernel thread – Threads of a process managed independently • Threads need to synchronise – Can be costly because users can cause many kernel • Many synchronisation primitives are geared towards threads to be created threads • Many-to-many: Several user threads to several kernel threads • In C: thread libraries – More flexibility than many-to-one • In Java: threads are part of the language – Less costly than one-to-one – Used for multithreading on a multiprocessor
SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.29 31 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.31 33 Threads vs. Processes Thread Programming Styles
Different ways to organise multithreaded programs: spawning forking Reactive: sibling child • Thread runs in an infinite loop • Continuously checks for certain events to occur joining waiting • Responds to the events when they occur shared memory copied memory Task oriented: shared resources copied resource access • Parent checks all relevant events • Parent thread creates a child thread to do a task various synchr. methods signals, pipes • Parent is notified when task is completed • Child terminates when task is completed SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.32 34 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.36 38 Process Synchronization — Background Data Exclusion
• Concurrent access to shared data may result in data • Data exclusion is the act of preventing access to data inconsistency. – Opposite of data sharing • Maintaining data consistency requires mechanisms to • Examples of data access that needs to be prevented: ensure the orderly execution of cooperating processes. – Hidden side effects • Race condition: The situation where several processes – Violations of data invariants access and manipulate shared data concurrently. The – Breached atomic actions final value of the shared data depends upon which process – Unauthorized accesses finishes last. • To prevent race conditions, concurrent processes must be synchronized.
SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.33 35 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.45 47 Mutual Exclusion Data Exclusion Approaches
An intended atomic action can be breached (fundamental • Immutable data structures safety failure) if processes (or threads) share data structures — you cannot change the data Two approaches for preventing breached atomic actions, based on mutual exclusion: • Side-effect-free sub-programs • At most one process can be in a critical section at a given — you will not change the data time • Data confinement • At most one process can be modifying a shared data structure at a given time — you cannot get at the data • Mutual exclusion These are special cases of process synchronization — you cannot interfere over the data The two main devices for implementation: • Semaphores • Monitors SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.53 55 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.62 64 Immutable Data Structures Semaphores
• A data structure is immutable if its state cannot be changed • The notion of a semaphore was invented by E.W. Dijkstra • Benefits: (The Structure of the “THE”-Multiprogramming System, – Local assumptions satisfied by an immutable data 1968) structure are invariants of the data structure • Provide two services: – Immutable data structures can be safely shared – Mutual exclusion • State changes realized by creating new immutable – Interprocess or interthread signaling structures • Two basic kinds: • Two kinds of immutable objects: – Stateless objects having no fields whatsoever –A binary semaphore serves as a resource access key – Objects having only “final” fields –A counting semaphore serves as a resource availability • Fully (and partially) immutable data structures should be measure used as much as possible • Semaphores reduce the general problem of breached atomic – Values in an abstract data type should be immutable actions to a simpler problem
SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.61 63 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.63 65 Data Confinement POSIX Condition Variables • Confinement can be used to ensure that only one thread can • Are declared independent of any mutex access a given data structure at a time • Each condition variable must be used together with always – Similar to security measures used to ensure information the same mutex privacy — programmer responsibility! – Analysis of data flow is crucial • Mechanisms for enforcing data confinement: pthread_mutex_lock(&m); pthread_mutex_lock(&m); – Information hiding: Interface is public; implementation while ( x ≠ y ) x++; is private pthread_cond_wait(&v,&m); pthread_cond_signal(&v); pthread_mutex_unlock(&m); – Lexical scoping: References to data structures are only /* modify visible in restricted portions of code x or y pthread_mutex_lock(&m); – Access control: permissions for object access required if necessary */ y−−; pthread_mutex_unlock(&m); – Calling sequence: Procedures are called in an order that pthread_mutex_unlock(&m); pthread_cond_signal(&v); excludes certain accesses to data SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.64 66 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.76 78 Condition Variables versus Semaphores Java Synchronization
Semaphore Condition Variable • Objects are a kind of monitor – Each object has a mutual exclusion lock wait decrements if positive waits always – A thread must obtain the lock before it can execute a ≤ 0 waits only if synchronized method or block of code signal increments if no no-op if no waiting – Unsynchronized methods can be executed at any time waiting wakes up one waiting • Each object has one unnamed condition variable, a wait wakes up one waiting set, and wait and signal methods – Wait methods: three versions of wait – Signal methods: notify and notifyAll – wait, notify, and notifyAll can only be called from within synchronized methods or blocks – Signal-and-continue approach is used, but it is not specified which thread in the waiting set is resumed
SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.75 77 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.77 79 Another Synchronisation Mechanism: Barriers notify() and wait()
A barrier is a synchronisation point: • obj.wait() causes current thread to wait until another thread • no process (or thread) passes the barrier before every other invokes a notify method for obj process has arrived at the barrier, too – can only be called inside a block synchronized on obj – releases lock on obj • Alternative: only some fixed number of processes is – can only continue after lock on obj has been re-acquired required to pass the barrier – waiting can be interrupted, causing InterruptedException • Related formalism: Petri nets: – overloaded variants wait(...) allow to specify time-out – implemented using wait set • obj.notifyAll() causes all threads in the wait set of obj to be runnable again – calling thread still has lock on obj and continues – awakened threads compete for lock on obj • obj.notify() wakes up only one thread – should only be used if this is known to be safe! SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.82 84 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.91 93 Method-Based Confinement Techniques Object-Based Confinement Techniques
A method creates an object and then uses one of the following • Adapters: An object with unsynchronized methods is techniques to confine it: wrapped with an object with synchronized methods • Local confinement: The object’s reference is not allowed to • Subclassing: Unsynchronized methods of a class are leave the scope of the method synchronized in a subclass • Tail call hand-off: The object’s reference is handed off in a • Transfer protocol: A resource object is passed between a tail call to another method group of objects according to a protocol • Call by value:The object’s value is given in a call to another – Example: Token ring method, i.e., – a copy of the object is passed, or – information needed to reconstruct the object is passed. • Trust: The object’s reference is given in a call to another method which is trusted not to modify the object or forward its reference.
SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.86 88 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.92 94 Thread-Based Confinement Techniques Summary
• Thread confinement: All fields accessible within a thread • Shell Programming are confined to the thread: • File Management – The thread behaves like a process with its own address • UNIX Processes: fork, wait, exec space. • POSIX signals: synchronous and asynchronous, handling • IPC via (named) pipes – Supported by java.lang.ThreadLocal. • Concurrency modelling using LTSs • Thread hand-off: The object references of a thread T • Implementing concurrency inside processes: Threads ′ are handed off to another thread T in a tail call in T’s run • Process and thread synchronisation method. – Critical sections — mutual exclusion • Current thread confinement: All thread object methods – Semaphors and monitors — waiting for a simple lock access the fields of the current thread object. – Condition variables — waiting for satisfaction of a complex condition • Scheduling SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.93 95 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.108 110 Goals Become A Power-User • Use a Real Editor: • Know what to expect from a computer – Work through the Emacs tutorial – Learn the basics of vi; check out vim • Know what not to expect from a computer – Check whether you like Emacs or XEmacs better • Improve your command-line skills! • Know what can go wrong, and why – Find out how to define shell functions and aliases – Do it! • Know what the public expects from your software • Document preparation: • Know how to achieve what you need – not only WYSIWIG, i.e., “what you see is all you get” – Check out LaTeX, Lout, and DocBook • Read about Literate Programming – Try out FunnelWeb or NoWeb
SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.94 96 SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.116 118 Interaction with a Computer Welcome to the World of Free Software
Different interaction paradigms: • Install some free software from source • Find out about the tools involved in the process • Point-and-click • Write complete, user-friendly makefiles for some project – High intuitivity (sometimes) • Read about GNU and open source licences – Low expressivity and abstraction capabilities • Find out about the differences between GNU/Linux, GNU/Hurd, FreeBSD, NetBSD, OpenBSD, Darwin, • Command line — linguistic Solaris – High expressivity, abstraction capabilities • Check out some technical mailing lists or USENET news – High intuitivity — once you know the language groups • See how problems can be attacked — attack your own! SFWR ENG 3BB4 — Software Design 3 — Concurrent System Design 12.118 120 More Skills
• Read the Documentation!
• Read the Questions!