Käyttöjärjestelmät

LUENTO 23

• Key problem: How do you initiate a system Booting, multimedia systems using only itself?

Exam issues • Booting <- Bootstraping • Bootstraping - why this term? • Baron Munchausen pulled himself out of swamp using his boot straps

1 2

View of the boot process BIOS tasks

• BIOS – Basic Input / Output System Power-up /Reset • BIOS refers to the firmware code run by a personal computer when first powered on. System startup BIOS/ Bootmonitor • BIOS can also be said to be a coded program embedded on a chip that recognizes and controls various devices that Stage 1 make up personal computers. (source: wikipedia)

Stage 2 bootloader LILO, GRUB, etc • Execution starts from a fixed location (on x386 that Kernel Linux location is 0xFFFF0000) • Check hardware, locate the boot device Operation Init -space • Disk, , ... • Load the first ’loader’ from Master Boot Record

Source: http://www-128.ibm.com/developerworks/library/l-linuxboot/index.html 3 4

BIOS loading the boot loader – BIOS example sequence pseudocode example

1. Check the CMOS Setup for custom • 0: set the P register to 8 2. Load the interrupt handlers and device drivers • 1: check paper tape reader ready 3. Initialize registers and power management • 2: if not ready, jump to 1 4. Perform the power-on self-test (POST) • 3: read a byte from paper tape reader to accumulator 5. Display system settings • 4: if end of tape, jump to 8 6. Determine which devices are bootable • 5: store accumulator to address in P register • 6: increment the P register 7. Initiate the bootstrap sequence • 7: jump to 1

5 6

Syksy 2006 / Luento 23 23-1 Käyttöjärjestelmät

Boot loader GRUB on disk

• Linux: Lilo, GRUB • Windows NT: NTLDR • Initial location on primary disk, for example: Master Boot Record

• Main task: Load and • Dual boot: offer menu and start the chosen one

7 8 http://www.pixelbeat.org/docs/disk/

GRUB - GRand Unified Bootloader Loading Linux

• GRUB is independent of any particular operating • Bootsector loads setup, decompression routines and system and may be thought of as a tiny, function- compressed kernel image. specific OS. • The kernel is uncompressed in protected mode. • It’s primary task is to load the actual operating system • Low-level initialisation is performed by asm code arch/i386/kernel/head.S: and pass control to it. • Initialise segment values and page tables. • GRUB is large service: • Enable paging by setting PG bit in %cr0. • It does not fit to MBR: needs to be handled in two phases • the first 2k of bootup parameters (kernel commandline). • Phase 1 (loaded from MBR by BIOS) load phases 1.5 and 2 • The first CPU calls start_kernel(), all others call initialize_secondary(). • High-level C initialisation • A lot of features: file systems, command line interface, chain-loading other boot loaders, ...

Source: http://www.faqs.org/docs/kernel_2_4/lki-1.html 9 10

OS Initialising: start_kernel() Process Init

• Arch-specific setup (memory layout analysis, copying boot command line again, etc.). • Init is the father of all processes. Its primary role • Initialise traps, irqs, data required for scheduler, keeping is to create processes from a script stored in the data, softirq subsystem. file /etc/inittab (man init) • Initialise console. • Enable interrupts. • Establishes and operates the entirety of user • Set a flag to indicate that a schedule should be invoked "next space. opportunity" • checking and mounting file systems, • Create a kernel thread init() • starting up necessary user services, and • Go into the idle loop, this is an idle thread with pid=0. • switching to a user-based environment when system Important thing to note here that the init() kernel thread calls do_basic_setup() which in turn calls do_initcalls() startup is completed • Uses etc/rc directory hierarchy detailed version available from: http://www.faqs.org/docs/kernel_2_4/lki-1.html 11 12

Syksy 2006 / Luento 23 23-2 Käyttöjärjestelmät

Runlevel Scripts Directory State etc/rc.d (Red Hat/Fedora Core) 0 /etc/rc.d/rc0.d/ /halt system 1 /etc/rc.d/rc1.d/ Single user mode • debian: run /etc/init.d/rcS which runs: 2 /etc/rc.d/rc2.d/ Multiuser with no network services • /etc/rcS.d/S* scripts exported • /etc/rc.boot/* (depreciated) 3 /etc/rc.d/rc3.d/ Default text/console only start. Full • run programs specified in /etc/inittab multiuser 4 /etc/rc.d/rc4.d/ Reserved for local use. Also X-windows (Slackware/BSD) • Scripts in /etc/rc*.d/* are symlinks to /etc/init.d 5 /etc/rc.d/rc5.d/ XDM X-windows GUI mode • Scripts prefixed with S will be started when the runlevel is (Redhat/System V) entered, eg /etc/rc5.d/S99xdm 6 /etc/rc.d/rc6.d/ Reboot • Scripts prefixed with K will be killed when the runlevel is entered, s or S Single user/Maintenance mode eg /etc/rc6.d/K20apache (Slackware) • Executed in numerical order M Multiuser mode (Slackware)

13 http://www.yolinux.com/TUTORIALS/LinuxTutorialInitProcess.html 14

Windows NT

• Power-on self test (POST) phase • Initial startup phase • Boot loader phase: NTLDR • Startup phase: create interrupt descriptor table, page tables and enable paging • Detect and configure hardware phase • osloader.exe: knows file systems, Multimediajärjestelmät • read boot.ini, check if hibernated, • Load Ntoskrnl.ece, hal.dll () • Load boot drivers (hard drive, , etc) • Kernel loading phase • Interrupt controller, memory manager, , process manager, • Load and initialize system drivers • Logon phase • Start Session manager subsystem

15 16

Objectives What is Multimedia?

• To identify the characteristics of • Multimedia data includes multimedia data - audio and video clips (i.e. MP3 and MPEG • To examine several algorithms used to files) compress multimedia data - live webcasts • To explore the operating system requirements of multimedia data, • Multimedia data may be delivered to including CPU and disk and - desktop PC’s network management - handheld devices (PDAs, smart phones)

17 18

Syksy 2006 / Luento 23 23-3 Käyttöjärjestelmät

Media Delivery Streaming

• Multimedia data is stored in the file system like • Streaming is delivering a multimedia file from a other ordinary data. to a client - typically the deliver occurs • However, multimedia data must be accessed over a network connection. with specific timing requirements. • There are two different types of streaming: • For example, video must be displayed at 24-30 1. Progressive download - the client begins playback frames per second. Multimedia video data must of the multimedia file as it is delivered. The file is be delivered at a rate which guarantees 24-30 ultimately stored on the client computer. frames/second. 2. Real-time streaming - the multimedia file is delivered to - but not stored on - the client’s computer. • Continuous-media data is data with specific rate requirements. 19 20

Multimedia Systems Real-time Streaming Characteristics • There are two types of real-time streaming: • Multimedia files can be quite large. (1) Live streaming - used to deliver a live event • Continuous media data may require very high while it is occurring. data rates. (2) On-demand streaming - used to deliver • Multimedia applications may be sensitive to media streams such as movies, archived timing delays during playback of the media. lectures, etc. The events are not delivered in real-time.

21 22

Compression Operating Systems Issues

• Because of the size and rate requirements of • The operating system must guarantee the specific data multimedia systems, multimedia files are often rate and timing requirements. compressed into a smaller form. • Such requirements are generally known as Quality-of- Service (QoS) guarantees. • MPEG Compression: • Guaranteeing QoS has effects in a computer system: (1) MPEG-1 - 352 X 240 @ 30 frames/second (1) CPU processing (2) MPEG-2 - Used for compressing DVD and high- (2) Scheduling definition television (HDTV) (3) File systems (3) MPEG-4 - Used to transmit audio, video, and (4) Network protocols graphics. Can be delivered over very slow connections (56 Kbps) 23 24

Syksy 2006 / Luento 23 23-4 Käyttöjärjestelmät

Requirement of Multimedia Parameters Defining QoS Operating Systems

• There are three levels of QoS • Throughput - the total amount of work completed during a specific time interval. (1) Best-effort service - the system makes a best effort with no QoS guarantees. • Delay - the elapsed time from when a request is first (2) Soft QoS - allows different traffic streams to be submitted to when the desired result is produced. prioritized, however no QoS guarantees are made. • Jitter - the delays that occur during playback of a (3) Hard QoS - the QoS rquirements are guaranteed. stream. • Reliability - how errors are handled during transmission and processing of continuous media.

25 26

Further QoS Issues Figure 20.1 Resources on a file server • QoS may be negotiated between the client and server. • Operating systems often use an admission control algorithm that admits a request for a service only if the server has sufficient resources to satisfy the request.

27 28

CPU Scheduling Disk Scheduling

• Multimedia systems require hard realtime • Disk scheduling algorithms must be optimized to meet scheduling to ensure critical tasks will be the timing deadlines and rate requirements of serviced within timing deadlines. continuous media. • Earliest-Deadline-First (EDF) Scheduling • The EDF scheduler uses a queue to order requests • Most hard realtime CPU scheduling algorithms according to the time it must be completed (its deadline.). assign realtime processes static priorities that do • SCAN-EDF Scheduling not change over time. • SCAN-EDF scheduling is similar to EDF except that requests with the same deadline are ordered according to a SCAN policy.

29 30

Syksy 2006 / Luento 23 23-5 Käyttöjärjestelmät

RealTime Streaming Protocol (RTSP) Network Management • Standard HTTP is stateless whereby the server • Three general methods for delivering content does not maintain the status of its connection from a server to a client across a network: with the client. [SGG07] Fig 20.3 (1) Unicasting - the server delivers the content to a single client. (2) Broadcasting - the server delivers the content to all clients, regardless whether they want the content or not. (3) Multicasting - the server delivers the content to a group of receivers who indicate they wish to receive the content.

31 32

RTSP States CineBlitz Multimedia Server

• SETUP - the server allocates resources for a client • CineBlitz supports both realtime and non- session. realtime clients. • PLAY - the server delivers a stream to a client session. • PAUSE - the server suspends delivery of a stream. • CineBlitz provides hard QoS guarantees to • TEARDOWN - the server breaks down the connection realtime clients using an admission control and releases the resources allocated for the session. algorithm. [SGG07] Fig 20.4 • The disk scheduler orders requests using C- SCAN order. 33 34

Figure 20.05 CineBlitz Admission Controller Double buffering in CineBlitz • Total buffer space required for N clients

where client has rate requirement of ri

35 36

Syksy 2006 / Luento 23 23-6 Käyttöjärjestelmät

LUENTO 24

2. Course exam Scheduling

Thursday 13.12 klo 16.00 A111 Exam covers: - Book chapters 9-16 (not 13) + Appendix B.4 - Lectures 11-24, exercises 7-12 - Team task 3

37 38

When? • Long-term • Accept new process? • Enough memory? swap? • Medium-term • Bring a suspended process back, when? • Enough free memory? • MPL, multiprocessing level? • Short-term Vuorotuksen jonomalli • Which process is switched to processor? • I/O • Service order of I/O requests (of processes) ? 39 40

Priorities high pri Algoritmit 0 1 missile avoidance process • First-Come-First-Served FCFS Real time 2 flight surface control processes fixed 3 navigation pri • Round Robin RR … 63 • Virtual Round Robin VRR 64 65 load control • Shortest Process Next SPN OS kernel 66 swapper fixed … cleaner pri • Shortest Remaining Time SRT 127 128 • Highest Response Ratio Next HRRN 129 Waiting for a long duration 130 calculator (128-140) variable • Multilevel Feedback feedback 131 pri … Used a lot of CPU 192 (f.ex. full time slice) low pri 41 • Fair Share Scheduling FSS 42

Syksy 2006 / Luento 23 23-7 Käyttöjärjestelmät

RT scheduling Schedulability test Tbl 10.4 [Stal05]

• Timely execution the most important issue • Obviously for any schedule, the schedulable utilisation: • No fairness, no minimising the response time Ui = C1/T1 + C2/T2 + .. + Cn/Tn <= 1 • Os can only guess the actual execution time • Sufficient condition for schedulability using RM • Programmer gives some estimates, period gives hints scheduling 1/n • Quite often: just schedule rt-tasks fast C1/T1 + C2/T2 + .. + Cn/Tn <= n(2 - 1) • High priority, predetermined schedule plan, others can suffer • N is number of tasks, the upper limit for the value is ln2 ~ 0.693 -> for any i U < 0.693 • Two main alternative for dynamic scheduling i • Earliest Deadline First (EDF) • For EDF the sufficient and effective condition is

• Rate Monotonic Scheduling (RM) Ui = C1/T1 + C2/T2 + .. + Cn/Tn <= 1 43 44

Logical I/O System calls

Device independent - file system. buffering I/O Device dependent - device drivers - queueing, ordering - interrupt handling

Physical I/O Devices - transfer, DMA - interrupts 45 46 Kuva 11.4

Disk access delays & scheduling Levyn vuorotusalgoritmeja

(Fig 11.6 [Stal05]) • Random? FIFO? PRI? LIFO? • Not good, no usage of current position of the arm • Considers the position • SSTF • SCAN • C-SCAN • N-step-SCAN ja FSCAN 47 48

Syksy 2006 / Luento 23 23-8 Käyttöjärjestelmät

RAID - Redundant Array of Independent Disks • RAID 0 (no redundancy, no replication) • RAID 1 (mirror, replicated disk) • RAID 2 (Hamming) block buffers • RAID 3 (parity bit) • RAID 4 (parity block) Ch 11.7 [Stal 05] • RAID 5 (distributed parity block) • RAID 6 (2 distributed parity blocks)

49 50

Most Recently Used – MRU Three block buffers, disk buffers Sections

• Buffer area in main memory to temporarily store disk • Most Recently Used – MRU Three Sections blocks • parannus: jaa jono kolmeen osaan • Principle of locality • poistot aina viimeisestä osasta • Prefetch / delayed write • etuosasta pudonneelle jää aikaa vanheta • tulos: parempi algoritmi kuin LRU tai LFU • Replacement algorithms • LRU: Least Recently Used • LFU: Least Frequently Used • Most Recently Used – MRU FIFO (Frequency Based Replacement)

51 Fig 11.9 (b) [Stal 05] 52

Directory = File, that contains information about other files • Only OS is allowed directly to access these files • All changes through system calls only File system • Root directory, home directories • Processes can create subdirectories • Fixed location for root directory on disk 53 54

Syksy 2006 / Luento 23 23-9 Käyttöjärjestelmät

Allocation: Three alternatives Free space Tan01 6-21

• Contiguous block • Chained • Indexed

55 Block list Bit map 56

UNIX Tan01 6-38 UNIX Tan01 6-39

57 58

Shared file • • Direct link from several directories • Multiple owners, all with same rights • removed from one directory ĺ available still from other locations • Soft link (symbolic link) LINUX • New file, which has file : symbolic link • Content of the new file: path to the actual file (string that file system : see slides of week 8 contains directories and file name) • Original file exists only in one directory ( -> just one owner) • owner removes ĺ the others cannot access, the link is invalid

59 60

Syksy 2006 / Luento 23 23 - 10 Käyttöjärjestelmät

LINUX Virtual Tiedostojen yhteiskäyttö File System

[Tane 01]

61 62

Linux file systems

• ext2fs (second extended file system)

• /proc

[Tane01] • ext3fs

63 64

ext2fs 0 1 2 3 4 5 6 7 0 Number of i-nodes Number of blocks Linux ext2fs superlohko 8 Number of reserved Number of free blocks • block groups (superblock) blocks • Each group is one continuous area of the disk 16 Number of free i-nodes First data block • 1 lohko • i-nodes and datablock of one group cose to each other 24 Block size Fragment size • Shorter track distances within one file • Kuvaa koko 32 Blocks per group Fragments per group • All block sizes are identical(1 KB) ext2fs-partition 40 i-nodes per group Time of mounting rakenteen • All i-node sizes are 128B (trad. UNIX 64B) 48 Time of last write Status Max. mnt • Kopio jokaisen cnt lohkoryhmän 56 Ext2signat Status Error Pad word alussa . behav. Fig. 10-35 • luotettavuus, 64 Time of last test Max test interval [Tane 01] virheestä 72 Operating system File system revision toipuminen 80 RESUID RESGID Pad word

Pad words 65 blocksize 66

Syksy 2006 / Luento 23 23 - 11 Käyttöjärjestelmät

ext2fs 0 1 2 3 4 5 6 7 Journaling (logging) file system i-node 0 Mode Uid File size 8 Access time Time of creation • Kirjataan kaikki muutokset (journalointi?) 16 Time of modification Time of deletion • Pitää tiedostojärjestelmän eheänä 24 Gid Link counter No. of blocks • Kirjataan • Vain metatieto muutoksista – journal 32 File attributes Reserved (OS-dependent) • Sekä metatieto että itse data – loki Access 40 12 direct blocks • Tarve: Control List 88 One-stage indirect block Two-stage indirect block • Tiedostojärjestelmän tarkistus (check) kestää liian kauan, jos epänormaali ’kaatuminen’ 96 Three-stage indirect block File version • Valtaosa levyoperaatioista on kirjoituksia, 104 File ACL Directory ACL lukuoperaatiot tehdään puskureista 112 Fragment address Reserved (OS-dependent) • Useimmat kirjoitukset pieniä päivityksiä 120 Reserved (OS-dependent) • levyn hakuvarsi liikkuu paljon, vähän dataa siirtyy

67 68

Perusidea NTFS: Piirteitä • Kaatumisista ja levyvirheistä toipuminen • Ongelma tavallisen tiedostojärjestelmän uuden tiedoston X • LFS lokitiedoston avulla luomisessa: • Käyttöoikeudet • kirjoita hakemiston i-node, hakemisto, tiedoston i-node • pääsylistat (security descriptor) ja lopulta tiedosto • Sallii suuret levyt ja tiedostot • virta poikki (tms vika) kesken kaiken? Oooops. • FAT32’ssa vain 232 lohkoa, suuri allokointitaulu • Ratkaisu: tapahtumaloki, joka takaa tiedostojärjestelmän • Tiedosto-oliot ovat (arvo, attribuutti) -pareja eheyden - vrt tietokantojen loki • Mahdollisuus indeksointiin tiedoston käsittelyn nopeuttamiseksi • Esim: NTFS, Red Hat Linux ext3fs • Lohko, cluster • yksi tai useampi peräkkäinen sektori (esim. 512 B - 4 KB) • 32 GB levyllä 128 sektoria/lohko (Ļ lohko 64-512 KB) • varauksen ja kirjanpidon perusyksikkö • Partitio, volume 69 70 • fyysinen levyn looginen osa, jolla oma tiedostojärjestelmä

(Fig. 12.17 [Stal 05]) NTFS-partitio NTFS – MFT (Master File Table)

• 1 KB:n kokoisia MFT-tietueita • jokainen kuvaa yhden taltiolla olevan tiedoston • myös hakemisto on tiedosto • vaihtelevanmittainen osa käytössä • Boottilohko • (attribuutti, arvo) pareja (ei paikkasidonnainen!) • partition ja tiedostojärj. rakenne, boottitietue ja -koodi • data attribuutti, 'arvo' = lohkojen sijainti • MFT:n sijainti • 16 ensimmäistä tietuetta varattu ns. metadatalle • MFT • tietoa tiedostoista, hakemistoista (folders) ja vapaasta tilasta • 16 $-alkuista tiedostoa • System Files (~ 1MB) • Jos pieni tiedosto, tietue sisältää myös datan • kopio MFT:n alkuosasta • Jos iso tiedosto, data erillisellä tallealueella • virheistätoipumisloki, bittikartta vapaat/varatut lohkot, attribuuttien kuvaustaulu • MFT-tietuessa lohkonumeroita • File Area - tiedostojen lohkoille 71 • kuvaus voi jatkua useampaan MFT-tietueseen 72

Syksy 2006 / Luento 23 23 - 12 Käyttöjärjestelmät

Hakemiston MFT-tietue (Fig. 11-38 [Tane01])

pieni hakemisto Hajautettu prosessi

• Pienissä hakemistoissa MFT-tietueet peräkkäisjärjestyksessä • Isoissa hakemistoissa MFT-tietueessa B-puun (B-) indeksirakenne • nimen etsintä ei ole peräkkäishakua 73 74

Asiakas-palvelija luokitteluja Process communication : Ch 14 message passing • Reliable / unreliable • Binding • Synchronous / • Based on network asynchronous topology or message content • Persistent / transient • Static / Dynamic

1-29 Alternative client-server organizations.

75 76

Process communication: RPC RPC

• Remote procedure call looks to the calling client as any other local procedure (or function or method) call.

77 78

Syksy 2006 / Luento 23 23 - 13 Käyttöjärjestelmät

Steps in DCE model Hajautettuja olioita implementation (Distributed objects) of the client and

the server client machine server machine object server client state client method invokes same a method interface interface proxy skeleton same client OS method server OS

network marshalled invocation 79 80

Middleware services CORBA – Common Object Request Broker Architecture

Interface IDL Implementation Repository Compiler Repository

operation() in args Object Client OBJ (Servant) REF out args + return

IDL DSI SKEL IDL ORB DII STUBS INTERFACE Object Adapter

ORB CORE GIOP/IIOP/ESIOPS

•CORBA shields applications from heterogeneous platform dependencies CACM 45, 6 pp 45 81 •e.g., languages, operating systems, networking protocols, hardware 82

Cluster: classification Cluster Computer Architecture Clustering Method Description Benefits Limitations Passive Standby A secondary server Easy to implement. High cost because the secondary server takes over in case of is unavailable for other processing primary server failure. tasks. Active Secondary: The secondary server is Reduced cost because Increased complexity. also used for processing secondary servers can tasks. be used for processing. - Separate Servers Separate servers have High availability. High network and server overhead due their own disks. Data is to copying operations. continuously copied from primary to secondary server. - Servers Connected to Servers are cabled to Reduced network and Usually requires disk mirroring or Disks the same disks, but each server overhead due to RAID technology to compensate for server owns its disks. If elimination of copying risk of disk failure. one server fails, its operations. disks are taken over by the other server. - Servers Share Disks Multiple servers Low network and Requires lock manager software. simultaneously share server overhead. Usually used with disk mirroring or access to disks. Reduced risk of RAID technology. downtime caused by disk failure. 83 84

Syksy 2006 / Luento 23 23 - 14 Käyttöjärjestelmät

Process Siirtopolitiikkoja Migration • Eager (all): Prosessi 3 (solmu S) • Siirrä kaikki (muistialueet+muut) Prosessiksi 4 (solmu D) • Precopy: • Prosessia vielä suoritetaan, kun muistialueita jo kopioidaan • Load balancing, • Eager (dirty): • Siirrä vain se osa muistiavaruutta, joka on keskusmuistissa ja availability, jota on muutettu closer to a • Copy-on-reference: resource • Siirrä sivua vain viittattaessa • Flushing: • Kopioi prosessin muuttuneet sivut levylle

85 86

Questions from previous exams

• Scheduling • One processor: Multilevel feedback, Fair share, … • Multiprocessor: kimppavuorotus Security (see last week) • Real time.: Rate monotonic, EDF • I/O and file systems • Buffering, buffer allocation • Disk access: SCAN, FIFO, SSTF • Ext2fs, NTFS, file protection • Security • Distribution: • Clusters • RPC, CORBA 87 88

-- END --

Käyttöjärjestelmät

89

Syksy 2006 / Luento 23 23 - 15