COS 318: Operating Systems I/O Device Interactions and Drivers

Home , Data buffer, Device driver, Streams

Topics

COS 318: Operating Systems u So far: l Management of CPU and concurrency l Management of main memory and virtual memory I/O Device Interactions and u Next: Management of the I/O system Drivers l Interacting with I/O devices l Device drivers Jaswinder Pal Singh l Storage Devices Computer Science Department Princeton University u Then, File Systems l File System Structure (http://www.cs.princeton.edu/courses/cos318/) l Naming and Directories l Efficiency/Performance

l Reliability and Protection 2

Input and Output Revisit Hardware

u A computer u Compute hardware CPU l CPU cores and caches CPU l Computation (CPU, memory hierarchy) CPUCPU CPU l Memory $ l Move data into and out of a system (locketween I/O devices Chip l I/O and memory hierarchy) l Controllers and logic Memory I/O bridge u Challenges with I/O devices I/O bus l Different categories with different characteristics: storage, u I/O Hardware networking, displays, keyboard, mouse ... l I/O bus or interconnect l Large number of device drivers to support l I/O device l I/O controller or adapter l Device drivers run in kernel mode and can crash systems • Often on parent board Network u Goals of the OS • Cable connects it to device • Often using standard interfaces: IDE, l Provide a generic, consistent, convenient and reliable way to SATA, SCSI, USB, FireWire… access I/O devices • Has registers for control, data signals • Processor gives commands and/or l Achieve potential I/O performance in a system data to controller to do I/O • Special I/O instructions (w. port addr.) 3 or memory mapped I/O 4

1 I/O Hierarchy A typical PC bus structure

u As with memory, fast I/O with less “capacity” near CPU, slower I/O with greater “capacity” further away

Performance Characteristics Interacting with Devices

u Overhead u A device has an interface, and an implementation l CPU time to initiate an operation l Interface exposed to external software, typically by device u Latency controller Initiate Data transfer l Time to transfer one bit l Implementation may be hardware, firmware, software l Overhead + time for 1 bit to reach destination Time u Mechanisms u Bandwidth l Programmed I/O (PIO) l Rate at which subsequent bits are Device Transfer rate l Interrupts transferred or reach destination Keyboard 10Bytes/sec l Bits/sec or Bytes/sec Mouse 100Bytes/sec l Direct Memory Access (DMA) u In general … … 10GE NIC 1.2GBytes/sec l Different transfer rates l Abstraction of byte transfers l Amortize overhead over block of bytes as transfer unit 7 8

2 Programmed I/O Polling in Programmed I/O u Example u Wait until device is not “busy” l RS-232 serial port CPU l A polling loop u Simple serial controller l May also poll to wait for device to complete its work l Status registers (ready, busy, … ) Memory u Advantages l Data register I/O Bus u Output l Simple CPU: Serial u Disadvantage l Wait until device is not “busy” Busy Ready … controller l Slow l Write data to “data” register Data l Waste CPU cycles l Tell device “ready” Device u Example l Wait until “ready” l If a device runs 100 operations / second, CPU may need to l Clear “ready” and set “busy” wait for 10 msec or 10,000,000 CPU cycles (1Ghz CPU) l Take data from “data” register l Clear “busy”

9 10

Interrupt-Driven Device Interrupt Handling Revisited/Refined u Allows CPU to avoid polling u Save more context u Example: Mouse CPU u Mask interrupts if needed u Simple mouse controller l Status registers (done, int, …) u Set up a context for interrupt service l Data registers (ΔX, ΔY, button) Memory I/O Bus u Set up a stack for interrupt service u Input Mouse: u Acknowledge the interrupt controller, enable it if needed Mouse l Wait until “done” u Save context to PCB l Store ΔX, ΔY, and button into Done Int … controller data registers ΔX u Run the interrupt service l Raise interrupt ΔY u Unmask interrupts if needed CPU (interrupt handler) Button l Clear “done” u Possibly change the priority of the process l Move ΔX, ΔY, and button into kernel buffer u Run the scheduler l Set “done” l Call scheduler

3 Another Problem Direct Memory Access (DMA) u CPU has to copy data from memory to device u Takes many CPU cycles, esp for larger I/Os u Can we get the CPU out of the copying loop, so it can do other things in parallel while data are being copied?

Direct Memory Access (DMA) Where Are these I/O “Registers?” u Example of disk u Explicit I/O “ports” for devices I/O device u A simple disk adaptor l Status register (ready, …) l Accessed by privileged I/O device l DMA command instructions (in, out) … l DMA memory address and size CPU l DMA data buffer u Memory mapped I/O Kernel ALU/FPU u DMA Write Kernel memory Memory l A portion of physical memory CPU: registers l Wait until DMA device is “ready” Data I/O Bus for each device l Clear “ready” User Caches l Set DMAWrite, address, size l Advantages Disk memory l Set “start” • Simple and uniform Memory Ready Start Int … adaptor l Block current thread/process • CPU instructions can access Mapped Disk adaptor: DMA Command these “registers” as memory I/O l DMA data to device address size (size--; address++) l Issues Memory l Interrupt when “size == 0” Data CPU (interrupt handler): Data • These memory locations should l Put the blocked thread/process into DMA buffer not be cached. Why? ready queue • Mark them not cacheable Disk: Move data to disk u Both approaches are used 15 16

4 Device I/O port locations on PCs (partial) I/O Software Stack

User-Level I/O Software Device-Independent OS software Device Drivers

Interrupt handlers

Hardware

I/O Interface and Device Drivers I/O Interface and Device Drivers

u I/O system calls encapsulate device behaviors in

Device Device generic classes Device driver controller u Device-driver layer hides differences among I/O

controllers from kernel

Device Device Device Rest of the driver controller u Devices vary in many dimensions operating l Character-stream or block . . system . . l Sequential or random-access Device InterruptHandling l Sharable or dedicated Device Device l Speed of operation driver controller Device l Read-write, read only, or write only Drivers

Operating System Hardware 19

5 Example Kernel I/O Structure Characteristics of I/O Devices

What Does A Device Driver Do? Device Driver Operations

u Provide “the rest of the OS” with APIs u Init ( deviceNumber ) l Init, Open, Close, Read, Write, … l Initialize hardware u Interface with controllers u Open( deviceNumber ) l Commands and data transfers with hardware controllers l Initialize driver and allocate resources u Driver operations u Close( deviceNumber ) l Initialize devices l Cleanup, deallocate, and possibly turnoff l Interpret outstanding requests u Device driver types l Manage data transfers l Character: variable sized data transfer l Accept and process interrupts l Block: fixed sized block data transfer l Maintain the integrity of driver and kernel data structures l Terminal: character driver with terminal control l Network: streams for networking

23 24

6 Character and Block Interfaces Network Devices

u Character device interface (keyboard, mouse, ports) u Different enough from the block & character devices to l read( deviceNumber, bufferAddr, size ) have own interface • Reads “size” bytes from a byte stream device to “bufferAddr” l write( deviceNumber, bufferAddr, size ) u Unix and Windows/NT include socket interface • Write “size” bytes from “bufferAddr” to a byte stream device l Separates network protocol from network operation

u Block device interface (disk drives) u Approaches vary widely (pipes, FIFOs, streams, l read( deviceNumber, deviceAddr, bufferAddr ) queues, mailboxes) • Transfer a block of data from “deviceAddr” to “bufferAddr” l write( deviceNumber, deviceAddr, bufferAddr ) • Transfer a block of data from “bufferAddr” to “deviceAddr” l seek( deviceNumber, deviceAddress ) • Move the head to the correct position • Usually not necessary

Clocks and Timers Unix Device Driver Entry Points

u Provide current time, elapsed time, timer u init() l Initialize hardware u start() l Boot time initialization (require system services) u if programmable interval time used for timings, periodic u open(dev, flag, id)and close(dev, flag, id) interrupts l Initialization resources for read or write and release resources u halt() l Call before the system is shutdown u ioctl (on UNIX) covers odd aspects of I/O such as u intr(vector) clocks and timers l Called by the kernel on a hardware interrupt u read(…) and write() calls l Data transfer u poll(pri) l Called by the kernel 25 to 100 times a second u ioctl(dev, cmd, arg, mode) l special request processing

7 Synchronous and Asynchronous I/O Synchronous Blocking Read

u Synchronous I/O Application Kernel HW Device l Calling process waits for I/O call to return before doing anything l Blocking I/O Switch to syscall Kernel context • Read() or write() will block a user process until its completion • Easy to use and understand Driver Initiates • OS overlaps synchronous I/O with another process DMA read block l Nonblocking I/O DMA • Return as much data (and count of it) as avaialble right away read Interrupt u Asynchronous I/O Copy to l Process runs while I/O executes User buf l Let user process do other things before I/O completion Switch to Unblock l I/O completion will notify the user process user context returnl

29 30

Synchronous Blocking Read Asynchronous Read

u A process issues a read call which executes a system call Application Kernel HW Device u System call code checks for correctness and buffer cache Switch to u If it needs to perform I/O, it will issue a device driver call aio_read Kernel context u Device driver allocates a buffer for read and schedules I/O Driver initiates u Initiate DMA read transfer DMA read u Block the current process and schedule a ready process Do u Device controller performs DMA read transfer other DMA u Device sends an interrupt on completion work read Interrupt u Interrupt handler wakes up blocked process (make it ready) aio_return Copy to u Move data from kernel buffer to user buffer User buf incomplete u System call returns to user code Complete u User process continues aio_return

Complete

31 32

8 Asynchronous I/O Why Buffering in Kernel?

POSIX P1003.4 Asynchronous I/O interface functions: u Speed mismatch between the producer and consumer (available in Solaris, AIX, Tru64 Unix, Linux 2.6,…) l Character device and block device, for example u aio_read: begin asynchronous read l Adapt different data transfer sizes (packets vs. streams) u aio_write: begin asynchronous write u DMA requires contiguous physical memory u aio_cancel: cancel asynchronous read/write requests l I/O devices see physical memory l User programs use virtual memory u aio_error: retrieve Asynchronous I/O error status u Spooling u aio_fsync: asynchronously force I/O completion, and sets errno to ENOSYS l Avoid deadlock problems u Caching u aio_return: retrieve status of Asynchronous I/O operation l Reduce I/O operations u aio_suspend: suspend until Asynchronous I/O completes u lio_listio: issue list of I/O requests

33 34

Other Device Driver Design Issues Dynamic Binding of Device Drivers

u Statically install device drivers Open(1,…) l Reboot OS to install a new device driver u Indirection Driver (dev 0) l Indirect table for all Open: u Dynamically download device drivers device driver entry points open read Read: l No reboot, but use an indirection Write: u Download a driver write

l Load drivers into kernel memory l Allocate kernel memory open Driver (dev 1) Open: l Install entry points and maintain related data structures read l Store driver code Read: l Initialize the device drivers write l Link up all entry points Write:

u Delete a driver Driver (dev 1) Open: l Unlink entry points Read: Write: l Deallocate kernel memory

35 36

9 Issues with Device Drivers I/O Software Stack

u Flexible for users, ISVs and IHVs l Users can download and install device drivers l Vendors can work with open hardware platforms User-Level I/O Software u Dangerous l Device drivers run in kernel mode Device-Independent l Bad device drivers can cause kernel crashes and introduce OS software security holes Device Drivers u Progress on making device driver more secure Interrupt handlers

u How much of OS code is device drivers? Hardware

Next: Kernel I/O Subsystem Kernel I/O subsystem: “Scheduling”

u Some I/O request ordering via per-device queue u Some OSes try fairness

Device status table

10 Kernel I/O subsystem (contd.) Error handling u Buffering - store data in memory while transferring between devices u OS can recover from disk read, device unavailable, l To cope with device speed mismatch transient write failures l To cope with device transfer size mismatch (e.g., packets in networking) l To maintain “copy semantics” • Copy data from user buffer to kernel buffer u Most return an error no. or code when I/O request fails u How to deal with address translation? u System error logs hold problem reports l I/O devices see physical memory, but programs use virtual memory l E.g. DMA may require contiguous physical addresses u Caching - fast memory holding copy of data l Reduce need to go to devices, key to performance u Spooling - hold output for a device l If a device can serve only one request at a time, i.e., printing l Used to avoid deadlock problems

I/O protection Life cycle of an I/O request u User process may accidentally or purposefully attempt to disrupt normal operation via illegal I/O instructions

l All I/O instructions defined to be privileged

l I/O must be performed via system calls • Memory-mapped and I/O port memory locations must be protected too

11 Kernel data structures From User Request to Hardware Operations

u State info for I/O components, including open file tables, u Consider reading a file from disk for a process: network connections, character device state l Determine device holding file l Translate name to device representation u Many complex data structures to track buffers, memory l Physically read data from disk into buffer allocation, “dirty” blocks l Make data available to requesting process l Return control to process u Some use object-oriented methods and message passing to implement I/O

Another example: blocked read w. DMA Summary

u A process issues a read call which executes a system call u IO Devices u System call code checks for correctness and cache l Programmed I/O is simple but inefficient u If it needs to perform I/O, it will issues a device driver call l Interrupt mechanism supports overlap of CPU with I/O u Device driver allocates a buffer for read and schedules I/O l DMA is efficient, but requires sophisticated software u Controller performs DMA data transfer, blocks the process u u Device generates an interrupt on completion Synchronous and Asynchronous I/O u Interrupt handler stores any data and notifies completion l Asynchronous I/O allows user code to perform overlapping u Move data from kernel buffer to user buffer and wakeup blocked u Device drivers process l Dominate the code size of OS u User process continues l Dynamic binding is desirable for many devices l Device drivers can introduce security holes l Progress on secure code for device drivers but completely removing device driver security is still an open problem u Role of device-independent kernel software