Interrupt Handling

CCChahahaptptpteeerrr 555 KKKeeerrrnnneeelll SSSyyynnnccchhhrrrooonnniiizzzaaatttionionion Hsung-Pin Chang Department of Computer Science National Chung Hsing University Outline • Kernel Control Paths • When Synchronization Is Not Necessary • Synchronization Primitives • Synchronizing Accesses to Kernel Data Structure • Examples of Race Condition Prevention Kernel Control Paths • Kernel control path – A sequence of instructions executed by the kernel to handle interrupts of different kinds – Each kernel request is handled by a different kernel control path Kernel Control Paths (Cont.) • Kernel requests may be issued in several possible ways – A process executing in User Mode causes an exception-for instance, by executing at int0x80 instruction – An external devices sends a signal to a Programmable Interrupt Controller Kernel Control Paths (Cont.) – A process executing in Kernel Mode causes a Page Fault exception – A process running in a MP system and executing in Kernel Mode raises an interprocessorinterrupt Kernel Control Paths (Cont.) • Kernel control path is quite similar to the process, except – Does not have a descriptor – Not scheduled through scheduler, but rather by inserting sequence of instructions into the kernel code Kernel Control Paths (Cont.) • In some cases, the CPU interleaves kernel control paths when one of the following event occurs – A process switch occurs, i.e., when the schedule() function is invoked – An interrupt occurs while the CPU is running a kernel control path with interrupt enabled – A deferrable function is executed Kernel Control Paths (Cont.) • Thus, some kernel data structures must be protected to prevent race condition – The code to modify these data structures must be in a critical section When Synchronization Is Not Necessary • Linux kernel is not preemptive – A running process cannot be preempted while it remains in Kernel Mode • As a result, in Linux – No process running in Kernel Mode may be replaced by another process, except when the former voluntarily relinquishes control of CPU When Synchronization Is Not Necessary (Cont.) – Interrupt, exception or softirqhandling can interrupt a process running in Kernel Mode • However, when the handler terminates, kernel control path of the process is resumed – A process control path performing interrupt handling cannot be interrupted by a kernel control path executing a deferrable function or a system call service routine When Synchronization Is Not Necessary (Cont.) • Thus, on uniprocessor – Kernel data structures that are not updated by interrupt, exception, or softirqhandlers can be safely accessed • However, on MP, things are much more complicated • The rest describes what to do when synchronization is necessary Synchronization Primitives • Atomic operations • Memory Barriers • Spin Locks • Read/Write Spin Locks • The Big Reader Lock • Semaphore • Read/Write Semaphores • Completions • Local Interrupt Disabling • Global Interrupt Disabling • Disabling Deferrable Functions Synchronization Primitives (Cont.) Atomic operation Atomic read-modify-write ALL CPUs instruction to a counter Memory barrier Avoid instruction re-ordering Local CPU Spin lock Lock with busy wait ALL CPUs Semaphore Lock with blocking wait (sleep) ALL CPUs Local interrupt Forbid interrupt handling on a Local CPU disabling single CPU Local softirq Forbid deferrable function Local CPU disabling handling on a single CPU Global interrupt Forbid interrupt and softirq ALL CPUs disabling handling on all CPUs Atomic Operations • Some instructions are of type “read- modify-write” • If two such instructions are nonatomicthat issued by two CPUs to access the same location – Memory arbiter may assign memory to the second one while the first one has not yet been completed – Race condition Atomic Operations (Cont.) • To prevent race conditions – Provide operations that are atomic at chip level – Thus, cannot be interrupted in the middle and avoid access to the same memory location by other CPUs • Atomic operations acts as base of other, more flexible mechanisms to create critical sections Atomic Operations (Cont.) • 80x86 instructions that are atomic – Instructions that make zero or one aligned memory access – Read-modify-write, e.g., inc or dec, are atomic if no other processor has taken the memory bus in the middle • In a uniprocessor, no memory bus stealing Atomic Operations (Cont.) – Read-modify-write instruction whose opcodeis prefixed by the lock byte (0xf0) are atomic even on MP • Control unit (CU) lock the memory bus until the instruction is completed – Instructions whose opcodeis prefixed by a rep (0xf2) byte is not atomic • Rep: CU repeat the same instructions several times • CU check pending interrupts before a new iteration Atomic Operations (Cont.) • We don’t know the compiler will use a single, atomic instruction for an operation – Linux thus provides • Atomic_t data type that are 24-bit atomically accessible counter • atmoicoperations – Table 5-2 and 5-3 Memory Barriers • Compiler may optimizing the code – Reorder the execution of instructions • However, for synchronization – Instructions reordering must be avoided – Thus, all synchronization primitives act as memory barriers Memory Barriers (Cont.) • A memory barriers primitives ensures that – the operations placed before the primitives are finished before starting the operations placed after the primitives – Like a firewall that cannot be passed by any outside instructions Memory Barriers (Cont.) • 80x86’s instructions that are “serializing” because they act as memory barriers – Instructions operate on I/O ports – Instructions perfixedby the lock type – Instructions that writes to control registers, system registers, or debug registers – A few special instructions, e.g., iret Memory Barriers in Linux Macro Description mb() Memory barrier for MP and UP rmb() Read memory barrier for MP or UP wmb() Write memory barrier for MP and UP smp_mb() Memory barrier for MP only, do nothing for UP smp_rmb() Read memory barrier for MP only, do nothing for UP smp_wmb() Write memory barrier for MP only, do nothing for UP Linux Implementation of Memory Barrier • Depends on system architecture • On the Intel platform – rmb() expands to • asmvolatile(“lock; addl$0,0(%%esp)”:::”memory”) – Volatile: Spin Locks • Spin locks are a special kind of lock designed to work in a MP system – If the lock is closed, spin around, i.e., repeatedly executing a tight loop, until the lock is released – Useless in a UP system • The waiting kernel control path would keep running, and the holding kernel control path have no chance to release the lock Spin Locks (Cont.) • Spin locks are useful since many kernel resources are locked for a fraction of milliseconds only – Thus, it would be far more time- consuming to release the CPU and reacquire it later Spin Locks (Cont.) • Five functions are used to initialize, test, and set spin locks – All these functions are based on atomic operations Spin Locks (Cont.) spin_lock_init() Set the spin lock to 1 (unlocked) spin_lock() Cycle until spin lock becomes 1 (unlocked), then set to 0 (locked) spin_unlock() Set the spin lock to 1 (unlocked) spin_unlock_wait() Wait until the spin lock becomes 1 (unlocked) spin_is_locked() Return 0 if the spin lock is 1 (unlocked); 0 otherwise spin_trylock() Set the spin lock to 0 (locked), and return 1 if the lock is obtained; 0 otherwise Spin Locks (Cont.) • spin_lock(): acquire a spin lock – 1: lock; decbslp ; (1)slp: spin lock’s address; (2) atomic jns3f ; jump if not signed (Positive=>JUMP) 2: cmpb$0, slp ; compare 0 with slp? pause ; see the following jle2b ; jump if less than or equal jmp1b ; check whether other processor ; has grabbed the lock 3: ; acquire the lock • Pause: P4 instruction that optimizing the execution of spin lock – Backward compatible to rep;nop, equal to do nothing Spin Locks (Cont.) • spin_unlock(): release a spin lock – lock; movb$1, slp Read/Write Spin Locks • Introduced to increase the amount of concurrency inside the kernel – Allow several kernel control path to simultaneously read the same DS • As long as no one modifies it – However, once to write, must acquire the write lock The Big Reader Lock • For MP systems – Skip!!! Semaphores • A lock primitive that allows waiters to sleep until the desired resource become free • Linux provides two kinds of semaphores – Kernel semaphores used by kernel control path – System V IPC semaphores used by User Mode process Semaphores (Cont.) • Kernel semaphores is similar to a spin lock – Does not allow a kernel control path to proceed until the lock is open – However, if resource is protected • Process is suspended (blocked) • Thus, kernel semaphores can be acquired only by functions that are allowed to sleep – Interrupt handlers and deferrable functions cannot use them Semaphores (Cont.) • Kernel semaphore: an object of type struct_semaphore – Count: store an atomic_t value • >0: the resource is free • =0: the resource is busy but no one is waiting • <0: the resource is unavailable – Wait • Store the address of a wait queue list that includes all sleeping processes waiting for this semaphore positive – Sleepers • Store a flag that indicates whether some processes are sleeping on the semaphore Semaphores (Cont.) • UP() – Increment count value – If (count > 0) • No process waiting, do nothing – Else • Wake up one sleeping process Semaphores (Cont.) • Down – Decrement the count value – If (count >= 0) • Acquire the resource – Else • Change the state from TASK_RUNNING to TASK_UNITERRUPTIBLE • Put the process in the semaphore

Load more