Real-Time (RT-Preempt)

+

SCHED DEADLINE ( 3.14)

Josef Widder

BRDS 2015 I Real-time computing is equivalent to fast computing

I Real-time programming is assembly coding

I Real-time is meaningless if hardware will fail or software contains bugs

I Real-time systems function in static environments

Misconceptions about Real-Time Computing [Sta88]

I Advances in hardware will take care of real-time requiremens

2 I Real-time programming is assembly coding

I Real-time is meaningless if hardware will fail or software contains bugs

I Real-time systems function in static environments

Misconceptions about Real-Time Computing [Sta88]

I Advances in supercomputer hardware will take care of real-time requiremens

I Real-time computing is equivalent to fast computing

2 I Real-time is meaningless if hardware will fail or software contains bugs

I Real-time systems function in static environments

Misconceptions about Real-Time Computing [Sta88]

I Advances in supercomputer hardware will take care of real-time requiremens

I Real-time computing is equivalent to fast computing

I Real-time programming is assembly coding

2 I Real-time systems function in static environments

Misconceptions about Real-Time Computing [Sta88]

I Advances in supercomputer hardware will take care of real-time requiremens

I Real-time computing is equivalent to fast computing

I Real-time programming is assembly coding

I Real-time is meaningless if hardware will fail or software contains bugs

2 Misconceptions about Real-Time Computing [Sta88]

I Advances in supercomputer hardware will take care of real-time requiremens

I Real-time computing is equivalent to fast computing

I Real-time programming is assembly coding

I Real-time is meaningless if hardware will fail or software contains bugs

I Real-time systems function in static environments

2 general-purpose most of the time good behavior

I high throughput

I under normal (avg. case) behavior

Real-Time vs. general-purpose OS

real-time always predictable behavior

I timing guarantees

I under worst case behavior

3 Real-Time vs. general-purpose OS

real-time always predictable behavior

I timing guarantees

I under worst case behavior

general-purpose most of the time good behavior

I high throughput

I under normal (avg. case) behavior

3 (even with patch, convenient Linux features should not be touched)

Why (Vanilla) Linux is not Real-time (bird’s eye view)

I general-purpose

I high functionality

I optimized for average case

I latency my vary and may not be bounded

4 Why (Vanilla) Linux is not Real-time (bird’s eye view)

I general-purpose

I high functionality

I optimized for average case

I latency my vary and may not be bounded

(even with patch, convenient Linux features should not be touched)

4 Why Linux is not Real-time (a closer look)

I Applications run in

I Hardware interaction is in kernel

I kernel not preempted by user threads ⇒

I event triggers high-priority thread

I thread cannot preempt kernel ⇒ large latency possible > 100ms

I “classic” time sharing

5 CONFIG PREEMPT RT patch

to improve RT behavior, “reduce special status” of kernel

I kernel code outside of

I spinlocks protected areas

I interrupt handlers

may be preempted

I WC latency drops to (single digit) ms

Unlike semaphores, spinlocks may be used in code that cannot sleep, such as interrupt handlers. When properly used, spinlocks offer higher performance than semaphores in general.

6 I citical sections protected by spinlock t and rwlock t are now preemptible. rwlock t allows multiple readers in critical section

I priority inheritance for in-kernel spinlocks and semaphores.

I converting interrupt handlers into preemptible kernel threads

I user space POSIX timers with high resolution.

How does it work? [faq07]

The patch converts Linux into a fully preemptible kernel:

I in-kernel locking-primitives (using spinlocks) preemptible through reimplementation with rtmutexes. RT-mutexes extend the semantics of simple mutexes by the priority inheritance protocol

7 I converting interrupt handlers into preemptible kernel threads

I user space POSIX timers with high resolution.

How does it work? [faq07]

The patch converts Linux into a fully preemptible kernel:

I in-kernel locking-primitives (using spinlocks) preemptible through reimplementation with rtmutexes. RT-mutexes extend the semantics of simple mutexes by the priority inheritance protocol

I citical sections protected by spinlock t and rwlock t are now preemptible. rwlock t allows multiple readers in critical section

I priority inheritance for in-kernel spinlocks and semaphores.

7 I user space POSIX timers with high resolution.

How does it work? [faq07]

The patch converts Linux into a fully preemptible kernel:

I in-kernel locking-primitives (using spinlocks) preemptible through reimplementation with rtmutexes. RT-mutexes extend the semantics of simple mutexes by the priority inheritance protocol

I citical sections protected by spinlock t and rwlock t are now preemptible. rwlock t allows multiple readers in critical section

I priority inheritance for in-kernel spinlocks and semaphores.

I converting interrupt handlers into preemptible kernel threads

7 How does it work? [faq07]

The patch converts Linux into a fully preemptible kernel:

I in-kernel locking-primitives (using spinlocks) preemptible through reimplementation with rtmutexes. RT-mutexes extend the semantics of simple mutexes by the priority inheritance protocol

I citical sections protected by spinlock t and rwlock t are now preemptible. rwlock t allows multiple readers in critical section

I priority inheritance for in-kernel spinlocks and semaphores.

I converting interrupt handlers into preemptible kernel threads

I user space POSIX timers with high resolution.

7 . . . and it is still Linux ⇒ certain features must not be used. . .

What sort of real-time? [faq07]

it depends. . .

I speed of the CPU and the architecture

I device drivers

I hardware

e.g., DMA activity can introduce significant latencies

e.g., some firmwares can stop the system for housekeeping (SMI, Service Management Interrupts) SMIs can not be trapped by the OS

8 What sort of real-time? [faq07]

it depends. . .

I speed of the CPU and the architecture

I device drivers

I hardware

e.g., DMA activity can introduce significant latencies

e.g., some firmwares can stop the system for housekeeping (SMI, Service Management Interrupts) SMIs can not be trapped by the OS

. . . and it is still Linux ⇒ certain features must not be used. . .

8 I Create all threads at startup time of the application i.e., never start threads dynamically during mission time

I Touch each page of the entire stack of each thread

I Never use system calls that are known to generate page faults, such as fopen(). opening files does mmap() system call ⇒ page fault

Guidelines for writing RT applications [faq07]

I Call mlockall() as soon as possible from main(). causes all of the pages mapped by the address space of a process to be memory resident i.e., preventing that memory is paged to the swap area

9 I Never use system calls that are known to generate page faults, such as fopen(). opening files does mmap() system call ⇒ page fault

Guidelines for writing RT applications [faq07]

I Call mlockall() as soon as possible from main(). causes all of the pages mapped by the address space of a process to be memory resident i.e., preventing that memory is paged to the swap area

I Create all threads at startup time of the application i.e., never start threads dynamically during mission time

I Touch each page of the entire stack of each thread

9 Guidelines for writing RT applications [faq07]

I Call mlockall() as soon as possible from main(). causes all of the pages mapped by the address space of a process to be memory resident i.e., preventing that memory is paged to the swap area

I Create all threads at startup time of the application i.e., never start threads dynamically during mission time

I Touch each page of the entire stack of each thread

I Never use system calls that are known to generate page faults, such as fopen(). opening files does mmap() system call ⇒ page fault

9 Power management Watts vs. latency Hyperthreading and out-of-order execution performance vs. latency

How to not build RT applications [how07]

Don’t use fancy stuff! Direct memory access

I latency in microseconds by

I SATA/PATA/SCSI devices

I network adapters

I video cards

I can sometimes be controlled by device drivers throughput vs. latency

I this problem independent of OS

10 Hyperthreading and out-of-order execution performance vs. latency

How to not build RT applications [how07]

Don’t use fancy stuff! Direct memory access

I latency in microseconds by

I SATA/PATA/SCSI devices

I network adapters

I video cards

I can sometimes be controlled by device drivers throughput vs. latency

I this problem independent of OS

Power management Watts vs. latency

10 How to not build RT applications [how07]

Don’t use fancy stuff! Direct memory access

I latency in microseconds by

I SATA/PATA/SCSI devices

I network adapters

I video cards

I can sometimes be controlled by device drivers throughput vs. latency

I this problem independent of OS

Power management Watts vs. latency Hyperthreading and out-of-order execution performance vs. latency

10 I take care of page faults (swapping) during start-up

I Call directly from the main() entry the mlockall() call

I Create all threads at startup time of the application.

I Reserve a pool of memory to do new/delete or malloc/free

I Never use system calls that are known to generate pagefaults.

I do not configure application with priority 99 some watchdog threads need higher priority than appl.

I use mmap to pass data around

More Guidelines. . . [how07]

I do not use VGA text console very large latency

I use serial console, ssh, telnet

11 I do not configure application with priority 99 some watchdog threads need higher priority than appl.

I use mmap to pass data around

More Guidelines. . . [how07]

I do not use VGA text console very large latency

I use serial console, ssh, telnet

I take care of page faults (swapping) during start-up

I Call directly from the main() entry the mlockall() call

I Create all threads at startup time of the application.

I Reserve a pool of memory to do new/delete or malloc/free

I Never use system calls that are known to generate pagefaults.

11 More Guidelines. . . [how07]

I do not use VGA text console very large latency

I use serial console, ssh, telnet

I take care of page faults (swapping) during start-up

I Call directly from the main() entry the mlockall() call

I Create all threads at startup time of the application.

I Reserve a pool of memory to do new/delete or malloc/free

I Never use system calls that are known to generate pagefaults.

I do not configure application with priority 99 some watchdog threads need higher priority than appl.

I use mmap to pass data around

11 let’s see “hello, world.” and kernel detection [rth06]

Install [rth06]

it is a kernel patch. . . thus something like this:

getting the sources:

# wget ftp://ftp.kernel.org/pub/linux/kernel/v2.6/linux-2.6.23.1.tar.bz2 # wget http://www.kernel.org/pub/linux/kernel/projects/rt/patch-2.6.23.1-rt11.bz2

Patching the Sources

# tar xfj linux-2.6.23.1.tar.bz2 # cd linux-2.6.23.1 # bzcat ../patch-2.6.23.1-rt11.bz2 | patch -p1

configuring and build. . .

12 Install [rth06]

it is a kernel patch. . . thus something like this:

getting the sources:

# wget ftp://ftp.kernel.org/pub/linux/kernel/v2.6/linux-2.6.23.1.tar.bz2 # wget http://www.kernel.org/pub/linux/kernel/projects/rt/patch-2.6.23.1-rt11.bz2

Patching the Sources

# tar xfj linux-2.6.23.1.tar.bz2 # cd linux-2.6.23.1 # bzcat ../patch-2.6.23.1-rt11.bz2 | patch -p1

configuring and build. . .

let’s see “hello, world.” and kernel detection [rth06]

12 SCHED DEADLINE Linux real-time schedulers with static priorities:

I SCHED FIFO: first in first out no time slicing

I SCHED RR: round robin time slices with preemption

“Classic” Linux Schedulers

Normal scheduling policies

I SCHED OTHER standard round-robin time-sharing policy

I SCHED BATCH for ”batch” style execution of processes “middle priority”

I SCHED IDLE for running very low priority background jobs

14 “Classic” Linux Schedulers

Normal scheduling policies

I SCHED OTHER standard round-robin time-sharing policy

I SCHED BATCH for ”batch” style execution of processes “middle priority”

I SCHED IDLE for running very low priority background jobs

Linux real-time schedulers with static priorities:

I SCHED FIFO: first in first out no time slicing

I SCHED RR: round robin time slices with preemption

14 SCHED DEADLINE

I real-time scheduling class in Linux

I earliest deadline first (EDF)

I dynamic priorities

I shortest deadline = highest priority

I optimal (restrictions apply)

I taskset is schedulable iff U ≤ 1

I released in kernel 3.14 on March 30, 2014.

15 let’s see Linux implementation of EDF. . .

There are some issues with EDF

I critical section priority inheritance (deadline inheritance still a to-do

I precendence constraints shifting deadlines w.r.t. dependent tasks

I overloads admission control

I context switch for free “in theory there is no difference between theory and practice. . . ”

. . . see RT Scheduling lecture

16 There are some issues with EDF

I critical section priority inheritance (deadline inheritance still a to-do

I precendence constraints shifting deadlines w.r.t. dependent tasks

I overloads admission control

I context switch for free “in theory there is no difference between theory and practice. . . ”

. . . see RT Scheduling lecture

let’s see Linux implementation of EDF. . .

16 sched-deadline.txt [EDF]

0. WARNING ======

Fiddling with these settings can result in an unpredictable or even unstable system behavior. As for -rt (group) scheduling, it is assumed that root users know what they’re doing.

17 Scheduling Guarantee 3 parameters

I runtime

I period

I deadline

The task receives runtime time units within deadline if a proper admission control strategy is used.

WCET vs. runtime?

In the example: deadline = period

18 Tasks in CBS:

I Task cannot exceed its registered budget,

I Task cannot be unblocked when a ratio between remaining budget and remaining deadline is higher than bandwidth.

Constant Bandwidth Server Scheduling (CBS) [EDF]

EDF-based scheduling algorithm that allows:

I periodic (sporadic) tasks with hard deadlines utilization Up

I a CB server s with given utilization Us for “soft tasks” CBS ensures bounded utilization even in overload

I schedulable iff Us + Up ≤ 1

19 Constant Bandwidth Server Scheduling (CBS) [EDF]

EDF-based scheduling algorithm that allows:

I periodic (sporadic) tasks with hard deadlines utilization Up

I a CB server s with given utilization Us for “soft tasks” CBS ensures bounded utilization even in overload

I schedulable iff Us + Up ≤ 1

Tasks in CBS:

I Task cannot exceed its registered budget,

I Task cannot be unblocked when a ratio between remaining budget and remaining deadline is higher than bandwidth.

19 Therefore:

I struct sched attr with all the necessary fields

I sched setattr() and sched getattr() scheduling related syscalls that manipulate sched attr

Task interface

Specifying a periodic/sporadic task requires:

I a (maximum/typical) instance execution time,

I a minimum interval between consecutive instances,

I a time constraint by which each instance must be completed.

20 Task interface

Specifying a periodic/sporadic task requires:

I a (maximum/typical) instance execution time,

I a minimum interval between consecutive instances,

I a time constraint by which each instance must be completed.

Therefore:

I struct sched attr with all the necessary fields

I sched setattr() and sched getattr() scheduling related syscalls that manipulate sched attr

20 Admission control

I interface parameters are only used at admission control time when user calls sched setattr()

I system-wide settings under proc file system

I for a root domain comprising M CPUs, -deadline tasks can be created while the sum of their bandwidths stays below: M * (sched rt runtime us / sched rt period us)

I disable by writing -1 in /proc/sys/kernel/sched rt runtime us

21 minimal main. . . [EDF]

22 Thanks! ReferencesI

Deadline task scheduling. https: //www.kernel.org/doc/Documentation/scheduler/sched-deadline.txt. Accessed: Oct, 2015. Frequently Asked Questions: RTwiki. https://rt.wiki.kernel.org/index.php/Frequently_Asked_Questions, 2007. Accessed: Oct, 2015. HOWTO: Build an RT-application Jump to: navigation, search. https://rt.wiki.kernel.org/index.php/HOWTO:_Build_an_RT-application, 2007. Accessed: Oct, 2015. RT PREEMPT HOWTO. https://rt.wiki.kernel.org/index.php/RT_PREEMPT_HOWTO, 2006. Accessed: Oct, 2015. John A. Stankovic. Misconceptions about real-time computing. IEEE Computer, 21(10):10–19, 1988.

24