RTOSReal-Time Operating Systems

Chenyang Lu

Consequence of Deadline Miss

ØHard deadlineq System fails if missed.

q Goal: guarantee no deadline miss.

Ø Soft deadlineq User may notice, but system does not fail.

q Goal: meet most deadlines most of the time.

Ø Since the application interacts with the physical world, its computation must be completed under a time constraint.

Ø CPS are built from, and depend upon, the seamless integration of computational algorithms and physical components. [NSF]

Cyber-Physical Systems (CPS)

3

Cyber-PhysicalBoundary

^ Robert L. and Terry L. Bowen Large Scale Structures Laboratory at Purdue University

Real-Time Hybrid Simulation (RTHS)

Cyber-Physical Systems (CPS)

4

Cyber-PhysicalBoundary

Interactive Cloud Services (ICS)

Need to respond within100ms for users to find responsive*.

5

Search the web* Jeff Dean et al. (Google) "The tail at scale." Communications of the ACM 56.2 (2013)

2nd phase ranking

Snippet generator

doc

Doc. index search

Response

Query

Interactive Cloud Services (ICS)

Need to respond within100ms for users to find responsive*.

E.g., web search, online gaming, stock trading etc.

6* Jeff Dean et al. (Google) "The tail at scale." Communications of the ACM 56.2 (2013)

Search the web

General-Purpose vs. Real-Time

ØGeneral-purpose systemsq Fairness to all tasks (no starvation)

q Optimize throughput

q Optimize average performance

ØReal-time systemsq Meet deadlines.q Fairness or throughput is not important

q Hard real-time: worry about worst case performance

Chenyang Lu 7

OS Support for Real-Time

ØReal-Time OSq Required for hard real-time systems

ØReal-time extensions to general-purpose OSq Suitable for soft real-time systems

Chenyang Lu 8

RTOS: Features for Efficiency

Ø Small

Ø Minimal set of functionalityØ Fast context switch

Ø Fast and time bounded response to interruptsØ Fixed or variable partitions of memory

q May not support paging or virtual memory

q May support locking code and data in memory

Ø Sequential file that can accumulate data at a fast rateq May be memory-based

Chenyang Lu 9

Code Size

Chenyang Lu 10

Name Code Size Target CPUpOSEK 2K MicrocontrollerspSOSystem PII->ARM ThumbVxWorks 286K Pentium -> Strong ARMQNX Nutrino >100K Pentium II -> NECQNX RealTime 100K Pentium II -> SH4OS-9 Pentium -> SH4Chorus OS 10K Pentium -> Strong ARMARIEL 19K SH2, ARM ThumbCreem 560 bytes ATMEL 8051

RTOS: Features for Real-Time

Ø Preemptive priority schedulingq At least 32 priority levels, commonly 128-256 priority levels

q Priority inheritance/ceiling protocol

Ø System callsq Bounded execution timesq Short non-preemptible code

Ø High-resolution system clockq Resolution down to nanosecondsq But it takes about a microsecond to process a timer interrupt

Chenyang Lu 11

Other Important Features

ØConformance to standardsq Real-Time POSIX API

q TCP/IP

ØModularity and configurabilityq Small kernel

q Pluggable modules

Chenyang Lu 12

Example: VRTX

Ø VRTXsaq RT-POSIX compliant

q Full real-time support

Ø VRTXmcq Optimized for power and footprint

Ø First RTOS certified by FAAq 100% code coverage in testing

Ø Runs the Hubble Space Telescope

Chenyang Lu 13

Real-Time Extensions to General OS

Ø Generally slower and less predictable than RTOS

Ø More functionality and development support

Ø Standard interfaces

Ø Suitable for soft real-time applications

Chenyang Lu 14

How Real-Time Is Linux?Ø I believe that Linux is ready to handle applications requiring sub-

millisecond process-scheduling and interrupt latencies with 99.99+ percent probabilities of success. No, that does not cover every imaginable real-time application, but it does cover a very large and important subset.

Ø There are patches out there that are expected to provide latencies in the tens of microseconds. These patches need some work, but are maturing quickly.

Paul McKenney, IBM Linux Technology CenterShrinking slices: Looking at real time for Linux, PowerPC, and Cell

Chenyang Lu 15

Why isn’t Linux real-time?Ø The Linux kernel only allows a process to preempt another under certain

circumstances:q when the CPU is running user-mode code;q when kernel code returns from a system call or an interrupt to user space;q when kernel code blocks on a mutex, or explicitly yields control to another

process.

Ø If kernel code is executing, a high priority thread cannot preempt the kernel code until it yields control.

Ø In the worst case, the latency could potentially be hundreds milliseconds or more.

Chenyang Lu 16

Source: https://rt.wiki.kernel.org/

Linux Scheduling

Ø Real-time scheduling classq Fixed priority

• SCHED_FIFO: First-In-First-Out• SCHED_RR: Round-Robin

q Earliest Deadline First• SCHED_DEADLINE

Ø Non-real-time scheduling class (SCHED_NORMAL)

Ø Defaultq Real-time: 0 – 99

q Non-real-time: 100 – 139

Chenyang Lu 17

Real-time Linux

Ø Core kernel modifications: patchesq Require modifications to Linux kernel

q Example: CONFIG_PREEMPT_RT patch

Ø Compliant kernels: modified native RTOSq Linux binaries can run without modificationsq Example: LynxOS

Ø Dual kernels: real-time kernel sits below Linuxq Real-time kernel traps all interrupts and schedules all processes

q Linux runs as a low-priority process

q Example: RT-Linux (FSLabs)

Chenyang Lu 18

CONFIG_PREEMPT_RT PatchØ Convert Linux into a fully preemptible kernel.

Ø Locksq Make in-kernel locking-primitives preemptible through reimplementation with

rt-mutexes.

q Critical sections protected by spinlock_t and rwlock_t are preemptible. q Priority inheritance for in-kernel spinlocks and semaphores.

Ø Convert interrupt handlers into preemptible kernel threads.

Ø Convert timer API into separate infrastructures for high resolution kernel timers plus one for timeouts, leading to user space POSIX timers with high resolution.

Chenyang Lu 19

Source: https://rt.wiki.kernel.org/

Multi-core Real-Time Scheduler Ø Push-pull scheduler schedules tasks across CPUs.

Ø Every CPU has a runqueue.q Avoid contention over a global runqueueq Need to enforce global priority order in a decentralized fashion

Ø Push: considers all the runqueues to find the one of a lower priority than the task being pushed.q Push a task when it wakes up on a runqueue running a task of higher priorityq Push a task when a higher-priority task on the same runqueue wakes up and

preempts it

Ø Pull: whenever a runqueue is about to schedule a task that is lower in priority than the previous one, it checks whether it can pull tasks of a higher priority from other runqueues.

Chenyang Lu 20

Chenyang Lu 21

Example: Push-Pull

Source:Ankita Garg, Real-Time Linux Kernel Scheduler, Linux Journal, 8/2009. https://www.linuxjournal.com/article/10165

https://www.linuxjournal.com/article/10165

RT-Linux

Chenyang Lu 22

Linux Process

Linux Kernel

Real-Time Kernel

Linux Process

RT Process

Software Interrupts

Hardware Interrupts

Scheduling

• Real-time kernel traps all interrupts and schedules all processes• Linux runs as a low-priority process• Real-time applications cannot take advantage of Linux calls

More Information

Ø Real-Time Linux Wiki: https://rt.wiki.kernel.org/index.php/Frequently_Asked_Questions

Ø Real-Time Linux Kernel Scheduler: http://www.linuxjournal.com/magazine/real-time-linux-kernel-scheduler

Chenyang Lu 23

https://rt.wiki.kernel.org/index.php/Frequently_Asked_Questionshttp://www.linuxjournal.com/magazine/real-time-linux-kernel-scheduler