Realizing Compositional Scheduling through Virtualization

Jaewoo Lee, Sisu Xi, Sanjian Chen, Linh T.X. Phan Chris Gill, Insup Lee, Chenyang Lu, Oleg Sokolsky

2

Virtualization The benefits of virtualization

Consolidate legacy systems Integrate large, complex systems

Key challenges of virtualization for safety-critical systems Temporal isolation Real-time guarantee

Hypervisor

Legacy System

Virtualization Platform

Domains

Legacy System

3

Compositional Scheduling Compositional Scheduling Framework (CSF)

Provides temporal isolation and real-time guarantee Computes the minimum-bandwidth resource model for the component

The gap between CSF theory and system Realizing CSF though virtualization can bridge the gap

Resource Model Resource Model

Resource Model

Parent component

Child components

Workload Workload

Periodic Tasks

Component

Scheduler

Rate MonotonicScheduler

Scheduler

Periodic Resource Model (period, budget)

4

Contributions Compositional Scheduling Architecture (CSA)

Confederation of compositional scheduling and virtualization• Enhancement to periodic server design in CSA • Extension to CSF for quantum-based platforms

Performance evaluation of CSA Synthetic workloads and avionic workloads

First open-source real-time virtualization with CSF Extensible with new domain-scheduling algorithms

5

Overview of Our Work Compositional Scheduling Architecture (CSA)

Component domain Periodic Resource Model (PRM) Periodic Server (PS) Task model: independent, CPU-intensive, periodic task Scheduling algorithm: rate monotonic

App App App App

Domain

Hypervisor

RT-Xen

Hardware

Task Task Task Task

Component Component

Root Component

Compositional Scheduling

PSPRM PRM

S. Xi, J. Wilson, C. Lu, C. Gill, RT-Xen: Real-Time Virtualization Based on Hierarchical Scheduling, EMSOFT, 2011

DomainPS

6

Theory Pessimism in CSF Interface considers the worst case: UPRM – UW ≥ 0

Interface overhead leads to underutilization of the component

Resource model periodic server in CSA

Periodic server does not fully utilize its budget Slacks : tasks do not always execute for WCETs Interface overhead

Underutilization of periodic server long response times of real-time tasks

Using idle times, we propose enhanced periodic servers

Interface Overhead

7

Periodic Server Design Purely Time-driven Periodic Server (PTPS)

If currently scheduled domain is idle, its budget is wasted Not work-conserving

tΔ

DH

DL

Budget

Budget

time

Task Release

Task Complete

Execution of tasks in DH

Execution of tasks in DL

Current Domain

8

Periodic Server Design Work-Conserving Periodic Server (WCPS)

If currently scheduled domain is idle, the hypervisor picks a lower-priority domain that has tasks to execute

Early execution of the lower-priority domain during idle period does not affect schedulability

t Δ

DH

DL

Budget

Budget

time

Task Release

Task Complete

Execution of tasks in DH

Execution of tasks in DL

Current Domain

9

Periodic Server Design Capacity Reclaiming Periodic Server (CRPS)

If currently scheduled domain is idle, we can re-assign this idled budget to any other domain that has tasks to execute

Early execution of the other domain during idle period does not affect schedulability

t Δ

DH

DL

Budget

Budget

time

Task Release

Task Complete

Execution of tasks in DH

Execution of tasks in DL

Current Domain

10

CSF Extension for Quantum-based Platforms

P:

To find the minimum-bandwidth resource model for workload W.

Real-number-based resource model

3 tasks

Task period Task execution time

of resource model

of

res

ou

rce

mo

de

l

B/P:

11

CSF Extension for Quantum-based Platform

infeasible bandwidth

Real-number-based resource model

Quantum-basedresource model

Necessary condition for schedulability

To find the minimum-bandwidth resource model for workload W.

the upper bound of the period to find min-BW resource model?

Non-decreasing

P: of resource model

of

res

ou

rce

mo

de

l

B/P:

1 2

12

CSF Extension for Quantum-based Platform

the upper bound of the period to find min-BW resource model?

infeasible bandwidth

Non-decreasing

Real-number-based resource model

Quantum-basedresource model

Necessary condition for schedulability

To find the minimum-bandwidth resource model for workload W.

Min-BW resource model

P:

B/P:

1 2of resource model

of

res

ou

rce

mo

de

l

13

System Architecture Implemented in Xen 4.0

only re-compile Xen, keep Kernel untouched

All source code available at RT-Xen website: https://sites.google.com/site/realtimexen/

Current Limitations: one VCPU per domain (single core) CPU intensive workload

Xen Scheduling Framework

Real-Time Sub Framework

PTPS WCPS CRPS

14

Evaluation – Setup

VCPU

Core 0 Core 1HW

Schedulers (PTPS, CRPS, WCPS)

Dom0

AppApp

VCPU

Dom1

VCPU

Dom5

AppApp

Scheduling Algorithms(PTPS, CRPS, WCPS)

Parameters for each Domain

IDLE

…

Responsiveness: response time / deadlineDeadline Miss Ratio

Use Rate Monotonic within Domain

…

15

Evaluation – Synthetic workloads Randomly generate task sets, then compute interface Sources of idle time:

interface overhead: UPRM – UW

slacks: over-estimation of tasks’ execution time

Range workload periods -> different interface overhead UW: 0.7, 0.8, 0.9, 1.0 Periods: [550ms, 650ms], [350ms, 850ms], [100ms, 1100ms]

Range Execution Time Factor (ETF) -> different slacks For all tasks in highest three priorities domains: 100%, 50%, 10% Using period [550ms, 650ms], pick Uw from 0.7, 0.8, 0.9, 1.0

typical overloaded situation

most interface overhead

uniform distribute [wcet*ETF, wcet]

extremely overloaded situation

16

Evaluation – Interface Overhead

100%

60%

0%

CRPS ≥ WCPS ≥ PTPS

deadline miss

CD

F P

lot,

Pro

bab

ilit

y

17

Evaluation – ETF

SchedETF = 100 % ETF = 50 % ETF = 10 %

median 95 % max median 95 % max median 95 % max

PTPS 3 3 3 3 3 3 3 3 3

WCPS 3 3 3 0.5045 3 3 0.3996 3 3

CRPS 0.6192 3 3 0.0860 0.3213 0.7608 0.0573 0.1807 0.4864

( Response Time / Deadline ) for the Lowest Priority Domain

PTPS: non work conservative, can not utilize slacks

WCPS: consumes budget in parallel, still miss deadlines

CRPS: ‘reclaim’ budget more aggressively, utilize slacks effectively

18

Evaluation – ARINC-653 Workload 7 harmonic workloads, each represent a set of domains

scheduled on a single processor, with each domain consisting of a set of periodic tasks

A. Easwaran, I. Lee, O. Sokolsky, and S. Vestal, A Compositional Framework for Avionics (ARINC-653) Systems, Tech Report MS-CIS-09-04, 2009, University of Pennsylvania

UPRM = UW

if using real number parameters Overheads comes from

rounding up the budget period is fixed

CRPS > WCPS > PTPS

Interface Overhead (8%)

CD

F P

lot,

Pro

babi

lity

Response Time / Deadline

19

Conclusion Compositional Scheduling Architecture (CSA)

Enhanced version of the Pure Time-driven Periodic Server (PTPS)• WCPS: work conserving, consume budget in parallel• CRPS: aggressively reclaiming budget

Extension of CSF for quantum-based platforms

Extensive evaluation on synthetic and avionics workloads

Open Source: RT-Xen Website: https://sites.google.com/site/realtimexen/

20

Questions?

RT-Xenhttps://sites.google.com/site/realtimexen/

or just Google RT-Xen *^_^*

21

Backup : Interface overhead Interface considers the worst case: UPRM – UW ≥ 0

For example, a task T= (p = 3, e = 1) in a component• Resource model (3, 1) cannot schedule T• Resource model (3, 2) can schedule T

UPRM – UW

=2/3 – 1/3 = 1/3

Interface Overhead

0 1 2 3 4 5 6

Task Release

Task Deadline

0 1 2 3 4 5 6

Deadline miss

Resource supply ofresource model (3,1)

Resource supply ofresource model (3,2)

1st period of the resource supply

2nd period of the resource supply

Supplied resource

Backup : Simple rounding up does not work

The minimum-bandwidth resource model CSF allow real number in budget. But, budget should be an integer multiple of the scheduling

quantum in quantum-based platforms Ex:

Optimal algorithm : (1,0.38) rounding up (1,1) Only integer :

• (1,1), (2,1), (3,2) , (4,2),…• Among feasible resource

models, (5,2) is minimum bandwidth for quantum-based platforms

23

Backup: Upper bound of the period

P

B/P

infeasible bandwidth

Non-decreasing

Real-number-based resource model

Quantum-basedresource model

Necessary condition for schedulability

We can easily find the upper bound of the period for a given bandwidth

A given bandwidth

The upper bound of the period for a given bandwidth

24

Backup : Difference from reservation-based system CSA on RT-Xen virtualization

Support different local scheduler for each domain ( by running different guest OS)

Clean separation between local scheduler and global scheduler• Local OS does not know other task or other domain inside the system• Global scheduler does not know task information inside domain

Reservation-based native system Local scheduler is a part of the operating system

• We cannot provide a component with a local scheduler No clean separation between local scheduler and global scheduler

• Malformed local scheduler can affect global scheduler or other local schedulers

25

Backup : Related Work Crespo et al., “XtratuM”, EDDC ’10

A bare VMM with a fixed cyclic scheduling policy Cucinota et al., “Respecting Temporal Isolation...”, COMSAC

’09 KVM with a hard reservation behavior

CSA is different from above two works Only CSA support compositional scheduling CSA is implemented on Xen, different architecture from KVM

• KVM is integrated into the manager domain

26

CSF Extension for Quantum-based Platforms