ARM Architecture-based System Virtualization: Xen ARM open ......TcS1 TcS2 TcS3 Test Cases Test Environment Secure Xenon ARM Domain0 (IDD) Domain1 Linux kernel v2.6.21 I/O ACM Linux

© 2011 SAMSUNG Electronics Co.

Sang-bum Suh Vice President, Jae-min Ryu Research Staff

{sbuk.suh, jm77.ryu}@samsung.com

S/W Platform Team

DMC Research Center

SAMSUNG Electronics Co.

2011.08.

Xen Summit North America 2011

ARM Architecture-based System Virtualization: Xen ARM open source software project

mailto:[email protected]




Agenda

Overview

- History of Xen ARM

- Use Cases

Xen ARM: Core - Xen ARM Virtualization

- Performance Comparison

Xen ARM Application: Security - Mobile Malware

- Access Control

Xen ARM Application: Real-time - Xen ARM: Preemption

- Real-time Performance


Overview

3 / 22 SW Platform Team. © 2011 SAMSUNG Electronics Co.

History of Xen ARM

Supported Hardware & Guest OS

Samsung leads the Xen ARM project

http://wiki.xensource.com/xenwiki/XenARM

Xen-ARM Open Source Community

‘04 ‘10 ‘09 ‘08

x86 Xen Hypervisor Release (Cambridge university)

Xen ARM 1st Release: ARM9 Xen Hypervisor, Mini-OS (Samsung)

Xen ARM 2nd Release: Paravirtualized Linux kernel (v2.6.24), Xen tool (Samsung)

Xen ARM 4th Release: Performance Optimization (Samsung)

Xen ARM 3rd Release: ARM11MPCore Support (Samsung)

ARM926EJ-S (i.MX21, OMAP5912) Xscale 3rd Generation Architecture

(PXA310, Samsung SGH- i780) ARM1136/ARM1176(Core Only) Goldfish (QEMU Emulator) Versatile Platform Board ARM11MPCore (Realview PB11MP) Cortex-A9 (Tegra250)

Linux v2.6.11, v2.6.18, v2.6.21, v2.6.24, v2.6.27

(multicore supported) uC/OS-II

http://wiki.xensource.com/xenwiki/XenARM


Use Cases – HW Consolidation: AP(Application Processor) and BP(Baseband Processor) can share

multicore ARM CPU SoC in order to run both Linux and Real-time OS efficiently.

– OS Isolation: important call services can be effectively separated from downloaded third party applications by Xen ARM combined with access control.

– Rich User Experience: multiple OS domains can run concurrently on a single smartphone.

Hardware

Hypervisor

Secu

re

Kern

el

Linux

Andro

id

Nucle

us

Linux 2

Hypervisor H/W

Linux 1 Important services

Rich Applications from Multiple OS Secure Smartphone AP SoC +BP SoC -> Consolidated Multicore SoC

1

2

3

1 2 3

GPOS RTOS

Multi - Core

Core Core Core Core

Memory Peri

Virtualization SW ( Realtime Hypervisor) V - Core V - Core V - Core V - Core V - Core V - Core V - Core V - Core

를

Mirage_Fault_Isolation_demo_update3 (2).avi

vws2010_fullscreen.wmv

Mirage_Fault_Isolation_demo_update3 (2).avi


Xen ARM: Core


Xen ARM Virtualization

Light weight virtualization for secure 3G/4G mobile devices

High performance hypervisor based on ARM processor

Fine-grained access control fitted to mobile devices

CPU Peripheral Devices

Resource Allocator

Domain Manager

Access Control

Application Application Application Application

VM Interface VM Interface

Backend Drivers Frontend Drivers

Native Drivers

System Memory

Flash Memory

VM 0 VM 1

Hardware

Secure Xen-ARM hypervisor

Guest Domain

Goals

Architecture of Xen-ARM


Xen ARM Virtualization

CPU virtualization

Virtualization requires 3 privilege CPU level, but ARM supports 2 level

Xen-ARM mode: supervisor mode ( most privileged level)

Virtual kernel mode: User mode ( least privileged level)

Virtual user mode: User mode ( least privileged level)

Memory virtualization

VM’s own memory should be

protected from others

Xen-ARM switches VM’s virtual address space

using MMU

VM is not allowed to manipulate MMU directly

I/O virtualization

Split driver model of Xen-ARM

Client & Server architecture for shared I/O devices

Client: frontend driver

Server: native/backend driver

Logical mode split

virtual user mode

Xen-ARM mode

virtual kernel mode

Xen-ARM

VM 0 Address Spaces

VM 1 Address Spaces

VM 2 Address Spaces

MMU

Xen-ARM

Application Application Application Application Application Application

Native driver

Front-end driver

Back-end driver

Device

Interrupt I/O event

VM0 (Linux) VM1 (Linux )

Xen-ARM

VM 1

VM 0

Xen-ARM

Physical Address Space

Virtual Address Space

User Process User Process User Process

Kernel

User Process

VM 2

Overview


Performance Comparison

LMBENCH Micro Benchmark ( latency )

LMBENCH Micro Benchmark ( Bandwidth ) Evaluation Environments : Samsung

Blackjack Phone

CPU : Xscale PXA310, 624MHz

L1 Cache : 32KB + 32KB

L2 Cache : 256KB (Disabled)

Memory : 128MB

Guest OS: Linux-2.6.21

Higher is better

Lower is better

Micro-benchmark Results


0

200

400

600

800

1000

1200

1400

1600

Native Linux Para-virtualized Linux



Evaluation Environments : nVidia Tegra250

CPU : Cortex-A9 1GHz Dual Core


L2 Cache : 1MB

Memory : 1GB


Lower is better

Micro-benchmark Results

(Late

ncy

) use

c


0

1

2

3

4

5

6

7

8

9Xen/ARM L4

Rela

tive

Perform

ance

S : size(byte) P : # of processes

Higher is better



Evaluation Environments : Samsung Blackjack Phone

CPU : Xscale PXA310, 624MHz


L2 Cache : 256KB (Disabled)

Memory : 128MB


Benchmark Results

0

0.2

0.4

0.6

0.8

1

1 2 3

Native Linux

Xen/ARM

L4

Number of Tasks

Norm

alize

d P

erform

ance

AIM7 Macro Benchmark


Xen ARM Application For Security


Mobile Malware

0

200

400

600

2004 2005 2006 2007 2008

27

146

345 400 421

[Source: F-Secure] Number of mobile malware

- More than 420 mobile phone viruses (2008)

- Tens of thousands of infections worldwide

Concerns about mobile phone security – by market

0%

20%

40%

60%

80%

100%

UK US Japan Total

83.9 81.693.1 86.1

16.1 18.46.9 13.9

Feel safe

Concerned

[Source: McAfee]


Access Control

Technical Issues

Availability threat: denial of service (DoS) attack from a compromised domain in a mobile device

- CPU overuse: a greater share of CPU time than initial allocation

- Performance degradation: The Performance of other domains that share the same I/O device with the compromised domain

- Battery drain

Hardware

Secure Xen

DomainN

…

Devices

Domain1

Isolated Driver Domain (IDD)

Excessive I/O Requests

Malware

Front-end Device drivers

Front-end Device drivers

Back-end Device drivers

Native Device drivers

Definition: Access control is a system which enables an authority to control area and resources in a given physical facility or computer-based information system [source: Wikipedia]

Problem with performance isolation


Access Control

Technical Issues

Fine-grained I/O access control module in the IDD and coarse-grained access control module in Xen

Estimation of CPU consumption by each virtual I/O operation using regression analysis

- Network and storage devices

I/O access control enforcement based on the policy and regression equations

Domain0 (IDD) Kernel Domain1 Kernel

Frontend driver

Backenddriver

I/O Resource Accounting

Access Control Enforcer

(I/O Scheduler)

Secure Storage

Secure Xen on ARM

(1). Loading policy

(2). Loading equations

(3). Connecting to backend driver

(4). Asking for decision

(5). Virtual I/O CPU Usage

Hook

fMTD,READ(x) = 250 + 0.24 * x fMTD,READOOB(x) = 533 + 0.06 * x fMTD,WRITE(x) = 160 + 0.24 * x fMTD,WRITEOOB(x) = 583 fMTD,ERASE(x) = 153.33 + 0.02 * x fMTD,ISBADBLOCK(x) = 58 fMTD,MARKBAD(x) = 60

fNET, Tx(x) = 370.18 + 0.01 * x

Regression Analysis: Network

Regression Analysis: Storage

Target HW spec: XScale 624MHz, 128MB DRAM

x: bytes, f(x): usec

Approach


Access Control

Technical Issues

Network I/O Test Cases

Storage I/O Test Cases

No Attack TcN0 TcS0

Under Attack (No I/O ACM)

TcN1 TcS1

Under Attack (20% I/O ACM Policy)

TcN2 TcS2

Under Attack (10% I/O ACM Policy)

TcN3 TcS3

CPU Utilization: Storage

CPU Utilization: Network

0

20

40

60

80

100

3 6 9 12 15 18 21 24 27 30

TcN0

TcN1

TcN2

TcN3

0

20

40

60

80

100

3 6 9 12 15 18 21 24 27 30

TcS0

TcS1

TcS2

TcS3

Test Cases

Test Environment

Secure Xen on ARM

Domain0 (IDD) Domain1

Linux kernel v2.6.21

I/O ACM

LinuxKernelv2.6.21

iperf(client) Policy

Managerbonnie mtd_atk

net_atk

Linuxkernel

iperf(server)

minicom

SGH-i780

WT3000 power meter

Linux PC

Serial Cable MeasurementCable

net_atk: UDP packet flooding (sending out UDP packets with the size of 44,160 bytes every 1msec) mtd_atk: excessive NAND READ operations (scanning every directory in the filesystem and reading file contents)

CPU Usage (%)

CPU Usage (%)

Time (Sec)

Time (Sec)

Power Meter SGH-i780

Effectiveness


Access Control

0

100

200

300

400

500

600

700

800

TcN0 TcN1 TcN2 TcN3

UDP

TCP

Test Cases

Thro

ugh

pu

t (K

B/S

ec)

No attack

Under attack

0

500

1000

1500

2000

2500

3000

3500

4000

4500

TcS0 TcS1 TcS2 TcS3

Seq.out

Seq.in

Rand.seek

Test Cases

Thro

ugh

pu

t (K

B/S

ec)

No attack

Under attack

0

0.5

1

1.5

2

2.5

3

TcN0/TcS0 TcN1/TcS1 TcN2/TcS2 TcN3/TcS3

Network

Storage

Test Cases

No attack Under attack

Test Cases

Power Consumption

Throughput: Storage

Throughput: Network

Throughput increase and power consumption decrease even under malware attack

16

Effectiveness


Xen ARM Application For Real-time


Xen ARM: Pre-emption

Technical Issues

Latency caused by interrupt-disabled(DI) section Latency caused by non-preemptible(NP) section

EN DI EN

interrupt handler

Urgent Interrupt

Time

P NP P

Urgent Interrupt Wake up next VM

VM switch

interrupt handler

Time

Delay Delay

interrupt handler

VM switch

The jitter of timer interrupt latency by the hypervisor is bounded within 10% compared with native real-time OS.

DI and NP sections should be minimized.

Status of real-time support

Technical Issue

Guest Domain

Xen-ARM

Non RT Domain

(Running)

Interrupt

RT Domain

Interrupt occurs

Non RT Domain

(Running)

Timer

Send IRQ to Real-time Domain (As Non RT Domain has credits,

RT Domain is not scheduled. Furthermore Xen cannot tell difference

between Non RT Domain and RT Domain.)

Delay

Voluntary CPU release

Non RT Domain

(Running)

Interrupt

RT domain Interrupt

occurs

Scheduling Scheduling

Non RT domain

Scheduling

Preemption

time

Hypervisor should support RT Domain via priority-based scheduling, VMM pre-emption and so on.


Real-time Performance

Cyclictest benchmark repeats 1. RT task sleeps for 10ms 2. Timer interrupt will occur after 10ms 3. Timer interrupt wakes up the RT

domain(uC/OS-II) 4. uC/OS-II preempts Xen-ARM 5. RT task is scheduled 6. RT task logs timestamp

Category Description

H/W (Tegra250)

CPU Cortex-A9 / 1GHz / Dual Core

RAM 1GB

S/W Hypervisor Xen-ARM

Guest OS (DOM0)

Linux-2.6.29 (Running Busy Loop Task)

Guest OS (DOM1)

uC/OS-II (Running RT Task : Cyclictest benchmark)

Native(uC/OS-II)

Min Avg Max

9995 9996.810169 10000

Xen-ARM(uC/OS-II)

Min Avg Max

9996 9999.327119 10001

Unit : usec

Response Overhead(3us)

• Evaluation Environment


Q & A

Documents

ARM Architecture-based System Virtualization: Xen ARM open ......TcS1 TcS2 TcS3 Test Cases Test Environment Secure Xenon ARM Domain0 (IDD) Domain1 Linux kernel v2.6.21 I/O ACM Linux