K42 Internals

6/10/2005 FastOS PI Meeting/Workshop

K42 Internals

Dilma da Silva for K42 group

IBM TJ Watson Research


K42 Goals

• Scalability

• Flexibility/customizability

• Framework for OS research– Linux API and ABI and kernel module compatible– Available to collaborators under LGPL

• Maintainability/extensibility


Structure

• OS function in user-level library– examples: timers, thread

library...– allows OS services to be

customized for applications with specialized needs

– avoids interactions with kernel/servers, reduces space/time overhead in kernel/servers

• Object-oriented design at all levels Micro-kernel

Servers

Linux emulation

OS lib

Application

Linux emulation

OS lib

Application


Object-oriented design

standard OS

webserver database

parallel application web

server database

parallel application


OO Design: MM Objects

Process

FCM

FCM

FR

FR

HAT

RegionRegionRegion

RegionRegion

FCM: File Cache ManagementFR: File RepresentativeHAT: Hardware Address Translation


Process

FCM

FCM

FR

FR

Region

Region

Process

FCM FRRegion

Region


Region Region Region Region

Region Region

• OO design enables specialized implementations

Specialization and Clustered Objects


Clustered object infrastructure

• Per-processor level of indirection avoids any centralized bottleneck

• Provides library to simplify distributed implementation– Caching of local state– Incremental distribution of services– DHash

Focus on locality!• Read-Copy-Update techniques

– Use non-blocking synchronization– Avoid locking hierarchies


Hot-swapping

• change one system component/type for another without bringing system down

• potential uses– scalability– performance– monitoring– extensibility– testing

• Dynamic update: system availability


Hot-Swapping/Dynamic Update

• Depends on:– object-oriented structure of system– technology to establish a quiescent state– level of indirection

• Implementation has no additional overhead– when not swapping an object– for objects not being swapped

• Limitations:– is not instantaneous– initial prototype– Current work: coordinated swapping, interface changes, external

policies...


Independent workloadsmodified SDET

0

5000

10000

15000

20000

25000

30000

35000

40000

-1 4 9 14 19 24

Processors

Scri

pts

/H

ou

r

Linux

K42

Parallel PostMark

0

2

4

6

8

10

12

14

16

18

-1 4 9 14 19 24

Processors

Speedu

p

LinuxK42

Parallel Make (flex)

0

5

10

15

20

25

-1 4 9 14 19 24

Processors

Speedu

p

LinuxK42


More hot-swapping

0

20

40

60

80

100

120

140

160

180

0 1 2 3 4 5 6

Number of concurrent background streams

SDET

thro

ughp

ut (s

crip

ts/h

our)

LRU

Adaptive

0

200

400

600

800

1000

1200

1400

1600

1800

Shared Shared-Exclusive Shared-Exclusive /Small-Large

Tran

sact

ions

per

sec

ond

Adaptive paging Adaptive file imp.


Memclone benchmark: Memory intensive parallel application

0

500

1000

1 5 9 13 17 21

Linux 2.4.21

K42

0

500

1000

1 5 9 13 17 21

ms

ec

pe

r th

rea

d

All MM objects distributed

All MM objects shared


Performance monitoring

Emulation Layer FaultsFork-Child Faults

Process Init Faults

K42 App Faults

Emulation Layer

Process Init

Fork-Child SetupMisc. Kernel Time

Process CleanupK42 App Time

Linux User App Faults

Linux User App Time

• unified cheap scalable non-blocking tracing infrastructure for correctness and performance debugging

• key parts of design transferred to LTT• post processing tools easy to develop: lock contention, sampling, time

breakdown, visualization, caching effects, ... • in final numbers, disabled, but only %1.6 difference UP and 24 way


Status

http://www.research.ibm.com/K42

[email protected]

Documents

K42 Internals