Mirage: ML kernels in the cloud (ML Workshop 2010)

Mirage: ML kernels in the cloudAnil Madhavapeddy, University of Cambridge

Thomas Gazagnaire, INRIA

Computer Laboratory, 15 JJ Thomson Avenue, Cambridge, UK

Contributions from Tim Deegan (Citrix), Steven Hand (Cambridge), Steven Smith (Cambridge) , Jon Crowcroft (Cambridge)

Motivation: Layers

HardwareHardware

ProcessesProcesses

OS KernelOS Kernel

ThreadsThreads

ApplicationApplication

Motivation: Layers

HardwareHardware

ProcessesProcesses

OS KernelOS Kernel

ThreadsThreads

ApplicationApplication

Language RuntimeLanguage Runtime

Motivation: Layers

HardwareHardware

ProcessesProcesses

OS KernelOS Kernel

ThreadsThreads

ApplicationApplication

HypervisorHypervisor

Language RuntimeLanguage Runtime

Motivation: Security

• Linux Kernel

• Mar 1994: 176,250 LoC May 2010: 13,320,934 LoC

Most core Internet services still written in C / C++

Mirage: Approach

• Construct an OS designed to run on the cloud

• End-to-end static type safety using OCaml + DSLs

• More build-time analysis (“whole OS optimisation”)

• Simple single-threaded core, using the hypervisor to divide up cores

Mirage: OS “signature” begins

module Console : sig type t val create : unit -> t val write : t -> string -> int -> int -> unitend

Mirage: A simple “hello world”

• Xen runs para-virtualized kernels, that cooperate with the hypervisor.

• Most code runs unmodified

• Privileged instructions (page table updates) go via Xen hypercalls

$ echo ‘let _ = print_endline “hello Xen world!”’ > hello.ml

$ ocamlopt –output-obj –o app.o hello.ml

• Linked to a “Xen MiniOS” to make a bootable kernel.

• Boots in 64-bit mode directly, with starting memory all mapped.

• Is approximately 50-100KB in size.

OS Text and DataOS Text and Data

Network BuffersNetwork Buffers

ReservedReserved

OCaml minor heap

OCaml minor heap

OCaml major heap

OCaml major heap

Mirage: 64-bit Xen Memory Layout

Mirage: Network Buffers

OS Text and DataOS Text and Data

Network BuffersNetwork Buffers

ReservedReserved

OCaml minor heap

OCaml minor heap

OCaml major heap

OCaml major heap

IP Header

TCP Header

Transmit packet data

IP Header

TCP Header

Receive packet data

Mirage: x86 superpages for OCaml heap

OS Text and DataOS Text and Data

Network BuffersNetwork Buffers

ReservedReserved

OCaml minor heap

OCaml minor heap

OCaml major heap

OCaml major heap

• Reduces TLB pressure significantly.

• Is_in_heap check is much simpler

• Improve GC/cache interaction using PAT registers?

OS Text and DataOS Text and Data

Network BuffersNetwork Buffers

ReservedReserved

OCaml minor heap

OCaml minor heap

OCaml major heap

OCaml major heap

Mirage: Typed Memory Allocators

Buddy Allocatordyn_init(type) dyn_malloc(type, size)dyn_realloc(size) dyn_free(type)

Heap Allocatorheap_init(type, pages)heap_extend(type, pages)heap_shrink(type, pages)

Page Grant Allocatorgrant_alloc_page(type)grant_free_page(type)

Mirage: concurrency

• Xen provides an low-level event interface.

• Optional interrupts: a perfect fit for co-operative threading!

• We always know our next timeout (priority queue)

• So adapted the excellent LWT threading library

Block 5s

Mirage: extending the OS signature with timing

Mirage: concurrency using LWT

• Advantages:

• Core library is pure OCaml with no magic

• Excellent camlp4 extension to hide the bind monad.

• Function type now clearly indicates that it blocks.

• Open Issues:

• Creates a lot of runtime closures (lambda lifting, whole program opt?)

• Threat model: malicious code can now hang whole OS

Mirage: and parallelism?

• Xen divides up cores into vCPUs, LWT multiplexes on a single core

• Mirage “process” is a separate OS, communicating via event channels

• Open Question: parallelism model (JoCaml, OPIS, explicit futures)

vCPU 1vCPU 1 vCPU 2vCPU 2Mem 1Mem 1 Mem 2Mem 2SHMSHM

Mirage: I/O

• I/O comes in via Ethernet frames, and “zero copy” parsed via a DSL.

• We have Ethernet, ARP, ICMP, IPv4, DHCP, TCPv4, HTTP, DNS, SSH all implemented in pure OCaml.

• Performance in user-space is excellent (EuroSys 2007), now benchmarking under Xen (submission to PLDI 2011).

• Zero-copy, bounds optimisation is vital to performance.

EthernetIP

TCPData

Mirage: I/O via MPL

packet tcp { source_port: uint16; dest_port: uint16; sequence: uint32; ack_number: uint32; offset: bit[4] value(offset(header_end) / 4); reserved: bit[4] const(0); cwr: bit[1] default(0); ece: bit[1] default(0); urg: bit[1] default(0); ack: bit[1] default(0); psh: bit[1] default(0); rst: bit[1] default(0); syn: bit[1] default(0); fin: bit[1] default(0); window: uint16; checksum: uint16; urgent: uint16 default(0); header_end: label; options: byte[(offset * 4) - offset(header_end)] align(32); data: byte[remaining()];}

OCaml output can both construct and parse packets from this DSL.

Melange: Towards a ‘Functional’ InternetEuroSys 2007, Madhavapeddy et al.

Mirage: I/O via metaprogramming

• Seeking a more general solution

• PADS

• MetaOCaml BER

• Requirements:

• Replace OCaml backend with LLVM parser for data plane (goal is 10GB/s type-safe network I/O).

• Would like to specify file-systems in this way also.

Mirage: a “multi-scale” operating system

type sockaddr = | TCP of ipv4_addr * int | UDP of ipv4_addr * int

module Flow : sig type t val read: t -> string -> int -> int -> int Lwt.t val write: t -> string -> int -> int -> int Lwt.t val connect: sockaddr -> (t -> unit Lwt.t) -> unit Lwt.t val listen: (sockaddr -> t -> unit Lwt.t) -> sockaddr -> unit Lwt.tend

Mirage: a “multi-scale” operating system

• Extremely portable signature permits it to adapt to many environments!

• POSIX/TUNTAP: normal OCaml runtime + Ethernet tap using ML network stack (I/O: Ethernet and higher)

• POSIX: with normal sockets (I/O: TCP/UDP and higher)

• Both use high performance kqueue/epoll backend.

• Javascript using ocamljs (Jake Donham) with WebSockets I/O

• Google AppEngine using ocamljava (Xavier Clerc) with HTTP I/O

• Android, iMotes using ocamlopt ARM backend.

Mirage: roadmap