OSE 2011– processes & address spaces (lec2)1

Operating Systems Engineering

Processes & Address Spaces

By Dan Tsafrir

OSE 2011– processes & address spaces (lec2)2

Reminder: goal of “processes”

Give each process a private memory area For code, data, stack Illusion of its own dedicated machine

Prevent each process from reading/writing outside its address space

But allow sharing when needed

Bring a program to life

OSE 2011– processes & address spaces (lec2)3

HW & OS collaboration

OS manages processes (De)allocate their physical memory (create, grow, remove) Switch between them (multiplex HW) Keep track of them when they’re not executing Configure HW

HW performs address translation & protection Translate user addresses to physical addresses Detect & prevent accesses outside address space Allow cross-space transfers (system calls, interrupts, …)

Also OS needs its own address space OS should be able to easily read/write user memory

OSE 2011– processes & address spaces (lec2)4

HW & OS collaboration

HW support not necessarily corresponds well to what the OS wants (Linux/PPC vs. AIX/PPC)

Two main approaches Segments & page tables

Paging has won Most OSes are designed for paging Most modern CPU designs support only paging

BUT x86 provides many features only via segmentation HW

(interrupts, protection) So we must learn about x86 segments, a little bit Until recently xv6 used segments (rev 5 changed that)

OSE 2011– processes & address spaces (lec2)5

Case study: Unix v6

Early Unix OS for DEC PDP11 By Thompson & Ritchie, 1975 From Bell labs; the 1st version to be widely used outside

Bell Ancestor of all Unix flavors (Linux, *BSD, Solaris,…) (But much smaller) Written in C Recognizable (shell, multiuser, files, directories, …) Today’s Unix flavors have inherited many of the conceptual

ideas, even though they added lots of stuff (e.g., graphics) and improved performance

OSE 2011– processes & address spaces (lec2)6

Case study: Unix v6

1976 Commentary by Lions:a classic….

Despite its age still consideredan excellent commentary on simple but high quality code

For many years, was the onlyUnix kernel documentation publicly available

v6 allowed classroom use ofsource code; v7 & onwardsdidn’t

Commonly held to be one of the most copied books in computer science.

Reprinted in 1996; still sold

OSE 2011– processes & address spaces (lec2)7

Case study: Unix v6

We could have used it for OSE

But PDP11 Old K&R C style Missing some key issues in

modern OSes (such as paging)

Luckily…

OSE 2011– processes & address spaces (lec2)8

Case study: xv6

xv6 is an MIT reimplementation of Unix v6 Runs on x86 (well if you insist; we’ll use QEMU) Even smaller than v6 Preserves basic structure (processes, files, pipes, etc.) Runs on multicores Just this year (2011) got paging

To “get it”, you’ll need to read every line in the source code a few times; but it’s really is not that hard

Student’s goal: to explain to yourself every line; the (still rough) commentary on the webpage will greatly help

OSE 2011– processes & address spaces (lec2)9

Case study: xv6

First half of course, each lecture we will take one part of xv6 and study its source code (homework assignments will help you prepare for these lectures)

About half-way the term, you’ll understand the source code for one well-designed OS for an Intel-based

2nd half covers important OS concepts invented after Unix v6 We will study the more modern concepts by reading research papers & discussing them in lecture

You may also implement some of these newer concepts in your OS (JOS)

OSE 2011– processes & address spaces (lec2)10

xv6 – why?

You may wonder why we’re studying an aging OS instead of, say, Linux, or Windows, or FreeBSD, or Solaris, …

xv6 is big enough to illustrate the basic OS design & implementation

Yet xv6 is far smaller => much easier to understand Structure similar to many modern OSes

Once you've explored xv6 you’ll find your way inside kernels such as Linux

This is, after all, OSE… JOS occupies a very different point in the design &

implementation space from xv6 Finally, “getting” xv6 will help you build your own (J)OS

OSE 2011– processes & address spaces (lec2)11

xv6 address space

xv6 enforces memory address space isolation No process can write on another’s space or the kernel’s

xv6 process’s memory starts at 0 & (appears) contiguous C, as well as the linker, expect contiguity

x86 defines three kinds of memory addresses Virtual (used by program), which is transformed to Linear (accounting for segments), which is transformed to Physical (actual DRAM address)

xv6 nearly doesn’t use segments All their bases set to 0 (and their limits to the max) virtual address = linear address Henceforth we’ll use only the term “virtual”

OSE 2011– processes & address spaces (lec2)12

Paging in a nutshell

Virtual address (we assume 32bit space & 4KB pages):

Given a process to run, set cr3 = pgdir (“page directory”) Accessing pgdir[pdx], we find this PDE (page directory entry)

Flags (bits): PTE_P (present), PTE_W (write) , PTE_U (user) 20bits are enough, as we count pages of the size 2^12 (=4KB) The “page table” pgtab = pgdir[pdx] & 0x ffff f000

An actual physical address Accessing pgtab[ptx], we find this PTE (page table entry)

Target physical address = (pgdir[pdx] & 0x ffff f000) | (virt_adrs & 0x 0000 0fff)

pdx (10bit) ptx (10bit) page offset (12bit)

physical address (20bit) flags (12bit)

physical address (20bit) flags (12bit)

OSE 2011– processes & address spaces (lec2)13

xv6 virtual memory

Each process has its own unique pgdir (= address space) When the process is about to run, the cr3 register is

assigned with the corresponding pgdir Every process has at most 640 KB memory

Not more that 160 pages For each process, for VA higher than 640 KB, xv6 maps the

VA to an identical PA => Every process’s address space simultaneously

contains translations for both all of the process’s mem & all of the kernel’s mem

PTEs corresponding to addresses higher than 640KB have the PTE_U bit off, so processes can’t access them

The benefit: the kernel can use each process’s address space to access all of its memory

OSE 2011– processes & address spaces (lec2)14

xv6 virtual memory

Assume a process P size if 12KB (3 pages), and that it wants to sbrk another page (sbrk is the syscall that dynamically allocates memory)

Assume the free page xv6 decides to give P is in PA: 0x 2010 0000

Thus, to ensure contiguity, the 4th PTE of P, which covers VAs: 0x 0000 3000 – 0x 0000 3fff,

Should be mapped to physical page 0x20100 (= the upper 20 bits of PA 0x 2010 0000) Two different PTEs now refer to PA = 0x 2010 0000 PTE of VA 0x 2010 0000 (kernel), and PTE of VA 0x 0000 3000 (process)

The kernel can use both

OSE 2011– processes & address spaces (lec2)15

Code: memory allocator