C/C++ Linux System Programming

Session 15User-space System Programming

– session 5

Outline

● Virtual memory / Address space concepts● Memory Allocation● Memory Mapping● Shared Object Loading

Paging

● System memory is partitioned into pages● Pages stored in a multi-level hardware table ● Access to a page that does not have an entry

causes exception – Control relinquished to software

Process Address Space

● Segments:– Text – code, read and execute

– Data – heap

– Bss – uninitialized data

– Stack – grows downward

● Breakpoint● Just a representation (used to be accurate)

VM: Paging / address space interaction

● Each address space has its tables● On context switch, kernel switches out the

active table● Each segment has its pages (no breakpoint)● Page fault:

– Read/Exec:● Demand paging● Virtual memory: swap

– Writeable: Copy on Write

Heap Memory Allocation

● void *calloc(size_t nmemb, size_t size);● void *malloc(size_t size);● void free(void *ptr);● void *realloc(void *ptr, size_t size);● Process -> glibc -> kernel● int posix_memalign(void **memptr, size_t

alignment, size_t size);● Memory fragmentation

Off the Stack

● void *alloca(size_t size);● Fast● Limited in size

– int getrlimit(int resource, struct rlimit *rlim);

– int setrlimit(int resource, const struct rlimit *rlim);

– RLIMIT_STACK (SIGSEGV when exceeded)

● No fragmentation● Don't do it in a function call

Kernel

● System calls– int brk(void *end_data_segment);

– void *sbrk(intptr_t increment);

● Buddy allocator– Buffers, size in powers of 2

– Go for smallest fit

– Two empty blocks can be used as a single of next spot

Glibc

● On smaller sizes, Object Stack allocator– Grow and shrink (like a stack)

– When whole stack is freed, it can be released

● On large allocations, anonymous memory maps– No fragmentation / good for resize

– Page size granularity / always going to kernel

More Kernel Tweaks

● Huge pages

– On Intel, regular page 4k, huge page 4M

– Kernel: CONFIG_HUGETLB_FS=y, (CONFIG_HUGETLB_PAGE=y)

– /proc/sys/vm/nr_hugepages (# of pages)

– mount -t hugetlbfs none /somedir

– Open /somedir/somfile and mmap it ANONYMOUS

● Overcommitment

– /proc/sys/vm/{overcommit_memory,overcommit_ratio}

– OOM killer

Locking Memory

● Preventing swapping on a page– int mlock(const void *addr, size_t len);

– int munlock(const void *addr, size_t len);

– int mlockall(int flags);

– int munlockall(void);

● Why?– Speed

– Security

Malloc Manipulation

● int mallopt (int param, int value)– M_TRIM_THRESHOLD

– M_TOP_PAD

– M_MMAP_THRESHOLD

– M_MMAP_MAX

● struct mallinfo mallinfo (void)– Stats on memory usage

Malloc Tracing and Checks

● void mtrace () / void muntrace ()● Checking

– int mcheck (void (*abortfn) (enum mcheck status status))

– enum mcheck_status mprobe (void *pointer)● MCHECK_OK / HEAD / TAIL / FREE

● Hooks– __malloc_hook / realloc_hook / __free_hook

– Not on SUID/SGID programs

Glibc Object Stack Manipulation

● int obstack_init (struct obstack *obstack_ptr)

● void * obstack_alloc (struct obstack *obstack_ptr, int size)

● void * obstack_copy (struct obstack *obstack_ptr, void *address, int size)

● void obstack_free (struct obstack *obstack_ptr, void *object)

– Frees all the ones above it in stack

● void obstack_grow (struct obstack *obstack_ptr, void *data, int size)

● void * obstack_finish (struct obstack *obstack_ptr)

More mmaps - Shared Object Loading

● ELF Format– Interpreter ld

– Symbols

– GOT / PLT

● Mmap – Copy on write (PRIVATE)

● Ld – Map the loaded object

– Write out reloc – based adjustments to address space