153
Fun with LD_PRELOAD Kevin Pulo [email protected] Australian National University Supercomputing Facility, National Computational Infrastructure National Facility, Canberra, Australia 2009-01-23

Fun With Ld_preload

  • Upload
    kanneam

  • View
    104

  • Download
    5

Embed Size (px)

Citation preview

Page 1: Fun With Ld_preload

Fun with LD_PRELOAD

Kevin [email protected]

Australian National University Supercomputing Facility,National Computational Infrastructure National Facility,

Canberra, Australia

2009-01-23

Page 2: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edOverview

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 2 / 61

1. Intro: Dynamic linking and LD_PRELOAD

2. Review: Existing LD_PRELOAD hacks

3. Code: Writing LD_PRELOAD hacks

4. Advanced: xlibtrace and xmultiwin

Page 3: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edPart 1

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 3 / 61

1. Intro: Dynamic linking and LD_PRELOAD

2. Review: Existing LD_PRELOAD hacks

3. Code: Writing LD_PRELOAD hacks

4. Advanced: xlibtrace and xmultiwin

Page 4: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan ed$LD_PRELOAD?

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 4 / 61

• Ordinarily the dynamic linker loads shared libs inwhatever order it needs them

Page 5: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan ed$LD_PRELOAD?

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 4 / 61

• Ordinarily the dynamic linker loads shared libs inwhatever order it needs them

• $LD_PRELOAD is an environment variable containinga colon (or space) separated list of libraries thatthe dynamic linker loads before any others

Page 6: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan ed$LD_PRELOAD?

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 4 / 61

• Ordinarily the dynamic linker loads shared libs inwhatever order it needs them

• $LD_PRELOAD is an environment variable containinga colon (or space) separated list of libraries thatthe dynamic linker loads before any others

• Entries containing ’/’ are treated as filenames• Entries not containing ’/’ are searched for as usual

Page 7: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan ed$LD_PRELOAD?

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 4 / 61

• Ordinarily the dynamic linker loads shared libs inwhatever order it needs them

• $LD_PRELOAD is an environment variable containinga colon (or space) separated list of libraries thatthe dynamic linker loads before any others

• Entries containing ’/’ are treated as filenames• Entries not containing ’/’ are searched for as usual

• eg. LD_PRELOAD="libc.so.6"eg. LD_PRELOAD="/test/lib/libc.so.6"

Page 8: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan ed$LD_PRELOAD?

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 4 / 61

• Ordinarily the dynamic linker loads shared libs inwhatever order it needs them

• $LD_PRELOAD is an environment variable containinga colon (or space) separated list of libraries thatthe dynamic linker loads before any others

• Entries containing ’/’ are treated as filenames• Entries not containing ’/’ are searched for as usual

• eg. LD_PRELOAD="libc.so.6"eg. LD_PRELOAD="/test/lib/libc.so.6"

• man ld.so for full dynamic linker info

Page 9: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edOkay — so what?

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 5 / 61

• Preloading a library means that its functions will beused before others of the same name in later libraries

Page 10: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edOkay — so what?

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 5 / 61

• Preloading a library means that its functions will beused before others of the same name in later libraries

• Allows functions to be overridden/replaced/intercepted

Page 11: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edOkay — so what?

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 5 / 61

• Preloading a library means that its functions will beused before others of the same name in later libraries

• Allows functions to be overridden/replaced/intercepted

• Program behaviour can be modified “non-invasively”

Page 12: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edOkay — so what?

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 5 / 61

• Preloading a library means that its functions will beused before others of the same name in later libraries

• Allows functions to be overridden/replaced/intercepted

• Program behaviour can be modified “non-invasively”

ie. no recompile/relink necessary

Page 13: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edOkay — so what?

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 5 / 61

• Preloading a library means that its functions will beused before others of the same name in later libraries

• Allows functions to be overridden/replaced/intercepted

• Program behaviour can be modified “non-invasively”

ie. no recompile/relink necessary

Especially useful for closed-source programs

Page 14: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edOkay — so what?

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 5 / 61

• Preloading a library means that its functions will beused before others of the same name in later libraries

• Allows functions to be overridden/replaced/intercepted

• Program behaviour can be modified “non-invasively”

ie. no recompile/relink necessary

Especially useful for closed-source programs

And when the modifications don’t belong in theprogram or the library

Page 15: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edSystem-wide

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 6 / 61

• export LD_PRELOAD="foobar.so" in /etc/profile(unset LD_PRELOAD)

• /etc/ld.so.preload

(loaded before $LD_PRELOAD, non-overridable)

Page 16: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edSystem-wide

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 6 / 61

• export LD_PRELOAD="foobar.so" in /etc/profile(unset LD_PRELOAD)

• /etc/ld.so.preload

(loaded before $LD_PRELOAD, non-overridable)

• Affects all dynamically linked programs: be careful• Consider selective use:

LD_PRELOAD="foobar.so" someprogram (sh/bash)env LD_PRELOAD="foobar.so" someprogram (csh/any)

Page 17: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edIssues

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 7 / 61

• Affects dynamically linked programs only - not static

Page 18: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edIssues

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 7 / 61

• Affects dynamically linked programs only - not static• Also affects child processes

Page 19: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edIssues

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 7 / 61

• Affects dynamically linked programs only - not static• Also affects child processes• setuid/setgid programs: only libs in standard paths

that are also setuid/setgid will be preloaded

Page 20: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edIssues

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 7 / 61

• Affects dynamically linked programs only - not static• Also affects child processes• setuid/setgid programs: only libs in standard paths

that are also setuid/setgid will be preloaded• Can only override library functions, not system calls

(glibc wrappers are fair game)

Page 21: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edIssues

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 7 / 61

• Affects dynamically linked programs only - not static• Also affects child processes• setuid/setgid programs: only libs in standard paths

that are also setuid/setgid will be preloaded• Can only override library functions, not system calls

(glibc wrappers are fair game)

• Options must be passed via environment(could still use getopt() etc to parse an env var)

Page 22: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edPart 2

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 8 / 61

1. Intro: Dynamic linking and LD_PRELOAD

2. Review: Existing LD_PRELOAD hacks

3. Code: Writing LD_PRELOAD hacks

4. Advanced: xlibtrace and xmultiwin

Page 23: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edExisting LD_PRELOAD hacks

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 9 / 61

Vague broad categories:• Filesystem shenanigans• Network shenanigans• Debugging and testing• Annoyance reduction• Library tracing and logging• Graphical augmentation• Finessing/augmenting program behaviours

Page 24: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edFilesystem shenanigans

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 10 / 61

By intercepting libc functions that deal with thefilesystem, the preload library can modify how thefilesystem looks and behaves for programs.

Page 25: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edFilesystem shenanigans

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 10 / 61

By intercepting libc functions that deal with thefilesystem, the preload library can modify how thefilesystem looks and behaves for programs.• Anything that’s passed or returns filenames/paths• eg. open(), mkdir(), opendir(), stat(), etc

Page 26: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edFilesystem shenanigans

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 10 / 61

By intercepting libc functions that deal with thefilesystem, the preload library can modify how thefilesystem looks and behaves for programs.• Anything that’s passed or returns filenames/paths• eg. open(), mkdir(), opendir(), stat(), etc

• flcow (File Link Copy-On-Write) :http://www.xmailserver.org/flcow.htmlIntercepts open() and friends, breaks hard-links whenchanging files.

Page 27: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 11 / 61

$

Page 28: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 11 / 61

$ ls ­ld linux*

Page 29: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 11 / 61

$ ls ­ld linux*

drwxr­xr­x 21 4096 2009­01­17 15:37 linux­2.6.26.5

$

Page 30: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 11 / 61

$ ls ­ld linux*

drwxr­xr­x 21 4096 2009­01­17 15:37 linux­2.6.26.5

$ du ­sk linux*

Page 31: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 11 / 61

$ ls ­ld linux*

drwxr­xr­x 21 4096 2009­01­17 15:37 linux­2.6.26.5

$ du ­sk linux*

330592 linux­2.6.26.5

$

Page 32: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 11 / 61

$ ls ­ld linux*

drwxr­xr­x 21 4096 2009­01­17 15:37 linux­2.6.26.5

$ du ­sk linux*

330592 linux­2.6.26.5

$ cp ­al linux­2.6.26.5 linux­2.6.26.5­test

$

Page 33: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 11 / 61

$ ls ­ld linux*

drwxr­xr­x 21 4096 2009­01­17 15:37 linux­2.6.26.5

$ du ­sk linux*

330592 linux­2.6.26.5

$ cp ­al linux­2.6.26.5 linux­2.6.26.5­test

$ ls ­ld linux*

Page 34: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 11 / 61

$ ls ­ld linux*

drwxr­xr­x 21 4096 2009­01­17 15:37 linux­2.6.26.5

$ du ­sk linux*

330592 linux­2.6.26.5

$ cp ­al linux­2.6.26.5 linux­2.6.26.5­test

$ ls ­ld linux*

drwxr­xr­x 21 4096 2009­01­17 15:37 linux­2.6.26.5

drwxr­xr­x 21 4096 2009­01­17 15:38 linux­2.6.26.5­test

$

Page 35: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 11 / 61

$ ls ­ld linux*

drwxr­xr­x 21 4096 2009­01­17 15:37 linux­2.6.26.5

$ du ­sk linux*

330592 linux­2.6.26.5

$ cp ­al linux­2.6.26.5 linux­2.6.26.5­test

$ ls ­ld linux*

drwxr­xr­x 21 4096 2009­01­17 15:37 linux­2.6.26.5

drwxr­xr­x 21 4096 2009­01­17 15:38 linux­2.6.26.5­test

$ du ­sk linux*

Page 36: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 11 / 61

$ ls ­ld linux*

drwxr­xr­x 21 4096 2009­01­17 15:37 linux­2.6.26.5

$ du ­sk linux*

330592 linux­2.6.26.5

$ cp ­al linux­2.6.26.5 linux­2.6.26.5­test

$ ls ­ld linux*

drwxr­xr­x 21 4096 2009­01­17 15:37 linux­2.6.26.5

drwxr­xr­x 21 4096 2009­01­17 15:38 linux­2.6.26.5­test

$ du ­sk linux*

330592 linux­2.6.26.5

5816 linux­2.6.26.5­test

$

Page 37: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 12 / 61

$

Page 38: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 12 / 61

$ grep LOCALVERSION= linux*/.config

Page 39: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 12 / 61

$ grep LOCALVERSION= linux*/.config

linux­2.6.26.5­test/.config:CONFIG_LOCALVERSION=""

linux­2.6.26.5/.config:CONFIG_LOCALVERSION=""

$

Page 40: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 12 / 61

$ grep LOCALVERSION= linux*/.config

linux­2.6.26.5­test/.config:CONFIG_LOCALVERSION=""

linux­2.6.26.5/.config:CONFIG_LOCALVERSION=""

$ vim linux­2.6.26.5­test/.config

Page 41: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 12 / 61

$ grep LOCALVERSION= linux*/.config

linux­2.6.26.5­test/.config:CONFIG_LOCALVERSION=""

linux­2.6.26.5/.config:CONFIG_LOCALVERSION=""

$ vim linux­2.6.26.5­test/.config

$

Page 42: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 12 / 61

$ grep LOCALVERSION= linux*/.config

linux­2.6.26.5­test/.config:CONFIG_LOCALVERSION=""

linux­2.6.26.5/.config:CONFIG_LOCALVERSION=""

$ vim linux­2.6.26.5­test/.config

$ grep LOCALVERSION= linux*/.config

Page 43: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 12 / 61

$ grep LOCALVERSION= linux*/.config

linux­2.6.26.5­test/.config:CONFIG_LOCALVERSION=""

linux­2.6.26.5/.config:CONFIG_LOCALVERSION=""

$ vim linux­2.6.26.5­test/.config

$ grep LOCALVERSION= linux*/.config

linux­2.6.26.5­test/.config:CONFIG_LOCALVERSION="­test"

linux­2.6.26.5/.config:CONFIG_LOCALVERSION="­test"

$

Page 44: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 13 / 61

$ grep LOCALVERSION= linux*/.config

linux­2.6.26.5­test/.config:CONFIG_LOCALVERSION=""

linux­2.6.26.5/.config:CONFIG_LOCALVERSION=""

$

(was 5816)

Page 45: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 13 / 61

$ grep LOCALVERSION= linux*/.config

linux­2.6.26.5­test/.config:CONFIG_LOCALVERSION=""

linux­2.6.26.5/.config:CONFIG_LOCALVERSION=""

$ export LD_PRELOAD="libflcow.so:$LD_PRELOAD"

$

(was 5816)

Page 46: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 13 / 61

$ grep LOCALVERSION= linux*/.config

linux­2.6.26.5­test/.config:CONFIG_LOCALVERSION=""

linux­2.6.26.5/.config:CONFIG_LOCALVERSION=""

$ export LD_PRELOAD="libflcow.so:$LD_PRELOAD"

$ export FLCOW_PATH="$PWD"

$

(was 5816)

Page 47: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 13 / 61

$ grep LOCALVERSION= linux*/.config

linux­2.6.26.5­test/.config:CONFIG_LOCALVERSION=""

linux­2.6.26.5/.config:CONFIG_LOCALVERSION=""

$ export LD_PRELOAD="libflcow.so:$LD_PRELOAD"

$ export FLCOW_PATH="$PWD"

$ vim linux­2.6.26.5­test/.config

(was 5816)

Page 48: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 13 / 61

$ grep LOCALVERSION= linux*/.config

linux­2.6.26.5­test/.config:CONFIG_LOCALVERSION=""

linux­2.6.26.5/.config:CONFIG_LOCALVERSION=""

$ export LD_PRELOAD="libflcow.so:$LD_PRELOAD"

$ export FLCOW_PATH="$PWD"

$ vim linux­2.6.26.5­test/.config

$

(was 5816)

Page 49: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 13 / 61

$ grep LOCALVERSION= linux*/.config

linux­2.6.26.5­test/.config:CONFIG_LOCALVERSION=""

linux­2.6.26.5/.config:CONFIG_LOCALVERSION=""

$ export LD_PRELOAD="libflcow.so:$LD_PRELOAD"

$ export FLCOW_PATH="$PWD"

$ vim linux­2.6.26.5­test/.config

$ grep LOCALVERSION= linux*/.config

(was 5816)

Page 50: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 13 / 61

$ grep LOCALVERSION= linux*/.config

linux­2.6.26.5­test/.config:CONFIG_LOCALVERSION=""

linux­2.6.26.5/.config:CONFIG_LOCALVERSION=""

$ export LD_PRELOAD="libflcow.so:$LD_PRELOAD"

$ export FLCOW_PATH="$PWD"

$ vim linux­2.6.26.5­test/.config

$ grep LOCALVERSION= linux*/.config

linux­2.6.26.5­test/.config:CONFIG_LOCALVERSION="­test"

linux­2.6.26.5/.config:CONFIG_LOCALVERSION=""

$

(was 5816)

Page 51: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 13 / 61

$ grep LOCALVERSION= linux*/.config

linux­2.6.26.5­test/.config:CONFIG_LOCALVERSION=""

linux­2.6.26.5/.config:CONFIG_LOCALVERSION=""

$ export LD_PRELOAD="libflcow.so:$LD_PRELOAD"

$ export FLCOW_PATH="$PWD"

$ vim linux­2.6.26.5­test/.config

$ grep LOCALVERSION= linux*/.config

linux­2.6.26.5­test/.config:CONFIG_LOCALVERSION="­test"

linux­2.6.26.5/.config:CONFIG_LOCALVERSION=""

$ du ­sk linux*

(was 5816)

Page 52: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edflcow demo

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 13 / 61

$ grep LOCALVERSION= linux*/.config

linux­2.6.26.5­test/.config:CONFIG_LOCALVERSION=""

linux­2.6.26.5/.config:CONFIG_LOCALVERSION=""

$ export LD_PRELOAD="libflcow.so:$LD_PRELOAD"

$ export FLCOW_PATH="$PWD"

$ vim linux­2.6.26.5­test/.config

$ grep LOCALVERSION= linux*/.config

linux­2.6.26.5­test/.config:CONFIG_LOCALVERSION="­test"

linux­2.6.26.5/.config:CONFIG_LOCALVERSION=""

$ du ­sk linux*330592 linux­2.6.26.5

5856 linux­2.6.26.5­test (was 5816)

$

Page 53: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edOther fs-related stuff

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 14 / 61

• plasticfs : http://plasticfs.sourceforge.net/Intercepts everything in glibc that accepts/returnsfilenames, provides several wide-ranging “filters” thatcan be combined.

Page 54: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edOther fs-related stuff

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 14 / 61

• plasticfs : http://plasticfs.sourceforge.net/Intercepts everything in glibc that accepts/returnsfilenames, provides several wide-ranging “filters” thatcan be combined.

• installwatch : http://www.asic-linux.com.mx/~izto/checkinstall/installwatch.htmlIntercepts many functions to keep track of filescreated/modified by package installations

Page 55: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edNetwork shenanigans

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 15 / 61

By intercepting libc functions that deal with the network,the preload library can modify how the network looksand behaves for programs.• eg. socket(), connect(), bind(), listen(),

gethostbyname(), etc

Page 56: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edNetwork shenanigans

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 15 / 61

By intercepting libc functions that deal with the network,the preload library can modify how the network looksand behaves for programs.• eg. socket(), connect(), bind(), listen(),

gethostbyname(), etc

• libshape : http://freshmeat.net/projects/libshape/Limits the download rate of programs (ie. user-spacenetwork traffic rate-shaping).

• netjail : http://netjail.sourceforge.net/Controls where and how programs may connect tothe network. Useful for dealing with spyware, etc.

Page 57: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edDebugging and testing

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 16 / 61

By deliberately causing functions to fail, it’s possible totest if programs are able to deal with such situations.

Page 58: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edDebugging and testing

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 16 / 61

By deliberately causing functions to fail, it’s possible totest if programs are able to deal with such situations.• failmalloc : http://www.nongnu.org/failmalloc/

Intercepts malloc() and friends, induces them to fail(return NULL) more often.

• libeatmydata :http://www.flamingspork.com/projects/libeatmydata/Intercepts fsync() and friends, disables them.

Page 59: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edDebugging and testing

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 17 / 61

Critical functions can be intercepted to watch forpotential bugs, security holes, etc.• electricfence :

http://perens.com/works/software/ElectricFence/• segv_handler :

http://junkcode.samba.org/#segv_handler

Page 60: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edelectricfence

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 18 / 61

Intercepts malloc() and friends, surrounds allocatedmemory with protected memory to catch bufferover/under-runs.

Page 61: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edelectricfence

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 18 / 61

Intercepts malloc() and friends, surrounds allocatedmemory with protected memory to catch bufferover/under-runs.

Page 62: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edsegv_handler

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 19 / 61

Signal handler for segfaults that produces a backtrace.• Installs a signal handler for SIGSEGV and SIGBUS• Signal handler basically just runs gdb on current

process to dump backtrace into a logfile• (program is still free to use its own handler)

(lib could also intercept signal() to always force the backtrace)

• No overhead if system-wide in /etc/ld.so.preload• Tiny and simple: 34 lines

Page 63: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edGraphical augmentation

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 20 / 61

Since most graphical display is also done via libraries, itis also possible (though usually more complex) tointercept these graphical libraries to augment thedisplay in various ways.

Page 64: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edGraphical augmentation

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 20 / 61

Since most graphical display is also done via libraries, itis also possible (though usually more complex) tointercept these graphical libraries to augment thedisplay in various ways.• libglfps :

http://users.dakotacom.net/~donut/programs/libglfps.htmlAdds a framerate display to OpenGL programs.

Page 65: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edPart 3

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 21 / 61

1. Intro: Dynamic linking and LD_PRELOAD

2. Review: Existing LD_PRELOAD hacks

3. Code: Writing LD_PRELOAD hacks

4. Advanced: xlibtrace and xmultiwin

Page 66: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan eddlfcn

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 22 / 61

• #include <dlfcn.h> (Dynamic Linker FunCtioNs)

• Two main functions:void* dlopen(const char* filename, int flag)

void* dlsym(void* handle, const char* symbol)

• Link against libdl.so, ie. ­ldl

Page 67: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edMotivating example

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 23 / 61

• Suppose we have some closed-source multi-threadedapplication that we are stuck with

Page 68: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edMotivating example

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 23 / 61

• Suppose we have some closed-source multi-threadedapplication that we are stuck with

• This app is spawning too many threads (one for eachCPU in the system), AND has no way to config it

Page 69: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edMotivating example

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 23 / 61

• Suppose we have some closed-source multi-threadedapplication that we are stuck with

• This app is spawning too many threads (one for eachCPU in the system), AND has no way to config it

• Digging around with strace, ltrace, gdb,/usr/include/bits, etc we find that it’s callingsysconf(_SC_NPROCESSORS_CONF)

Page 70: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edMotivating example

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 23 / 61

• Suppose we have some closed-source multi-threadedapplication that we are stuck with

• This app is spawning too many threads (one for eachCPU in the system), AND has no way to config it

• Digging around with strace, ltrace, gdb,/usr/include/bits, etc we find that it’s callingsysconf(_SC_NPROCESSORS_CONF)

• So we want a preload lib which overrides the returnvalue of sysconf() for this case — “behaviourfinessing”

Page 71: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edlibsysconfcpus

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 24 / 61

http://www.kev.pulo.com.au/libsysconfcpus/

We will construct this library from the ground up, as asimple example of how to create a preload library

Page 72: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edlibsysconfcpus

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 24 / 61

http://www.kev.pulo.com.au/libsysconfcpus/

We will construct this library from the ground up, as asimple example of how to create a preload library• Version 0.1: Empty base project

Page 73: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edlibsysconfcpus

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 24 / 61

http://www.kev.pulo.com.au/libsysconfcpus/

We will construct this library from the ground up, as asimple example of how to create a preload library• Version 0.1: Empty base project• Version 0.2: Library debug and init

Page 74: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edlibsysconfcpus

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 24 / 61

http://www.kev.pulo.com.au/libsysconfcpus/

We will construct this library from the ground up, as asimple example of how to create a preload library• Version 0.1: Empty base project• Version 0.2: Library debug and init• Version 0.3: Intercept sysconf()

Page 75: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edlibsysconfcpus

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 24 / 61

http://www.kev.pulo.com.au/libsysconfcpus/

We will construct this library from the ground up, as asimple example of how to create a preload library• Version 0.1: Empty base project• Version 0.2: Library debug and init• Version 0.3: Intercept sysconf()• Version 0.4: Adjust sysconf() behaviour

Page 76: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edlibsysconfcpus

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 24 / 61

http://www.kev.pulo.com.au/libsysconfcpus/

We will construct this library from the ground up, as asimple example of how to create a preload library• Version 0.1: Empty base project• Version 0.2: Library debug and init• Version 0.3: Intercept sysconf()• Version 0.4: Adjust sysconf() behaviour• Version 0.5: Extra features

Page 77: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: configure.ac

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 25 / 61

$ cat configure.ac

AC_PREREQ(2.5)

AC_INIT(libsysconfcpus, [0.1], [[email protected]])

AC_CONFIG_SRCDIR([src/libsysconfcpus.c])

AM_INIT_AUTOMAKE

AM_CONFIG_HEADER([config.h])

AC_PROG_CC

AC_PROG_INSTALL

AC_PROG_LN_S

...

Page 78: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: configure.ac

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 25 / 61

$ cat configure.ac

GenericAC/AMsetup

AC_PREREQ(2.5)

AC_INIT(libsysconfcpus, [0.1], [[email protected]])

AC_CONFIG_SRCDIR([src/libsysconfcpus.c])

AM_INIT_AUTOMAKE

AM_CONFIG_HEADER([config.h])

AC_PROG_CC

AC_PROG_INSTALL

AC_PROG_LN_S

...

Page 79: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: configure.ac

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 25 / 61

$ cat configure.ac

AC_PREREQ(2.5)

AC_INIT(libsysconfcpus, [0.1], [[email protected]])

AC_CONFIG_SRCDIR([src/libsysconfcpus.c])

AM_INIT_AUTOMAKE

AM_CONFIG_HEADER([config.h])

Check for programsAC_PROG_CC

AC_PROG_INSTALL

AC_PROG_LN_S

...

Page 80: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: configure.ac

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 26 / 61

...

AC_ENABLE_SHARED

AC_DISABLE_STATIC

AM_PROG_LIBTOOL

AC_HEADER_STDC

AC_C_CONST

AC_CONFIG_FILES([Makefile src/Makefile])

AC_OUTPUT

$

Page 81: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: configure.ac

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 26 / 61

...

libtool shared libsAC_ENABLE_SHARED

AC_DISABLE_STATIC

AM_PROG_LIBTOOL

AC_HEADER_STDC

AC_C_CONST

AC_CONFIG_FILES([Makefile src/Makefile])

AC_OUTPUT

$

Page 82: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: configure.ac

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 26 / 61

...

AC_ENABLE_SHARED

AC_DISABLE_STATIC

AM_PROG_LIBTOOL

Check for standard stuffAC_HEADER_STDC

AC_C_CONST

AC_CONFIG_FILES([Makefile src/Makefile])

AC_OUTPUT

$

Page 83: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: configure.ac

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 26 / 61

...

AC_ENABLE_SHARED

AC_DISABLE_STATIC

AM_PROG_LIBTOOL

AC_HEADER_STDC

AC_C_CONST

GenericAC/AMconclusion

AC_CONFIG_FILES([Makefile src/Makefile])

AC_OUTPUT

$

Page 84: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: Makefile.am

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 27 / 61

$ cat Makefile.am

SUBDIRS = src

$ cat src/Makefile.am

AUTOMAKE_OPTIONS = 1.4 foreign

CFLAGS += ­Wall

lib_LTLIBRARIES = libsysconfcpus.la

libsysconfcpus_la_SOURCES = libsysconfcpus.c

libsysconfcpus_la_CFLAGS = ­O1

libsysconfcpus_la_LIBADD = ­ldl

#dist_man_MANS = libsysconfcpus.1

$ touch src/libsysconfcpus.c

Page 85: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: Makefile.am

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 27 / 61

All source in src/$ cat Makefile.am

SUBDIRS = src

$ cat src/Makefile.am

AUTOMAKE_OPTIONS = 1.4 foreign

CFLAGS += ­Wall

lib_LTLIBRARIES = libsysconfcpus.la

libsysconfcpus_la_SOURCES = libsysconfcpus.c

libsysconfcpus_la_CFLAGS = ­O1

libsysconfcpus_la_LIBADD = ­ldl

#dist_man_MANS = libsysconfcpus.1

$ touch src/libsysconfcpus.c

Page 86: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: Makefile.am

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 27 / 61

$ cat Makefile.am

SUBDIRS = src

Generic AM setup$ cat src/Makefile.am

AUTOMAKE_OPTIONS = 1.4 foreign

CFLAGS += ­Wall

lib_LTLIBRARIES = libsysconfcpus.la

libsysconfcpus_la_SOURCES = libsysconfcpus.c

libsysconfcpus_la_CFLAGS = ­O1

libsysconfcpus_la_LIBADD = ­ldl

#dist_man_MANS = libsysconfcpus.1

$ touch src/libsysconfcpus.c

Page 87: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: Makefile.am

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 27 / 61

$ cat Makefile.am

SUBDIRS = src

$ cat src/Makefile.am

AUTOMAKE_OPTIONS = 1.4 foreign

CFLAGS += ­Wall

Setup forlibsysconf­

cpus.so

lib_LTLIBRARIES = libsysconfcpus.la

libsysconfcpus_la_SOURCES = libsysconfcpus.c

libsysconfcpus_la_CFLAGS = ­O1

libsysconfcpus_la_LIBADD = ­ldl

#dist_man_MANS = libsysconfcpus.1

$ touch src/libsysconfcpus.c

Page 88: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: Makefile.am

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 27 / 61

$ cat Makefile.am

SUBDIRS = src

$ cat src/Makefile.am

AUTOMAKE_OPTIONS = 1.4 foreign

CFLAGS += ­Wall

lib_LTLIBRARIES = libsysconfcpus.la

libsysconfcpus_la_SOURCES = libsysconfcpus.c

libsysconfcpus_la_CFLAGS = ­O1

libsysconfcpus_la_LIBADD = ­ldl

If you can bebothered

#dist_man_MANS = libsysconfcpus.1

$ touch src/libsysconfcpus.c

Page 89: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: Makefile.am

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 27 / 61

$ cat Makefile.am

SUBDIRS = src

$ cat src/Makefile.am

AUTOMAKE_OPTIONS = 1.4 foreign

CFLAGS += ­Wall

lib_LTLIBRARIES = libsysconfcpus.la

libsysconfcpus_la_SOURCES = libsysconfcpus.c

libsysconfcpus_la_CFLAGS = ­O1

libsysconfcpus_la_LIBADD = ­ldl

#dist_man_MANS = libsysconfcpus.1

Empty source file for now$ touch src/libsysconfcpus.c

Page 90: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: Generate configure

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 28 / 61

$

Page 91: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: Generate configure

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 28 / 61

$ touch NEWS README AUTHORS ChangeLog

Page 92: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: Generate configure

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 28 / 61

$ touch NEWS README AUTHORS ChangeLog

$

Page 93: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: Generate configure

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 28 / 61

$ touch NEWS README AUTHORS ChangeLog

$ autoreconf ­i

Page 94: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: Generate configure

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 28 / 61

$ touch NEWS README AUTHORS ChangeLog

$ autoreconf ­i

configure.ac: installing ‘./install­sh’

configure.ac: installing ‘./missing’

src/Makefile.am: installing ‘./compile’

src/Makefile.am: installing ‘./depcomp’

Makefile.am: installing ‘./INSTALL’

Makefile.am: installing ‘./COPYING’

$

Page 95: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: configure

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 29 / 61

$ ./configure ­­prefix=/tmp/sysconf

checking for a BSD­compatible install... /usr/bin/ginstall ­c

checking whether build environment is sane... yes

checking for gawk... gawk

checking whether make sets $(MAKE)... yes

checking for gcc... gcc

...

checking for an ANSI C­conforming const... yes

configure: creating ./config.status

config.status: creating Makefile

config.status: creating src/Makefile

config.status: creating config.h

config.status: executing depfiles commands

$

Page 96: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: make

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 30 / 61

$ make

...

/bin/sh ../libtool ­­tag=CC ­­mode=link gcc ­g ­O2 ­→

→Wall ­o libsysconfcpus.la ­rpath /home/kev/→

→libsysconfcpus/install/lib libsysconfcpus_la­→

→libsysconfcpus.lo ­ldl

gcc ­shared .libs/libsysconfcpus_la­libsysconfcpus.o →

→­ldl ­Wl,­soname ­Wl,libsysconfcpus.so.0 ­o .→

→libs/libsysconfcpus.so.0.0.0

...

$

Page 97: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: make install

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 31 / 61

$

Page 98: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: make install

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 31 / 61

$ make install

Page 99: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: make install

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 31 / 61

$ make install

...

$

Page 100: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: make install

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 31 / 61

$ make install

...

$ ls ­l /tmp/sysconf/

Page 101: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: make install

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 31 / 61

$ make install

...

$ ls ­l /tmp/sysconf/

drwxr­xr­x 2 4096 2008­12­29 08:39 lib

$

Page 102: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: make install

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 31 / 61

$ make install

...

$ ls ­l /tmp/sysconf/

drwxr­xr­x 2 4096 2008­12­29 08:39 lib

$ ls ­l /tmp/sysconf/lib/

Page 103: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: make install

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 31 / 61

$ make install

...

$ ls ­l /tmp/sysconf/

drwxr­xr­x 2 4096 2008­12­29 08:39 lib

$ ls ­l /tmp/sysconf/lib/

­rwxr­xr­x 1 852 2008­12­29 08:39 libsysconfcpus.la

lrwxrwxrwx 1 23 2008­12­29 08:39 libsysconfcpus.so ­>→

→ libsysconfcpus.so.0.0.0

lrwxrwxrwx 1 23 2008­12­29 08:39 libsysconfcpus.so.0 →

→­> libsysconfcpus.so.0.0.0

­rwxr­xr­x 1 6770 2008­12­29 08:39 libsysconfcpus.so→

→.0.0.0

$

Page 104: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.1: make dist

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 32 / 61

$ make dist

...

$ ls ­la *.tar.gz

­rw­r­­r­­ 1 313430 2008­12­29 09:26 libsysconfcpus→

→­0.1.tar.gz

$

Page 105: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.2: debug

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 33 / 61

src/libsysconfcpus.c#include <stdio.h>

#include <stdarg.h>

static int do_debug() {

return getenv("LIBSYSCONFCPUS_DEBUG");

}

static void dprintf(char *fmt, ...) {

va_list ap;

if (do_debug()) {

va_start(ap, fmt);

vfprintf(stderr, fmt, ap);

va_end(ap);

fflush(stderr);

}

}

Page 106: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.2: init/fini

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 34 / 61

src/libsysconfcpus.cstatic void _libsysconfcpus_init(void)

__attribute__ ((constructor));

static void _libsysconfcpus_fini(void)

__attribute__ ((destructor));

static void _libsysconfcpus_init(void) {

dprintf("libsysconfcpus: starting up\n");

}

static void _libsysconfcpus_fini(void) {

dprintf("libsysconfcpus: shutting down\n");

}

Page 107: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.2: support shell scripts

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 35 / 61

src/sysconfcpus.in#!/bin/sh

LD_PRELOAD="@libdir@/libsysconfcpus.so:$LD_PRELOAD" \

exec "$@"

Page 108: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.2: support shell scripts

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 35 / 61

src/sysconfcpus.in#!/bin/sh

LD_PRELOAD="@libdir@/libsysconfcpus.so:$LD_PRELOAD" \

exec "$@"

src/sysconfcpus-debug.in#!/bin/sh

LD_PRELOAD="@libdir@/libsysconfcpus.so:$LD_PRELOAD" \

LIBSYSCONFCPUS_DEBUG=y \

exec "$@"

Page 109: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.2: support shell scripts

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 36 / 61

src/Makefile.amedit = $(SED) \

­e ’s,@bindir\@,$(bindir),g’ \

­e ’s,@mandir\@,$(mandir),g’ \

­e ’s,@libdir\@,$(libdir),g’ \

­e ’s,@datadir\@,$(datadir),g’ \

­e ’s,@localstatedir\@,$(localstatedir),g’

sysconfcpus:: Makefile $(srcdir)/sysconfcpus.in

rm ­f sysconfcpus sysconfcpus.tmp && \

$(edit) $(srcdir)/sysconfcpus.in >sysconfcpus.tmp \

&& mv sysconfcpus.tmp sysconfcpus

Page 110: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.2: support shell scripts

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 37 / 61

src/Makefile.amnodist_bin_SCRIPTS = sysconfcpus

nodist_bin_SCRIPTS += sysconfcpus­debug

dist_noinst_DATA = sysconfcpus.in

dist_noinst_DATA += sysconfcpus­debug.in

DISTCLEANFILES = sysconfcpus

DISTCLEANFILES += sysconfcpus­debug

Page 111: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.2: rebuild

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 38 / 61

$

Page 112: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.2: rebuild

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 38 / 61

$ make clean

Page 113: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.2: rebuild

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 38 / 61

$ make clean

...

$

Page 114: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.2: rebuild

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 38 / 61

$ make clean

...

$ autoreconf ­m

Page 115: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.2: rebuild

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 38 / 61

$ make clean

...

$ autoreconf ­m

...

$

Page 116: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.2: rebuild

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 38 / 61

$ make clean

...

$ autoreconf ­m

...

$ make install

Page 117: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.2: rebuild

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 38 / 61

$ make clean

...

$ autoreconf ­m

...

$ make install

...

$

Page 118: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.2: rebuild

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 38 / 61

$ make clean

...

$ autoreconf ­m

...

$ make install

...

$ ls ­l /tmp/sysconf/

Page 119: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.2: rebuild

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 38 / 61

$ make clean

...

$ autoreconf ­m

...

$ make install

...

$ ls ­l /tmp/sysconf/

drwxr­xr­x 2 4096 2008­12­30 23:07 bin

drwxr­xr­x 2 4096 2008­12­30 23:07 lib

$

Page 120: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.2: rebuild

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 38 / 61

$ make clean

...

$ autoreconf ­m

...

$ make install

...

$ ls ­l /tmp/sysconf/

drwxr­xr­x 2 4096 2008­12­30 23:07 bin

drwxr­xr­x 2 4096 2008­12­30 23:07 lib

$ ls ­l /tmp/sysconf/bin/

Page 121: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.2: rebuild

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 38 / 61

$ make clean

...

$ autoreconf ­m

...

$ make install

...

$ ls ­l /tmp/sysconf/

drwxr­xr­x 2 4096 2008­12­30 23:07 bin

drwxr­xr­x 2 4096 2008­12­30 23:07 lib

$ ls ­l /tmp/sysconf/bin/

­rwxr­xr­x 1 102 2008­12­30 23:07 sysconfcpus

­rwxr­xr­x 1 125 2008­12­30 23:07 sysconfcpus­debug

$

Page 122: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.3: no error checking or debug

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 39 / 61

#include <dlfcn.h>

long sysconf(int name) {

static void *libc_handle = NULL;

static long (*underlying)(int name);

long retval;

if (!libc_handle) {

libc_handle = dlopen("libc.so.6", RTLD_LAZY);

underlying = dlsym(libc_handle, "sysconf");

}

retval = (*underlying)(name);

return retval;

}

Page 123: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.3: full code - part 1

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 40 / 61

long sysconf(int name) {

static void *libc_handle = NULL;

static long (*underlying)(int name);

long retval;

const char *err;

dprintf("libsysconfcpus: sysconf(%d): entered\n", name);

if (!libc_handle) {

libc_handle = dlopen("libc.so.6", RTLD_LAZY);

dprintf("libsysconfcpus: sysconf: libc_handle = 0x%x\n",

libc_handle);

if (!libc_handle) {

fprintf(stderr, "libsysconfcpus: Error: Unable to find "

"libc.so: %s\n", dlerror());

exit(1);

}

...

Page 124: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.3: full code - part 2

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 41 / 61

dlerror();

underlying = dlsym(libc_handle, "sysconf");

dprintf("libsysconfcpus: sysconf: underlying = 0x%x\n",

underlying);

err = dlerror();

if (err) {

dprintf("libsysconfcpus: sysconf: err = \"%s\"\n", err);

}

if (!underlying || err) {

fprintf(stderr, "libsysconfcpus: Error: Unable to find the "

"underlying sysconf(): %s\n", dlerror());

exit(1);

}

}

dprintf("libsysconfcpus: about to call underlying sysconf()\n");

retval = (*underlying)(name);

dprintf("libsysconfcpus: sysconf(%d) = %ld\n", name, retval);

return retval;

}

Page 125: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.3: test

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 42 / 61

$

Page 126: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.3: test

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 42 / 61

$ rudeapp

Page 127: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.3: test

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 42 / 61

$ rudeapp

Rude application v1.0

Spawning 8 threads

^C

$

Page 128: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.3: test

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 42 / 61

$ rudeapp

Rude application v1.0

Spawning 8 threads

^C

$ /tmp/sysconf/bin/sysconfcpus­debug rudeapp

Page 129: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.3: test

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 42 / 61

$ rudeapp

Rude application v1.0

Spawning 8 threads

^C

$ /tmp/sysconf/bin/sysconfcpus­debug rudeapp

libsysconfcpus: starting up

Rude application v1.0

libsysconfcpus: sysconf(83): entered

libsysconfcpus: sysconf: libc_handle = 0x1738eb78

libsysconfcpus: sysconf: underlying = 0x16cbc260

libsysconfcpus: about to call underlying sysconf()

libsysconfcpus: sysconf(83) = 8

Spawning 8 threads

^C

$

Page 130: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edVersion 0.4: Adjust sysconf()

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 43 / 61

long sysconf(int name) {

static void *libc_handle = NULL;

static long (*underlying)(int name);

long retval;

...

/* Opportunity to modify parameters */

retval = (*underlying)(name);

/* Opportunity to modify return value */

return retval;

}

Page 131: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.4: override retval

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 44 / 61

long v; char *env, *endptr;

...

if (name == _SC_NPROCESSORS_CONF ||

name == _SC_NPROCESSORS_ONLN) {

env = getenv("LIBSYSCONFCPUS");

if (env) {

v = strtol(env, &endptr, 10);

if (endptr == env)

dprintf("libsysconfcpus: Warning: $%s does not "

"contain a number\n", libsysconfcpus_envvar);

else

retval = v;

}

}

dprintf("libsysconfcpus: sysconf(%d)=%ld\n",name,retval);

Page 132: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.4: test

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 45 / 61

$

Page 133: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.4: test

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 45 / 61

$ LIBSYSCONFCPUS=2 /tmp/sysconf/bin/sysconfcpus­debug →

→rudeapp

Page 134: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edv0.4: test

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 45 / 61

$ LIBSYSCONFCPUS=2 /tmp/sysconf/bin/sysconfcpus­debug →

→rudeapp

libsysconfcpus: starting up

Rude application v1.0

libsysconfcpus: sysconf(83): entered

libsysconfcpus: sysconf: libc_handle = 0x1738eb78

libsysconfcpus: sysconf: underlying = 0x16cbc260

libsysconfcpus: about to call underlying sysconf()

libsysconfcpus: underlying sysconf(83) = 8

libsysconfcpus: sysconf(83) = 2

Spawning 2 threads

^C

$

Page 135: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edVersion 0.5: Extra features

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 46 / 61

• Use $LIBSYSCONFCPUS_CONF and $LIBSYSCONFCPUS_ONLN

with $LIBSYSCONFCPUS as a fallback• Make the shell scripts take command line arguments$ ../install/bin/sysconfcpus

sysconfcpus version 0.5

Usage:

sysconfcpus [options] <command> <args> ...

Valid options are:

­v, ­­version Display the sysconfcpus version

­h, ­­help Display this help

­d, ­­debug Display debug info

­n, ­­num <n> Number of processors (both conf and onln)

­c, ­­conf <n> Number of processors (conf(igured))

­o, ­­onln <n> Number of processors (online)

$

Page 136: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edRTLD_NEXT

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 47 / 61

• GNU extension: special library pseudo-handle• Find symbol in ‘next’ library after current

#define __USE_GNU // or #define _GNU_SOURCE

#include <dlfcn.h>

...

#if defined(RTLD_NEXT)

libc_handle = RTLD_NEXT;

#else

libc_handle = dlopen("libc.so.6", RTLD_LAZY);

#endif

underlying = dlsym(libc_handle, "sysconf");

Page 137: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edPart 4

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 48 / 61

1. Intro: Dynamic linking and LD_PRELOAD

2. Review: Existing LD_PRELOAD hacks

3. Code: Writing LD_PRELOAD hacks

4. Advanced: xlibtrace and xmultiwin

Page 138: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edxlibtrace

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 49 / 61

http://www.kev.pulo.com.au/xlibtrace/

The idea:• Intercept all functions in xlib• Make each output tracing info like strace/ltrace

Page 139: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edxlibtrace

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 49 / 61

http://www.kev.pulo.com.au/xlibtrace/

The idea:• Intercept all functions in xlib• Make each output tracing info like strace/ltrace

Why not just use ltrace?• Can verbosely interpret and output library-specific

data structures• Basis for wide-intercepting preload libs• Portable - doesn’t require per-architecture changes

Page 140: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edxlibtrace output

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 50 / 61

[32095] (0): int XNextEvent(Display * display = (0x60a3b0) (→

→opaque), XEvent * event_return = (0x7fffbd645870) { int→

→ type = ReparentNotify, unsigned long serial = 35, Bool→

→ send_event = False, Display * display = (0x60a3b0) (→

→opaque), Window event = 0x8400003, Window window = 0→

→x8400003, Window parent = 0x3203016, int x = 0, int y =→

→ 0, Bool override_redirect = False }) = <unfinished...>

[32095] (0): int XNextEvent(Display * display = (0x60a3b0) (→

→opaque), XEvent * event_return = (0x7fffbd645870) { int→

→ type = ConfigureNotify, unsigned long serial = 35, →

→Bool send_event = True, Display * display = (0x60a3b0) →

→(opaque), Window event = 0x8400003, Window window = 0→

→x8400003, int x = 2853, int y = 24, int width = 100, →

→int height = 100, int border_width = 0, Window above = →

→0x3203015, Bool override_redirect = False }) = 0

Page 141: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edxlibtrace skeleton

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 51 / 61

int XClearWindow(Display* display, Window w) {

static int (*underlying)(Display* display, Window w);

int retval;

if (!underlying) {

underlying = dlsym(RLTD_NEXT, "XClearWindow");

}

/* Print function name and parameters */

retval = (*underlying)(display, w);

/* Print return value */

return retval;

}

Page 142: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edxlibtrace intercept function

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 52 / 61

Heavy use of C preprocessor and awk to generate code/* int XClearWindow(Display * display, Window w) */

#define __TRACE_SAFERETTYPE_XClearWindow__ int

#define __TRACE_PROTOARGLIST_XClearWindow__ (Display* display,Window w)

#define __TRACE_ARGLIST_XClearWindow__ (display, w)

#define __TRACE_ADDITIONAL_LOCAL_VARS_XClearWindow__

#define __TRACE_ADDITIONAL_PRE_RUN_UNDERLYING_XClearWindow__

#define __TRACE_ADDITIONAL_POST_RUN_UNDERLYING_XClearWindow__

#define __TRACE_PRINTF_BODY_XClearWindow__ \

always_reprint = 1; \

__TRACE_PRINTF_ARG__(XClearWindow, Display_p, display) \

__PRINT_COMMA__(out) \

__TRACE_PRINTF_ARG__(XClearWindow, Window, w)

#define __TRACE_RUN_UNDERLYING_EPILOGUE_XClearWindow__ \

__TRACE_RUN_XFLUSH_XSYNC__(XClearWindow, Display_p, display)

__TRACE__(TYPED, FIXED, XClearWindow)

Page 143: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edxlibtrace print function

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 53 / 61

#define __REALTYPE__(t) __REALTYPE_##t##__

#define __REALTYPE_XPoint__ XPoint

#define __REALTYPE_XPoint_p__ XPoint *

#define __REALTYPE_XPoint_pp__ XPoint **

#define __TRACE_PRINT_TYPE_STRUCT_BODY_XPoint__(safetype) \

__TRACE_PRINT_STRUCT_MEMBER__(f,safetype, *value, short, x)\

__PRINT_COMMA__(f) \

__TRACE_PRINT_STRUCT_MEMBER__(f,safetype, *value, short, y)

__TRACE_PRINT_TYPE_STRUCT__(XPoint)

Page 144: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edxlibtrace print function

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 54 / 61

void print_type_XPoint_(FILE *f, const char *funcname, const →

→char *argname, const char *type, XPoint const * value)

{

fprintf(f, "{ ");

print_arg_name(f, "XPoint", "x", "short");

print_type_short_(f, "XPoint", "x", "short", (*value).x);

fprintf(f, ", ");

print_arg_name(f, "XPoint", "y", "short");

print_type_short_(f, "XPoint", "y", "short", (*value).y);

fprintf(f, " }");

}

Page 145: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edxlibtrace print function

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 55 / 61

void print_type_XPoint_p_(FILE *f, const char *funcname, const→

→ char *argname, const char *type, XPoint const * value)

{

if (value) {

fprintf(f, "(%p) ", value);

print_type_XPoint_(f, funcname, argname, type, value);

} else {

fprintf(f, "%p", value);

}

}

Same for XPoint** and XPoint***

Page 146: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edxmultiwin: multi-head display

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 56 / 61

Page 147: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edxmultiwin: multi-head display

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 57 / 61

Page 148: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edThe problem

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 58 / 61

I want some windows shown on both monitors at thesame time.

Page 149: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edThe problem

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 58 / 61

I want some windows shown on both monitors at thesame time.

Some simple ones I just run 2 instances of:• xload, pload, xapmload, xcal, xclock, xbiff, etc

Page 150: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edThe problem

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 58 / 61

I want some windows shown on both monitors at thesame time.

Some simple ones I just run 2 instances of:• xload, pload, xapmload, xcal, xclock, xbiff, etc

But some I can’t:• x11-ssh-askpass• xmms• xscreensaver dialogs• wmwork

Page 151: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edxmultiwin

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 59 / 61

http://www.kev.pulo.com.au/xmultiwin/

The idea:• Intercept all window-related functions in xlib• In XCreateWindow() automatically “clone” the window• Then duplicate all xlib calls to the cloned windows

Page 152: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edxmultiwin — “Fun”

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 60 / 61

• Parallel window trees

• Event translation from cloned windows

• Excessive redraw heuristics

• X extensions

• Support for higher level libs, eg OpenGL, cairo, etc

• Many monitors (> 1 clone)

• Clone on different $DISPLAY

• Clone to Xvfb: screen(1) for X?

• varargs with LD_PRELOAD?(current hack: count args and have generated cases)

• Silent 32/64 bit LD_PRELOAD?(without hacking glibc directly?)

Page 153: Fun With Ld_preload

1. Intro 2. Review 3. Code 4. Advan edThank you

LCA 2009 — Fun with LD_PRELOAD — Kevin Pulo 61 / 61

Questions?

+ LATEX+ powerdot: http://www.ctan.org/tex-archive/help/Catalogue/entries/powerdot.html+ impressive: http://impressive.sourceforge.net/