Privacy Protected ELF for

Private Computing on Public Platforms

Thomas Morris and V.S.S Nair

SMU HACNet Lab

Southern Methodist University

Dallas, TX, USA

{tmorris,nair}@engr.smu.edu

Abstract

Private Computing on Public Platforms (PCPP) is a

new technology designed to enable secure and private

execution of applications on remote potentially hostile

public platforms. PCPP uses a host assessment to validate

a host’s hardware and software configuration and then

uses 4 active security building blocks which together

allow an application to remain unaltered, unmonitored,

and unrecorded before, during, and after execution on the

public platform. Privacy Protected ELF (PPELF) is the

building block used by PCPP to protect application

executable code while it is stored on the public platform

and to provide a secure binary load mechanism. PPELF

is an encrypted binary executable format and

methodology which uses just in time decryption during

the executable load procedure. In this paper we describe

the PPELF file format, our new PPELF binary format

loader, and our updates to the GLIBC ELF interpreter to

support PPELF. We also provide experimental results

detailing the initial load-time and run-time penalty

associated with the use of PPELF.

1. Introduction

Distributed computing technologies which enable the

use of idle processors by remote users are abundant.

SETI@Home [13] is a popular distributed computing

application run by the University of California at

Berkeley which uses remote computers to download

and process data in the search for extra terrestrial

intelligence. The Globus Toolkit [12] is an open

source toolkit which has been used to build many

working grids which are collections of computers

linked to allow the sharing of computing resources

across locations. Although there have been many

successes in this area there has been a lack of large-

scale commercial acceptance. We believe that this

trend stems from a need for better application security

and privacy while applications are stored and executed

on remote platforms. Private Computing on Public

Platforms (PCPP) [4] [5] [6] was developed as a

solution to this problem. PCPP enables the secure and

private use of remote platforms by first performing a

host assessment to validate the hardware and software

configuration of the system and then using 4 active

security building blocks which assure that applications

executed on the remote platforms remain unaltered,

unmonitored, and unrecorded before, during, and after

execution.

Privacy Protected ELF (PPELF) is one of the 4

PCPP active security building blocks. PPELF was

designed to protect executable programs while they are

stored on a public platform. PPELF is an encrypted

extension of the ELF [1] binary format and includes a

methodology for securely loading the encrypted

program into memory. Rather, than decrypting the

executable to a separate plaintext copy prior to

execution and then counting on access control to thwart

any foe, we load the encrypted program into the

memory locations specified by the original ELF

program headers and then decrypt. This allows PPELF

programs to never be stored as plaintext ELF files in

non-volatile memory and only exist as plaintext in

volatile memory. Because the program is always

encrypted while stored on disk (non-volatile memory)

alteration, copying, and examination are prevented

before execution and erasing the program after

execution is made simple. The encrypted PPELF

program stored on disk may be safely deleted after

execution with the standard Linux rm command. The

deleted program may be recovered by an attacker;

however, the un-erased program will be in its encrypted

form and is therefore useless with out the decryption

The Third International Conference on Availability, Reliability and Security

0-7695-3102-4/08 $25.00 © 2008 IEEEDOI 10.1109/ARES.2008.160

452

The Third International Conference on Availability, Reliability and Security

0-7695-3102-4/08 $25.00 © 2008 IEEEDOI 10.1109/ARES.2008.160

452

The Third International Conference on Availability, Reliability and Security

0-7695-3102-4/08 $25.00 © 2008 IEEEDOI 10.1109/ARES.2008.160

452

key.

We chose ELF [1] as a starting point for our work

because it is a commonly used binary format with the

Linux operating system and it is open source.

2. Related Work

There are many programs call packers which are

used to pack executable code into an encrypted and

obfuscated bundle. These packers generally wrap the

executable with a second program which decrypts and

or de-obfuscates the executable as it exectutes. These

programs are often developed by and used by hackers

to attempt to stop their executables from being

scanned, reverse engineered, and traced. Shiva [9] is

one such system which uses both encryption and

obfuscation to protect ELF executables. Shiva embeds

a decryptor program in the program which decrypts

and de-obfuscates small blocks of code as they are

needed at run time. The Shiva authors also note that

large programs suffer run time delays. UPX [10] is also

an executable packer. UPX makes no claims about

providing executable privacy; rather it is intended as an

executable compression tool. We mention it here

because UPX decompresses ELF programs into

memory rather than decompressing onto disk then

executing the decompressed program. UPX actually

decompresses the program into a memory image then

calls exec() to launch the decompressed code. This is

not the same mechanism used to decrypt PPELF but

both avoid transforming the executable only to write it

back to nonvolatile storage.

File system encryption [11] encrypts the data written

to a file system or volume. In such systems, which are

readily available on Linux and Windows operating

systems data is encrypted as part of the write process

and decrypted when read from the file system. Such

systems do not address executable encryption directly,

but, since all data on the file system is encrypted and

decrypted only when accessed this comes close to

PPELF. The main difference between file system

encryption and PPELF is that file system encryption is

a system level approach rather than application level.

If there is more than one file system mounted on a

platform and not all of the file systems are encrypted

then a user can easily decrypt a file by copying it from

the encrypted file system to a non-encrypted file

system. This is not true for PPELF, since the

encryption is at the application level, the simple act of

moving the program from one file system to another

will not decrypt the program.

4. Private Computing on Public Platforms

Private Computing on Public Platforms (PCPP) [4]

[5] [6] is a methodology and implementation for

executing applications on 3rd

party potentially

malicious remote platforms.

We define the following requirements for PCPP.

First, PCPP must ensure that all executable code, data, and control flow remain unaltered, unmonitored, and

unrecorded before, during, and after execution on the

remote platform. The second requirement is a software

only implementation, which enables a much larger set

of potential public platforms. Third, PCPP is an opt-in

system. PCPP will not execute applications on an idle

host without prior permission. Furthermore, the opt-in

process allows code to be downloaded and installed on

a platform before it is used. Fourth, PCPP must be

able to validate a public platform before use. This

ensures that the platform is capable of executing the

PCPP protected application and allows PCPP to assess

the relative safety of engaging with the public platform.

Fifth, PCPP provides application level protection. As

such, only chosen applications are subject to its

protections, and consequently only those applications

are subject to its overhead. Finally, as encryption is

fundamental to PCPP, all encryption keys must be

robustly protected while in use on the public platform.

PCPP uses a host assessment feature plus 4 active

security building blocks: PPELF, secure context

switch, and secure I/O, encryption key protection to

achieve its security goals.

The host assessment evaluates 3rd

party remote

platforms to ensure that their configuration matches the

execution and security requirements of the PCPP

application. PPELF is a new binary executable format

designed to protect the executable code while it is

stored in non-volatile memory on the 3rd

party remote

platform and also offers a secure executable launch

process. Secure context switch encrypts all PCPP

application state when the PCPP application loses

ownership of the host processor and decrypts the state

when the PCPP application regains control of the host

processor. Secure I/O protects open files by encrypting

the memory mapped contents of all open PCPP files

when the PCPP application loses ownership of the host

processor and decrypts the open files when the PCPP

application regains control of the host processor.

PPELF, secure context switch, and encrypted I/O all

rely upon robust encryption key security on the public

platform. Currently, we use Linux Key Retention

Service [7] key rings to store and protect the PCPP

453453453

keys. However, we plan to offer a more secure

solution as future work.

We are introducing PPELF, SCS, SIO in a set of

three separate papers. This paper discusses the first of

the three, the PPELF encrypted binary executable.

5. PPELF Overview

The ELF binary format uses three headers to

describe the contents of an ELF executable: ELF

headers, program headers, and section headers. The

ELF header describes the overall contents of the file

itself, including the size and location in the file of the

next two headers the program header and the section

header. The program headers describe how the

program should be loaded into memory including a

pointer to the interpreter to be used and PT_LOAD

segments which call out specific sections of the file to

be loaded at specific virtual addresses. The section

headers are pointers in the file to sections of code and

data. We made use of the ELF header and program

headers significantly, but were able to avoid any

parsing of the section headers.

To create a PPELF file we read in an ELF file and

then encrypt all contents after the ELF header. We use

a 128-bit AES encryption with a pseudo randomly

created initialization vector. After encryption we

change the magic number, the first 4 bytes of the file,

from ‘¿ELF’ to ‘¿PLF’. ‘¿’ is actually 0x177, which is

unprintable, and indicates the file is binary, rather than

a script. The remainder of the magic number gives the

file type.

We created a utility called elf2pcpp which creates

PPELF binaries from ELF binaries. PPELF reads the

encryption key read from a session key ring using the

Linux key retention service [7]. After encrypting the

program headers, sections headers, and all code and

data segments the initialization vector is stored at the

end of the file. We use 128-bit AES counter mode

encryption with a pseudo random initialization vector.

The use of counter mode encryption allows random

access to the file for decryption on a byte granularity.

When a PCPP session is started it first asks the user

for a password, which is then hashed with an MD5

implementation to create a 128-bit encryption key

which is attached to the PCPP sessions key ring. This

key ring is only available to applications subsequently

launched from the current shell running in the PCPP

session. We use this same methodology when running

PPELF applications. Normally, the same PCPP

session would not used to create and run PPELF

applications. We expect PPELF applications to be

created once in a PCPP session, then run at later times

and possible on other machines under a separate PCPP

session which was created by entering the same

password and therefore shares the same encryption key.

Encryption of PPELF programs is a simple step.

Our methodology for loading the binary with the Linux

binary into the memory locations from which it will be

executed before decryption provides the true novelty of

this approach. In a nutshell, Linux loads ELF, and now

PPELF, programs into memory by first calling a binary

format handler load_binary routine to copy program

segments into memory, and then calling the GLIBC

interpreter (ld.so) to load shared libraries and jump to

the program’s entry point. We modified this process to

support PPELF by creating a new Linux PPELF binary

format handler which decrypts the PPELF program

headers leaving the code and data segments encrypted

and then calls a modified interpreter which completes

the decryption of the actual code and data sections now

already in memory.

Figure 1 shows the procedure to load a PPELF

program, app, into memory prior to execution. We call

the program file app.exe to delineate the program file

from the executed process, app. Figure 1 repeats the

same set of images, with variations, 4 times

corresponding to 4 times steps in the PPELF load

procedure. We call these time steps frames. Each

frame contains a box to note the active process, a box

listing waiting processes, a hard disk drive (HDD)

showing an encrypted app.exe executable file, and

DRAM with multiple memory blocks labeled with their

corresponding process. The first frame shows the

exec() system call active. exec() calls the binary loader

routines from all registered binary formats until one

binary loader returns successfully. Since each binary

format starts with a unique magic number, all binary

loaders will fail except the binfmt_pcpp binary loader.

The second frame shows the binfmt_pcpp binary loader

mapping the contents of app.exe to memory pages

owned by the app process. Binfmt_pcpp first decrypts

the program headers to learn how to map the loadable

segments to memory. Then, binfmt_pcpp copies the

still encrypted code and data segments to memory.

Finally, binfmt_pcpp loads the specified interpreter,

called interp in Figure 1, and jumps to its start address.

After use in binfmt_pcpp, the decrypted program

headers in DRAM are overwritten with zeroes and the

memory is returned to the operating system. The third

time step in Figure 1 shows the interpreter running.

Since the binfmt_pcpp binary loader mapped the code

454454454

and data segments of app.exe to the virtual addresses

specified by its program headers, the interpreter can

decrypt these segments in situ, meaning no other

memory is allocated to hold intermediate encryption

results, leaving nothing to clean-up. After decryption,

the interpreter loads any required shared libraries and

then jumps to the app start point. The final frame

shows app, with decrypted code and data segments in

DRAM, as the active process.

The PPELF program is never stored as plaintext in

non-volatile memory. This is shown in all four frames

of Figure 1.

We chose to safely dispose of the decrypted

program headers used in the binfmt_pcpp routine rather

than leave them decrypted when we jump into the

interpreter. This provides extra security at the expense

of decrypting the program headers a second time. This

is possible since the program headers are relatively

small, totaling a 200-1000 bytes for a typical program.

This choice was made because the binfmt_pcpp routine

runs with interrupts OFF, which stops context switches,

while the interpreter does not have this luxury.

Between leaving the binfmt_pcpp routine and starting

in the interpreter there may be a context switch and

unrelated process may get control of the processor

before the interpreter runs. As such, it is best to leave

all of the contents of the protected application

encrypted until the interpreter actually executes.

Since decryption takes place in the interpreter, we

modified the GNU GLIBC interpreter, the default

interpreter on Linux machines to handle PPELF. The

interpreter does not have access to the program’s file; it

doesn’t know its name or how to find it. The interpreter

can only access the segments of the file which were

loaded into memory by binfmt_pcpp. The ELF header

and program headers are always loaded into memory,

even though they are not executable segments. These

are used by the interpreter when loading the share

libraries. Before the interpreter can use the program

headers we must decrypt them. We use the ELF

headers to first determine if we are loading a PPELF or

ELF program and then to calculate the location of the

program headers and decrypt these in situ. We then

loop through the program headers decrypting all the

segments in memory. Again, we read the encryption

key and initialization vector from a Linux key retention

service session key ring. After completing decryption

the rest of the interpreter proceeds unchanged.

5.1 PPELF versus ELF

We converted three user applications and one set of

benchmark programs from ELF to PPELF. We used

these applications to validate the correctness of our

PPELF implementation and to measure application

launch times and run times for comparison between

PPELF and ELF.

We used the three user applications: helloworld,

Vim and Firefox to show the launch time increase

added to PPELF programs over their ELF cousins. We

used the LMBench benchmark programs to

demonstrate that except for the launch time increase;

there is no other run time impact when using a PPELF

program.

LMBench [8] is a commonly used benchmark suite

which measures operating system efficiency.

LMBench contains multiple benchmarks, each of which

has an associated ELF program. We ran the LMBench

Waiting

DRAM DRAM

Waiting

Active Active

Time

HDD

app.exe

Active

Waiting

DRAM

exec

proc2 bash

HDD

app.exe

interp

HDD

app.exe

app

proc2

bash

HDD

app.exe

Active

Waiting

DRAM

binfmt_

pcpp

proc2

bash proc2

bash

app

binfmt_

pcpp

interp

proc2 bash

proc2 bash

proc2 bash

interp app app

proc2

bash

Figure 1: PPELF Load Procedure

Frame: 1 2 3 4

455455455

with the benchmark programs in ELF format and again

after converting to PPELF.

Helloworld is a simple program which prints “Hello

World” to standard out and then exits. This program

was primarily used during early debug. Vim and

Firefox were chosen for their popularity and size. By

choosing 3 programs of progressively larger sizes we

can show that the launch time increases as the program

size increases.

5.2 Launch and Run Time Comparison

We used Firefox, helloworld, and Vim to measure

the launch time penalty associated with using the

PPELF binary format. We took two measurements

when executing each program. First, we measured the

time spent in the kernel’s binary format handler module

by placing timer start and stop calls at the entry and

exit points of the PPELF and ELF binary format

handlers. The binary format handlers for ELF and

PPELF are separate Linux kernel modules, however,

the PPELF version is a modification of the ELF

version. This made it easy to insert the timer start and

stop calls at consistent locations in both binary format

handlers. Second, we used an existing timer

mechanism already in place in the GNU LIBC

interpreter to measure time spent in the interpreter.

The interpreter used for both the ELF programs and

PPELF programs was the same since the interpreter

was modified to handle both formats.

We ran each program 5 times in its PPELF form and

its ELF form while measuring the aforementioned

times. Table 1 and Table 2 show the average times for

each of the 3 applications. The columns marked binfmt

indicate the time spent in the binary format handler, the

columns marked interp indicate the time spent in the

interpreter, and the total is the sum of the time in the

binary format handler and the time in the interpreter.

Table 1: ELF Launch Times ELF Program Size

[MB] Binfmt

[uS]

Interp

[uS]

Total

[uS]

Helloworld 0.007 27 678 705

Vim 1.2 27 7000 7027

Firefox 10.5 28 142000 142028

Table 2: PPELF Launch Times PPELF Program Size

[MB] Binfmt

[uS]

Interp

[uS]

Total

[uS]

Helloworld 0.007 42 947 989

Vim 1.2 44 183000 183044

Firefox 10.5 41 1023000 1023041

The data from tables Table 1 and Table 2 illustrates

several points. First, the times marked binfmt is the

amount of time spent in the binary format handler.

These times are slower for PPELF than ELF.

However, the increased delay is static across the three

test programs. This is the result of only decrypting the

program headers in binfmt_pcpp. The program

headers are small at 32 bytes each. There are multiple

program headers per executable with 7 program

headers in the helloworld program, 8 in Firefox, and 9

in Vim. This makes the total number of bytes for

decryption in the binary format handler vary from 256

bytes to 288 bytes.

The times marked interp are the time spent in the

interpreter for each of the three programs. These times

dominate the total launch time both for ELF and

PPELF programs. For the ELF case much of the time

spent in the interpreter is spent loading share libraries

from disk and relocating non-reentrant shared library

code in memory. For PPELF the time spent in the

interpreter is dominated by the decryption of the

PT_LOAD segments. In all three cases the time spent

in the interpreter increases dramatically, increasing

approximately 50% for the helloworld program and an

order of magnitude for Firefox programs with the

amount of increase tracking with the program size.

The increase for Vim is more than 25x.

The 25x delta for Vim points out a significant

difference in how PPELF and ELF programs are

mapped in to volatile memory. Linux uses demand

paging when mapping files to memory, including,

executables. This means that the entire executable is

initially mapped to memory by assigning page table

entries and virtual addresses, but, the actual contents of

the file are not copied from the hard disk drive to

memory until the a memory mapped virtual address is

accessed. Since, PPELF decrypts all of the contents of

the program in the interpreter all the pages must be

read into memory from disk before decryption. Vim

does very little on start-up. It simply clears the screen,

prints a few characters, and then waits for input.

Because there is little activity on start-up the ELF

version accesses far less virtual memory pages than

PPELF which needs to access all of Vim’s instruction

memory pages for decryption.

With the exception of Firefox, the large increase in

launch time is still small enough to make it

unnoticeable to a human user. For Firefox, the same is

actually true because on our slow system although the

Firefox window boundary appears quickly, the total

time for all of the graphics to load and for Firefox to

456456456

become usable takes about 6 seconds when measured

on a stop watch. When using the PPELF version of

Firefox there is a 1 second increase, but we argue going

from 6 to 7 seconds on a slow system is still largely

unnoticeable to the user. We also note here that these

numbers were gathered on an AMD K7 processor

running at 900 MHz. On a modern processor the

relative delays may stay the same, but the noticeable

impact to the user would diminish.

In addition to the launch time comparisons above,

we also ran the LMBench benchmark suite with ELF

executables and with PPELF executables. The bench

mark suite measures a total of 44 marks. To do this the

suite uses a set of 37 ELF binary programs. We ran the

benchmarks 5 times with these 37 binary programs left

as ELF programs and then reran the benchmarks with

the binary programs converted to the PPELF format.

Of the 44 marks, we noticed a change only in one.

This was the benchmark exec proc which measure the

time to launch a new process with the execve. This

increase mirrors the results we saw in our launch time

experiments. In fact, the program used to measure the

exec proc was a simple hello world program. All other

marks did not change when running with PPELF

executables. This is the expected result since once a

PPELF program is in memory its memory image is

identical to that of its ELF ancestor.

All experimental data cited in this paper was

collected using a computer with a 900 MHz AMD K7

with 384 MB of system memory running Linux operating

system with kernel version 2.6.20.6 [2] and using our

modified ELF interpreter based on the ELF interpreter

included with the GNU C Library version 2.6 [3].

6. Future Work

We have not completed implementations of the

Secure Context Switch (SCS) and Secure I/O (SIO)

features. We plan future published works showing the

architectures of these building blocks and examining

the overhead related to these technologies.

Currently, we use the Linux Key Retention Service

for encryption key storage and protection. We hope to

develop a key storage methodology which makes it

computationally infeasible for non-PCPP concurrent

processes to access the key while still affording the

PCPP building blocks quick access to the protected

keys.

7. Conclusion

PPELF is an encrypted binary executable format for

the Linux operating system based upon the ELF binary

executable format. PPELF programs are encrypted

ELF program which are decrypted by the ELF

interpreter after being mapped from disk to dynamic

memory. Since PPELF programs are decrypted in

place after being mapped into memory PPELF

programs are never stored in a decrypted form on non-

volatile memory.

PPELF programs do incur a nominal delay in launch

time which increases with the size of the program, but

there is no increase in run time of the program after it is

loaded into memory.

PPELF solves one of the necessary problems of

PCPP by protecting the privacy of executable code

before and after it is executed and after it is deleted

from non-volatile memory.

8. References

[1] Youngdale, E. 1995. Kernel Korner: The ELF Object

File Format by Dissection. Linux J. 1995, 13es (May.

1995), 15

[2] The Linux Kernel Archives. http://www.kernel.org/

[3] GNU C Library.

http://www.gnu.org/software/libc/libc.html

[4] Morris, T.; Nair, V.S.S. PCPP: Private Computing on

Public Platforms a New Paradigm in Public Computing.

2nd IEEE International Symposium on Wireless

Pervasive Computing (ISWPC 2007), Feb. 2007

[5] Morris, T.; Nair, V.S.S. PCPP: On Remote Host

Assessment via Naive Bayesian Classification. IEEE

International Parallel and Distributed Processing

Symposium (IDPDS 2007), 26-30 March 2007,

Pages:1-8

[6] Morris, T., Nair, V.S.S. Private Computing on a Public

Platform. Department of Computer Science and

Engineering, Southern Methodist University, Technical

Report 06-CSE-01, 2006

[7] Kumar A., Chopdekar S. Getting Started with the Linux

key retention service.

http://www.ibm.com/developerworks/linux/library/l-

key-retention.html

[8] LMBench – Tools for Performance Analysis.

http://www.bitmover.com/lmbench/

[9] Mehta N., Clowes S. (2003), “Shiva: Advances in ELF

Binary Encryption,” CanSecWest.

http://cansecwest.com/core03/shiva.ppt

[10] Oberhumer M., Molnár L., Reiser J., UPX:The

Ultimate Packer for Executables.

http://upx.sourceforge.net/

[11] Giles, B. 1998. Encrypted File Systems. Linux Journal

1998, Issue 51es (July 1998)

[12] Foster. IFIP International Conference on Network and

Parallel Computing, Springer-Verlag LNCS 3779, pp

2-13, 2005.

[13] SETI@Home. http://setiathome.berkeley.edu/

457457457