Privacy Protected ELF for
Private Computing on Public Platforms
Thomas Morris and V.S.S Nair
SMU HACNet Lab
Southern Methodist University
Dallas, TX, USA
{tmorris,nair}@engr.smu.edu
Abstract
Private Computing on Public Platforms (PCPP) is a
new technology designed to enable secure and private
execution of applications on remote potentially hostile
public platforms. PCPP uses a host assessment to validate
a host’s hardware and software configuration and then
uses 4 active security building blocks which together
allow an application to remain unaltered, unmonitored,
and unrecorded before, during, and after execution on the
public platform. Privacy Protected ELF (PPELF) is the
building block used by PCPP to protect application
executable code while it is stored on the public platform
and to provide a secure binary load mechanism. PPELF
is an encrypted binary executable format and
methodology which uses just in time decryption during
the executable load procedure. In this paper we describe
the PPELF file format, our new PPELF binary format
loader, and our updates to the GLIBC ELF interpreter to
support PPELF. We also provide experimental results
detailing the initial load-time and run-time penalty
associated with the use of PPELF.
1. Introduction
Distributed computing technologies which enable the
use of idle processors by remote users are abundant.
SETI@Home [13] is a popular distributed computing
application run by the University of California at
Berkeley which uses remote computers to download
and process data in the search for extra terrestrial
intelligence. The Globus Toolkit [12] is an open
source toolkit which has been used to build many
working grids which are collections of computers
linked to allow the sharing of computing resources
across locations. Although there have been many
successes in this area there has been a lack of large-
scale commercial acceptance. We believe that this
trend stems from a need for better application security
and privacy while applications are stored and executed
on remote platforms. Private Computing on Public
Platforms (PCPP) [4] [5] [6] was developed as a
solution to this problem. PCPP enables the secure and
private use of remote platforms by first performing a
host assessment to validate the hardware and software
configuration of the system and then using 4 active
security building blocks which assure that applications
executed on the remote platforms remain unaltered,
unmonitored, and unrecorded before, during, and after
execution.
Privacy Protected ELF (PPELF) is one of the 4
PCPP active security building blocks. PPELF was
designed to protect executable programs while they are
stored on a public platform. PPELF is an encrypted
extension of the ELF [1] binary format and includes a
methodology for securely loading the encrypted
program into memory. Rather, than decrypting the
executable to a separate plaintext copy prior to
execution and then counting on access control to thwart
any foe, we load the encrypted program into the
memory locations specified by the original ELF
program headers and then decrypt. This allows PPELF
programs to never be stored as plaintext ELF files in
non-volatile memory and only exist as plaintext in
volatile memory. Because the program is always
encrypted while stored on disk (non-volatile memory)
alteration, copying, and examination are prevented
before execution and erasing the program after
execution is made simple. The encrypted PPELF
program stored on disk may be safely deleted after
execution with the standard Linux rm command. The
deleted program may be recovered by an attacker;
however, the un-erased program will be in its encrypted
form and is therefore useless with out the decryption
The Third International Conference on Availability, Reliability and Security
0-7695-3102-4/08 $25.00 © 2008 IEEEDOI 10.1109/ARES.2008.160
452
The Third International Conference on Availability, Reliability and Security
0-7695-3102-4/08 $25.00 © 2008 IEEEDOI 10.1109/ARES.2008.160
452
The Third International Conference on Availability, Reliability and Security
0-7695-3102-4/08 $25.00 © 2008 IEEEDOI 10.1109/ARES.2008.160
452
key.
We chose ELF [1] as a starting point for our work
because it is a commonly used binary format with the
Linux operating system and it is open source.
2. Related Work
There are many programs call packers which are
used to pack executable code into an encrypted and
obfuscated bundle. These packers generally wrap the
executable with a second program which decrypts and
or de-obfuscates the executable as it exectutes. These
programs are often developed by and used by hackers
to attempt to stop their executables from being
scanned, reverse engineered, and traced. Shiva [9] is
one such system which uses both encryption and
obfuscation to protect ELF executables. Shiva embeds
a decryptor program in the program which decrypts
and de-obfuscates small blocks of code as they are
needed at run time. The Shiva authors also note that
large programs suffer run time delays. UPX [10] is also
an executable packer. UPX makes no claims about
providing executable privacy; rather it is intended as an
executable compression tool. We mention it here
because UPX decompresses ELF programs into
memory rather than decompressing onto disk then
executing the decompressed program. UPX actually
decompresses the program into a memory image then
calls exec() to launch the decompressed code. This is
not the same mechanism used to decrypt PPELF but
both avoid transforming the executable only to write it
back to nonvolatile storage.
File system encryption [11] encrypts the data written
to a file system or volume. In such systems, which are
readily available on Linux and Windows operating
systems data is encrypted as part of the write process
and decrypted when read from the file system. Such
systems do not address executable encryption directly,
but, since all data on the file system is encrypted and
decrypted only when accessed this comes close to
PPELF. The main difference between file system
encryption and PPELF is that file system encryption is
a system level approach rather than application level.
If there is more than one file system mounted on a
platform and not all of the file systems are encrypted
then a user can easily decrypt a file by copying it from
the encrypted file system to a non-encrypted file
system. This is not true for PPELF, since the
encryption is at the application level, the simple act of
moving the program from one file system to another
will not decrypt the program.
4. Private Computing on Public Platforms
Private Computing on Public Platforms (PCPP) [4]
[5] [6] is a methodology and implementation for
executing applications on 3rd
party potentially
malicious remote platforms.
We define the following requirements for PCPP.
First, PCPP must ensure that all executable code, data, and control flow remain unaltered, unmonitored, and
unrecorded before, during, and after execution on the
remote platform. The second requirement is a software
only implementation, which enables a much larger set
of potential public platforms. Third, PCPP is an opt-in
system. PCPP will not execute applications on an idle
host without prior permission. Furthermore, the opt-in
process allows code to be downloaded and installed on
a platform before it is used. Fourth, PCPP must be
able to validate a public platform before use. This
ensures that the platform is capable of executing the
PCPP protected application and allows PCPP to assess
the relative safety of engaging with the public platform.
Fifth, PCPP provides application level protection. As
such, only chosen applications are subject to its
protections, and consequently only those applications
are subject to its overhead. Finally, as encryption is
fundamental to PCPP, all encryption keys must be
robustly protected while in use on the public platform.
PCPP uses a host assessment feature plus 4 active
security building blocks: PPELF, secure context
switch, and secure I/O, encryption key protection to
achieve its security goals.
The host assessment evaluates 3rd
party remote
platforms to ensure that their configuration matches the
execution and security requirements of the PCPP
application. PPELF is a new binary executable format
designed to protect the executable code while it is
stored in non-volatile memory on the 3rd
party remote
platform and also offers a secure executable launch
process. Secure context switch encrypts all PCPP
application state when the PCPP application loses
ownership of the host processor and decrypts the state
when the PCPP application regains control of the host
processor. Secure I/O protects open files by encrypting
the memory mapped contents of all open PCPP files
when the PCPP application loses ownership of the host
processor and decrypts the open files when the PCPP
application regains control of the host processor.
PPELF, secure context switch, and encrypted I/O all
rely upon robust encryption key security on the public
platform. Currently, we use Linux Key Retention
Service [7] key rings to store and protect the PCPP
453453453
keys. However, we plan to offer a more secure
solution as future work.
We are introducing PPELF, SCS, SIO in a set of
three separate papers. This paper discusses the first of
the three, the PPELF encrypted binary executable.
5. PPELF Overview
The ELF binary format uses three headers to
describe the contents of an ELF executable: ELF
headers, program headers, and section headers. The
ELF header describes the overall contents of the file
itself, including the size and location in the file of the
next two headers the program header and the section
header. The program headers describe how the
program should be loaded into memory including a
pointer to the interpreter to be used and PT_LOAD
segments which call out specific sections of the file to
be loaded at specific virtual addresses. The section
headers are pointers in the file to sections of code and
data. We made use of the ELF header and program
headers significantly, but were able to avoid any
parsing of the section headers.
To create a PPELF file we read in an ELF file and
then encrypt all contents after the ELF header. We use
a 128-bit AES encryption with a pseudo randomly
created initialization vector. After encryption we
change the magic number, the first 4 bytes of the file,
from ‘¿ELF’ to ‘¿PLF’. ‘¿’ is actually 0x177, which is
unprintable, and indicates the file is binary, rather than
a script. The remainder of the magic number gives the
file type.
We created a utility called elf2pcpp which creates
PPELF binaries from ELF binaries. PPELF reads the
encryption key read from a session key ring using the
Linux key retention service [7]. After encrypting the
program headers, sections headers, and all code and
data segments the initialization vector is stored at the
end of the file. We use 128-bit AES counter mode
encryption with a pseudo random initialization vector.
The use of counter mode encryption allows random
access to the file for decryption on a byte granularity.
When a PCPP session is started it first asks the user
for a password, which is then hashed with an MD5
implementation to create a 128-bit encryption key
which is attached to the PCPP sessions key ring. This
key ring is only available to applications subsequently
launched from the current shell running in the PCPP
session. We use this same methodology when running
PPELF applications. Normally, the same PCPP
session would not used to create and run PPELF
applications. We expect PPELF applications to be
created once in a PCPP session, then run at later times
and possible on other machines under a separate PCPP
session which was created by entering the same
password and therefore shares the same encryption key.
Encryption of PPELF programs is a simple step.
Our methodology for loading the binary with the Linux
binary into the memory locations from which it will be
executed before decryption provides the true novelty of
this approach. In a nutshell, Linux loads ELF, and now
PPELF, programs into memory by first calling a binary
format handler load_binary routine to copy program
segments into memory, and then calling the GLIBC
interpreter (ld.so) to load shared libraries and jump to
the program’s entry point. We modified this process to
support PPELF by creating a new Linux PPELF binary
format handler which decrypts the PPELF program
headers leaving the code and data segments encrypted
and then calls a modified interpreter which completes
the decryption of the actual code and data sections now
already in memory.
Figure 1 shows the procedure to load a PPELF
program, app, into memory prior to execution. We call
the program file app.exe to delineate the program file
from the executed process, app. Figure 1 repeats the
same set of images, with variations, 4 times
corresponding to 4 times steps in the PPELF load
procedure. We call these time steps frames. Each
frame contains a box to note the active process, a box
listing waiting processes, a hard disk drive (HDD)
showing an encrypted app.exe executable file, and
DRAM with multiple memory blocks labeled with their
corresponding process. The first frame shows the
exec() system call active. exec() calls the binary loader
routines from all registered binary formats until one
binary loader returns successfully. Since each binary
format starts with a unique magic number, all binary
loaders will fail except the binfmt_pcpp binary loader.
The second frame shows the binfmt_pcpp binary loader
mapping the contents of app.exe to memory pages
owned by the app process. Binfmt_pcpp first decrypts
the program headers to learn how to map the loadable
segments to memory. Then, binfmt_pcpp copies the
still encrypted code and data segments to memory.
Finally, binfmt_pcpp loads the specified interpreter,
called interp in Figure 1, and jumps to its start address.
After use in binfmt_pcpp, the decrypted program
headers in DRAM are overwritten with zeroes and the
memory is returned to the operating system. The third
time step in Figure 1 shows the interpreter running.
Since the binfmt_pcpp binary loader mapped the code
454454454
and data segments of app.exe to the virtual addresses
specified by its program headers, the interpreter can
decrypt these segments in situ, meaning no other
memory is allocated to hold intermediate encryption
results, leaving nothing to clean-up. After decryption,
the interpreter loads any required shared libraries and
then jumps to the app start point. The final frame
shows app, with decrypted code and data segments in
DRAM, as the active process.
The PPELF program is never stored as plaintext in
non-volatile memory. This is shown in all four frames
of Figure 1.
We chose to safely dispose of the decrypted
program headers used in the binfmt_pcpp routine rather
than leave them decrypted when we jump into the
interpreter. This provides extra security at the expense
of decrypting the program headers a second time. This
is possible since the program headers are relatively
small, totaling a 200-1000 bytes for a typical program.
This choice was made because the binfmt_pcpp routine
runs with interrupts OFF, which stops context switches,
while the interpreter does not have this luxury.
Between leaving the binfmt_pcpp routine and starting
in the interpreter there may be a context switch and
unrelated process may get control of the processor
before the interpreter runs. As such, it is best to leave
all of the contents of the protected application
encrypted until the interpreter actually executes.
Since decryption takes place in the interpreter, we
modified the GNU GLIBC interpreter, the default
interpreter on Linux machines to handle PPELF. The
interpreter does not have access to the program’s file; it
doesn’t know its name or how to find it. The interpreter
can only access the segments of the file which were
loaded into memory by binfmt_pcpp. The ELF header
and program headers are always loaded into memory,
even though they are not executable segments. These
are used by the interpreter when loading the share
libraries. Before the interpreter can use the program
headers we must decrypt them. We use the ELF
headers to first determine if we are loading a PPELF or
ELF program and then to calculate the location of the
program headers and decrypt these in situ. We then
loop through the program headers decrypting all the
segments in memory. Again, we read the encryption
key and initialization vector from a Linux key retention
service session key ring. After completing decryption
the rest of the interpreter proceeds unchanged.
5.1 PPELF versus ELF
We converted three user applications and one set of
benchmark programs from ELF to PPELF. We used
these applications to validate the correctness of our
PPELF implementation and to measure application
launch times and run times for comparison between
PPELF and ELF.
We used the three user applications: helloworld,
Vim and Firefox to show the launch time increase
added to PPELF programs over their ELF cousins. We
used the LMBench benchmark programs to
demonstrate that except for the launch time increase;
there is no other run time impact when using a PPELF
program.
LMBench [8] is a commonly used benchmark suite
which measures operating system efficiency.
LMBench contains multiple benchmarks, each of which
has an associated ELF program. We ran the LMBench
Waiting
DRAM DRAM
Waiting
Active Active
Time
HDD
app.exe
Active
Waiting
DRAM
exec
proc2 bash
HDD
app.exe
interp
HDD
app.exe
app
proc2
bash
HDD
app.exe
Active
Waiting
DRAM
binfmt_
pcpp
proc2
bash proc2
bash
app
binfmt_
pcpp
interp
proc2 bash
proc2 bash
proc2 bash
interp app app
proc2
bash
Figure 1: PPELF Load Procedure
Frame: 1 2 3 4
455455455
with the benchmark programs in ELF format and again
after converting to PPELF.
Helloworld is a simple program which prints “Hello
World” to standard out and then exits. This program
was primarily used during early debug. Vim and
Firefox were chosen for their popularity and size. By
choosing 3 programs of progressively larger sizes we
can show that the launch time increases as the program
size increases.
5.2 Launch and Run Time Comparison
We used Firefox, helloworld, and Vim to measure
the launch time penalty associated with using the
PPELF binary format. We took two measurements
when executing each program. First, we measured the
time spent in the kernel’s binary format handler module
by placing timer start and stop calls at the entry and
exit points of the PPELF and ELF binary format
handlers. The binary format handlers for ELF and
PPELF are separate Linux kernel modules, however,
the PPELF version is a modification of the ELF
version. This made it easy to insert the timer start and
stop calls at consistent locations in both binary format
handlers. Second, we used an existing timer
mechanism already in place in the GNU LIBC
interpreter to measure time spent in the interpreter.
The interpreter used for both the ELF programs and
PPELF programs was the same since the interpreter
was modified to handle both formats.
We ran each program 5 times in its PPELF form and
its ELF form while measuring the aforementioned
times. Table 1 and Table 2 show the average times for
each of the 3 applications. The columns marked binfmt
indicate the time spent in the binary format handler, the
columns marked interp indicate the time spent in the
interpreter, and the total is the sum of the time in the
binary format handler and the time in the interpreter.
Table 1: ELF Launch Times ELF Program Size
[MB] Binfmt
[uS]
Interp
[uS]
Total
[uS]
Helloworld 0.007 27 678 705
Vim 1.2 27 7000 7027
Firefox 10.5 28 142000 142028
Table 2: PPELF Launch Times PPELF Program Size
[MB] Binfmt
[uS]
Interp
[uS]
Total
[uS]
Helloworld 0.007 42 947 989
Vim 1.2 44 183000 183044
Firefox 10.5 41 1023000 1023041
The data from tables Table 1 and Table 2 illustrates
several points. First, the times marked binfmt is the
amount of time spent in the binary format handler.
These times are slower for PPELF than ELF.
However, the increased delay is static across the three
test programs. This is the result of only decrypting the
program headers in binfmt_pcpp. The program
headers are small at 32 bytes each. There are multiple
program headers per executable with 7 program
headers in the helloworld program, 8 in Firefox, and 9
in Vim. This makes the total number of bytes for
decryption in the binary format handler vary from 256
bytes to 288 bytes.
The times marked interp are the time spent in the
interpreter for each of the three programs. These times
dominate the total launch time both for ELF and
PPELF programs. For the ELF case much of the time
spent in the interpreter is spent loading share libraries
from disk and relocating non-reentrant shared library
code in memory. For PPELF the time spent in the
interpreter is dominated by the decryption of the
PT_LOAD segments. In all three cases the time spent
in the interpreter increases dramatically, increasing
approximately 50% for the helloworld program and an
order of magnitude for Firefox programs with the
amount of increase tracking with the program size.
The increase for Vim is more than 25x.
The 25x delta for Vim points out a significant
difference in how PPELF and ELF programs are
mapped in to volatile memory. Linux uses demand
paging when mapping files to memory, including,
executables. This means that the entire executable is
initially mapped to memory by assigning page table
entries and virtual addresses, but, the actual contents of
the file are not copied from the hard disk drive to
memory until the a memory mapped virtual address is
accessed. Since, PPELF decrypts all of the contents of
the program in the interpreter all the pages must be
read into memory from disk before decryption. Vim
does very little on start-up. It simply clears the screen,
prints a few characters, and then waits for input.
Because there is little activity on start-up the ELF
version accesses far less virtual memory pages than
PPELF which needs to access all of Vim’s instruction
memory pages for decryption.
With the exception of Firefox, the large increase in
launch time is still small enough to make it
unnoticeable to a human user. For Firefox, the same is
actually true because on our slow system although the
Firefox window boundary appears quickly, the total
time for all of the graphics to load and for Firefox to
456456456
become usable takes about 6 seconds when measured
on a stop watch. When using the PPELF version of
Firefox there is a 1 second increase, but we argue going
from 6 to 7 seconds on a slow system is still largely
unnoticeable to the user. We also note here that these
numbers were gathered on an AMD K7 processor
running at 900 MHz. On a modern processor the
relative delays may stay the same, but the noticeable
impact to the user would diminish.
In addition to the launch time comparisons above,
we also ran the LMBench benchmark suite with ELF
executables and with PPELF executables. The bench
mark suite measures a total of 44 marks. To do this the
suite uses a set of 37 ELF binary programs. We ran the
benchmarks 5 times with these 37 binary programs left
as ELF programs and then reran the benchmarks with
the binary programs converted to the PPELF format.
Of the 44 marks, we noticed a change only in one.
This was the benchmark exec proc which measure the
time to launch a new process with the execve. This
increase mirrors the results we saw in our launch time
experiments. In fact, the program used to measure the
exec proc was a simple hello world program. All other
marks did not change when running with PPELF
executables. This is the expected result since once a
PPELF program is in memory its memory image is
identical to that of its ELF ancestor.
All experimental data cited in this paper was
collected using a computer with a 900 MHz AMD K7
with 384 MB of system memory running Linux operating
system with kernel version 2.6.20.6 [2] and using our
modified ELF interpreter based on the ELF interpreter
included with the GNU C Library version 2.6 [3].
6. Future Work
We have not completed implementations of the
Secure Context Switch (SCS) and Secure I/O (SIO)
features. We plan future published works showing the
architectures of these building blocks and examining
the overhead related to these technologies.
Currently, we use the Linux Key Retention Service
for encryption key storage and protection. We hope to
develop a key storage methodology which makes it
computationally infeasible for non-PCPP concurrent
processes to access the key while still affording the
PCPP building blocks quick access to the protected
keys.
7. Conclusion
PPELF is an encrypted binary executable format for
the Linux operating system based upon the ELF binary
executable format. PPELF programs are encrypted
ELF program which are decrypted by the ELF
interpreter after being mapped from disk to dynamic
memory. Since PPELF programs are decrypted in
place after being mapped into memory PPELF
programs are never stored in a decrypted form on non-
volatile memory.
PPELF programs do incur a nominal delay in launch
time which increases with the size of the program, but
there is no increase in run time of the program after it is
loaded into memory.
PPELF solves one of the necessary problems of
PCPP by protecting the privacy of executable code
before and after it is executed and after it is deleted
from non-volatile memory.
8. References
[1] Youngdale, E. 1995. Kernel Korner: The ELF Object
File Format by Dissection. Linux J. 1995, 13es (May.
1995), 15
[2] The Linux Kernel Archives. http://www.kernel.org/
[3] GNU C Library.
http://www.gnu.org/software/libc/libc.html
[4] Morris, T.; Nair, V.S.S. PCPP: Private Computing on
Public Platforms a New Paradigm in Public Computing.
2nd IEEE International Symposium on Wireless
Pervasive Computing (ISWPC 2007), Feb. 2007
[5] Morris, T.; Nair, V.S.S. PCPP: On Remote Host
Assessment via Naive Bayesian Classification. IEEE
International Parallel and Distributed Processing
Symposium (IDPDS 2007), 26-30 March 2007,
Pages:1-8
[6] Morris, T., Nair, V.S.S. Private Computing on a Public
Platform. Department of Computer Science and
Engineering, Southern Methodist University, Technical
Report 06-CSE-01, 2006
[7] Kumar A., Chopdekar S. Getting Started with the Linux
key retention service.
http://www.ibm.com/developerworks/linux/library/l-
key-retention.html
[8] LMBench – Tools for Performance Analysis.
http://www.bitmover.com/lmbench/
[9] Mehta N., Clowes S. (2003), “Shiva: Advances in ELF
Binary Encryption,” CanSecWest.
http://cansecwest.com/core03/shiva.ppt
[10] Oberhumer M., Molnár L., Reiser J., UPX:The
Ultimate Packer for Executables.
http://upx.sourceforge.net/
[11] Giles, B. 1998. Encrypted File Systems. Linux Journal
1998, Issue 51es (July 1998)
[12] Foster. IFIP International Conference on Network and
Parallel Computing, Springer-Verlag LNCS 3779, pp
2-13, 2005.
[13] SETI@Home. http://setiathome.berkeley.edu/
457457457