Fidelius Charm: Isolating Unsafe Rust Code

Hussain M. J. AlmohriDepartment of Computer Science

Kuwait Universityalmohri@ieee.org

David EvansDepartment of Computer Science

University of Virginiaevans@virginia.edu

Abstract

The Rust programming language has a safe mem-ory model that promises to eliminate critical memorybugs. While the language is strong in doing so, itsmemory guarantees are lost when any unsafe blocksare used. Unsafe code is often needed to call libraryfunctions written in an unsafe language inside a Rustprogram. We present Fidelius Charm (FC)1, a sys-tem that protects a programmer-specified subset ofdata in memory from unauthorized access throughvulnerable unsafe libraries. FC does this by limitingaccess to the program’s memory while executing un-safe libraries. FC uses standard features of Rust andutilizes the Linux kernel as a trusted base for split-ting the address space into a trusted privileged regionunder the control of functions written in Rust and aregion available to unsafe external libraries. This pa-per presents our design and implementation of FC,presents two case studies for using FC in Rust TLSlibraries, and reports on experiments showing its per-formance overhead is low for typical uses.

1 Introduction

Rust is designed to provide strong memory safety, butprovides a way to escape its strict checking rules usingan explicit unsafe keyword. This enables systems-level Rust programming, and supports easy integra-tion with libraries written in unsafe languages suchas C. Code within an unsafe region can use all mem-ory in the Rust program’s process in arbitrary ways,jeopardizing all the safety guarantees made by theRust compiler. Unsafe regions enable calling unsafeand untrustworthy external libraries through Rust’sforeign function interface (FFI). When using FFI, theRust compiler cannot reason about memory vulner-

1A first version of this work was accepted for the proceed-ings of CODASPY’18.

abilities, repudiating all the safety guarantees Rustprogrammers work so hard to obtain.

The goal of this work is to enable FFI calls whileisolating some of the already allocated memory, limit-ing the potentially-vulnerable external code (runningwithin the same address space) from reading sensi-tive data. Rust has no mechanism to isolate an un-safe external function or limit its impact. A practicalisolation mechanism must be able to limit the dataavailable to an unsafe external function while allow-ing it to execute normally without the need to modifyor even inspect the unsafe code (which may only beavailable as a binary). Traditional ways to addressthis problem use computationally-intensive runtimesystems for compartmentalization of untrustworthycode, or complex (and seldom usable) modifications ofthe language’s compiler for monitoring and sandbox-ing vulnerable or unsafe regions of the code. Thesesolutions would require major changes to either theRust compiler or the unsafe code itself, both of whichwe want to avoid. Instead, we focus on a solutionthat leverages existing language and operating sys-tem mechanisms.

While no prior work addressed memory isolationfor unsafe Rust regions in particular, several previousworks have sought to confine code executing within asingle address space. Recent work such as Shreds [7],lwC [14], and SpaceJMP [12] provide thread-like ab-stractions for isolating memory. Codejail [29], a mem-ory sandbox system, has fewer dependencies and pro-vides a sandbox of the memory for unsafe libraries.Our approach reverses the sandbox model by isolat-ing a subset of the trusted region of the program andproviding the rest of the memory to the unsafe li-braries.

All the previous works, except Codejail, requiresome static analysis or special abstractions. We aimto have a practical and lightweight solution that al-lows programmers to make choices about which partsof the memory to protect, that requires only mem-

Page 1

ory page permissions that are supported by modernhardware, and that avoids hard modifications to theoperating system, only involving simple kernel exten-sions. Our approach is to (i) move sensitive programdata to protected pages before entering unsafe code,(ii) allow unsafe code to run normally without modi-fications, (iii) restore visibility of the protected statewhen unsafe code execution completes, and (iv) in-corporate a precise and efficient kernel-level monitorto ensure unsafe code cannot circumvent protections.

Threat Model Our solution assumes a trustedstarting state in which the operating system, andunderlying hardware, are not malicious and are im-plemented correctly. FC is designed to protect thememory of processes executing FC-ified Rust pro-grams, in which sensitive memory regions are pro-tected by isolated secure compartments when inter-acting with foreign function interfaces. Thus, we as-sume the code written in Rust is trusted except whenusing an unsafe block to call an external library func-tion.

We assume attackers are remote and do not haveroot privileges on the target machine (that is, the ma-chine on which FC operates and is subject to attacks)or any way to interfere with program execution otherthan through the foreign function called by the Rustprogram. We assume the attacker can use memoryvulnerabilities in the unsafe code to access data allo-cated by the trusted program region, for example, us-ing poorly checked memory boundaries. Thus, the at-tacker has access to all memory pages available to theprocess in which the untrusted code executes, exceptmemory pages that are under FC’s protection. Theattacker aims to exploit vulnerabilities in untrustedcode to gain control over the program state in a Rustprogram.

Contributions We present Fidelius Charm2, aRust language library and kernel extension supportfor creating in-memory secure compartments by pro-tecting sensitive data in memory from an unsafe func-tion execution. We discuss the design decisions madefor developing FC, which intends to hide a subset ofthe trusted memory regions allocated while executingRust code. In particular, our work has four primarycontributions:

• Extending Rust’s memory ownership by al-lowing functions to own their allocated data insecure compartments and limit access from unau-thorized functions (Section 2).

2The name Fidelius Charm is inspired by Harry Potter’sFidelius Charm, which is a complex spell to conceal a secret ina person.

• Designing strong kernel-level protection tomaintain the integrity of FC’s secure compart-ments by monitoring and protecting the under-lying APIs, such as mprotect, as well as perform-ing stack inspection to ensure that access can bere-enabled only by the intended safe Rust code(Section 2.4).

• Controlling data sharing between a Rustfunction and a unsafe library function while pro-viding strong isolation of the safe and unsafecode using a narrow and controlled data inter-face (Section 2).

• Testing FC in case studies to explore its effec-tiveness and implementation efforts (Section 3).

FC’s on-demand secure compartments ensurememory protection and isolation within a single pro-cess using architectural support for memory page per-missions. FC’s design is cross-platform and thread-safe, and requires no modification to the Rust com-piler. Our implementation uses a Rust library andkernel extension, both of which are available under anopen source license. Using FC involves mostly simplemodifications to the program’s source code, and lowrun-time overhead (Section 4).

2 Design

To control access to memory allocated in Rust whilecalling an unsafe code, we designed a compartmental-ization technique which splits the address space intothree regions: (i) a private region that is inaccessi-ble from unsafe functions and is fully accessible fromsafe functions, (ii) an immutable region that couldbe read from any part of the program, and (iii) anexposed region, which is accessible from any part ofthe program. The private and the immutable regionscomprise secure compartments, which are collectionsof continuous memory pages with specific permissionbits. These compartments isolate sensitive data fromunsafe code by arranging memory appropriately onpages and changing their permission bits to read-onlyor no access. When an unsafe function executes, de-pending on the program’s policies (specified by theprogrammer), it has limited access to the secure com-partments.

2.1 Motivating Example

Consider developing a TCP server using workerthreads, excerpted in Figure 1. It uses a Worker struc-ture to hold data for a client session, and a Server

Page 2

fn process_traffic(traffic: [u8; TSIZE], worker: &Worker) -> [u8; TSIZE]{ fc_immutable(&traffic); let dec = unsafe { decrypt_traffic(&traffic worker.session_key); } fc_normal(&traffic); dec } fn main() { ... fc_private(&server.key); let dec=decrypt(&traffic, &worker); fc_normal(&server.key); ... }

.tex

t.d

ata

Unsafe library

Rust code

Available memory

FC-ified Rust program

Read

-onl

y

FC

SC

mprotect

User space

Isol

atio

n m

ode

Kernel

FC

Figure 1: FC’s user space and kernel components create and maintain secure compartments in the Rust pro-gram’s process. Access to mprotect calls is restricted to a specific region in the code, which controls the securecompartments. The FC-ified program shows a simplified usage of FC’s interfaces to create a private (no per-mission) secure compartment and an immutable (read-only) secure compartment.Other details of using FC aredescribed in Section 3.

structure to hold data for the main server program.To store client’s message for processing, define an ar-ray, let traffic = [0; TSIZE]. Also, consider aRust function process_traffic that invokes the un-safe external function decrypt_traffic, which de-crypts traffic using Worker.session_key.

This simple example includes several aspects thatmotivate the need for FC. First, when decrypting thetraffic using an unsafe library function, the Rust func-tion only needs to share the client’s session key inWorker.session_key, hiding the server’s private key(Server.key). Second, the Rust function must pro-tect the original copy of the client’s traffic and sendit as a read-only input to decrypt_traffic. Third,when processing a Worker’s client, the Rust functionmust isolate the sensitive data, for example the ses-sion keys of another Worker’s client. Also, the list ofWorkers, and other server-related data stored in aninstance of Server should be secured before execut-ing unsafe code. In Section 3, two case studies showthe of FC on actual Rust-based TLS servers.

Since the external library implemented in an unsafelanguage may have serious security flaws that couldallow arbitrary access to memory [25], any call todecrypt_traffic using unsafe in Rust potentiallyexposes all of program’s memory. FC isolates data ob-jects (i.e., individual variables, referred to as bindingsin Rust) to minimize the exposure when calling unsafecode. For example, when calling decrypt_traffic

to decrypt a client’s message using its session key,only the worker.session_key is exposed to the un-safe code; Server.key and all other sensitive data ob-jects stay in an isolated secure compartment and aretemporarily inaccessible throughout the entire pro-

gram.

2.2 Architecture of FC

FC consists of a user space library, a modified Rustprogram that links to the library, and a kernel modulethat maintains access control for the program’s safe(written in Rust) and unsafe code (arbitrary libraries)regions. Prior to an unsafe call, the programmer addscalls to FC’s user space component, which are inter-faces linked to the Rust program and facilitate creat-ing secure compartments. As shown in Figure 1, theuser memory’s data section is divided into pages thatform secure compartments and exposed pages withread and write access.

The second component of FC is the modified Rustprogram that wraps unsafe calls with invocations ofFC’s functions. For example, in the FC-ified pro-gram of Figure 1, process_traffic creates a read-only secure compartment. The code following a callto fc_immutable changes the program’s state to theisolated mode, in which some of the original memorypage permission bits are modified. A subsequent callto fc_normal reverses the program’s state to exposedmode with all data section page permissions are re-versed to read and write. The main function createsa private secure compartment that hides the server’ssensitive data.

FC’s user space library serves as a client for itsthird component, a kernel module that implements amandatory access control by separating the code sec-tion of the user space into two groups: (i) the codewritten in Rust that has the right to issue mprotect

calls and modify page permissions, and (ii) the code

Page 3

written in arbitrary languages, which cannot makemprotect calls on pages that are tagged as securecompartments. As detailed in Section 2.4, the ker-nel module maintains the secure compartments andmediates access to memory page permissions.

2.3 Secure Compartments

FC enables two memory permission modes for its se-cure compartments: immutable (read-only) and pri-vate (inaccessible). The default memory permissionsare set by the process at the time of allocating mem-ory pages, which FC does not change. The designof FC faces a key challenge to maintain the integrityof the secure compartments within a single virtualaddress space by preventing the unsafe foreign func-tions from accessing the enclosed memory pages. Theproblem is that both the trusted Rust code and theuntrustworthy foreign function are sharing a process,giving them equal operating system-level privilegesfor modifying memory page permissions. A seeminglysimple solution would be to isolate the unsafe code ina separate process. However, this solution requiresnontrivial changes to existing code, and can poten-tially interfere with concurrency in existing Rust pro-grams, which benefit from Rust’s clear and memory-safe concurrency model. Our solution preserves thecurrent concurrency structure of the code while pro-viding an inexpensive mechanism to maintain the in-tegrity of secure compartments, specifically by dis-abling the unsafe code from subverting the policiesset by the Rust code.

Creating and Reversing Secure Compart-ments As shown in the FC-ified code of Figure 1, oneprivate secure compartment holds server.key andan immutable secure compartment holds traffic

and worker.session_key (since both are on thesame memory page, one call to FC will createa shared secure compartment for traffic andworker.session_key).

Creating the secure compartments starts with atrusted Rust function call to fc_immutable(var) (orfc_private(var)) to provide protection for all dataobjects in the memory page where var exists. Inthe example code of Figure 1, this is repeated twicesince the secure compartments must be separated bythe level of access provided to the unsafe code (asecure compartment cannot be both immutable andprivate). First, FC examines the number of allocatedpages, determines the page addresses, and applies theappropriate permissions to the pages (i.e., PROT_READfor immutable and PROT_NONE for private), creatingtwo secure compartments. Next, FC issues a systemcall, sending the kernel a list of page addresses that

will be in the program’s secure compartments (re-gardless of being in immutable or private compart-ments) to deny using mprotect on the specified pages.FC also makes a system call to specify a designatedtrusted region (an address in the code section pointingto a function in FC) in the code, which will be allowed(by the kernel) to make mprotect calls for reversingpage permissions in the secure compartments. Thekernel records the trusted region’s address and pageaddresses in a protected address table (Section 2.4)and monitors requests to modify permissions of theprotected pages.

2.4 Kernel Module

To designate a trusted region for kernel protection,we implemented a Linux kernel module that trapsall mprotect calls (from those processes carrying aspecial signal from FC) and monitors the protectedmemory pages. The reason to use a kernel module isto secure FC’s runtime monitoring within the trustedoperating system without source code modifications.

FC’s kernel module maintains the secure compart-ments and ensures that only the trusted Rust codecan disable the page protections. The kernel exten-sion determines when a call to change page accesspermissions is legitimate based on code regions. Theidea is to designate a specific address range withinthe Rust code at the time FC creates a secure com-partment. The designated address range is communi-cated to the kernel extension, which will subsequentlyonly allow modifications of the secure compartmentsto originate and return to the specified address range.The address range is computed at run time accordingto the fully linked executable.

For security, FC’s kernel extension requires that(i) the loaded code to be immutable across the pro-cess, except by the operating system or the programloader, (ii) an address A in the code section (addressof a function in FC’s code, linked to the Rust pro-gram), which the kernel can trust to allow mprotect

calls to return to, and (iii) the first system call fromthe user space FC, which explicitly asks the kernelto restrict access to mprotect except those returningto A, is trustworthy (done before any unsafe code isexecuting).

Code Section Permissions The page permis-sions for the code section (.text in the linked ELF)must be set to PROT_READ, which is ensured by theRust compiler.3 FC’s kernel extension monitors thepage permissions for the code section and deny all

3As tested with the Rust’s compiler rustc 1.14.0 (e8a0123242016-12-16).

Page 4

mprotect calls to them. This monitoring is only forthe parts of the Rust code, which must be given therights to call mprotect, which is a small subset of theREADONLY .text section that fits in a memory page.(there is no theoretical restriction that this code ex-ceeds one page, but our implementation does not needmore than one).Designating the Trusted Region As described

in Section 2.3, FC’s user space component invokesthe kernel component with a system call (FC reusesexisting system calls to avoid modifying the kernel)before executing an unsafe function. After creat-ing secure compartments, a call to fc_protect(var)

will send the page address on which var exists alongwith the address of the trusted region to the kernel(through a system call). fc_protect(var) computesthe address of the trusted region as a fixed offset rel-ative to the linked address of fc_protect(var) it-self. That is, depending on the number of instruc-tions in fc_protect(var), the offset is manuallycomputed and hard coded in FC’s code and is rel-ative to the address given to fc_protect(var) bythe linker. The offset must only be changed if thelogic of fc_protect(var) was changed.

Designating the trusted region must come from atrusted part of the code, which is code written in Rustand is assumed to have compile-time memory guar-antees. As the programs must always start executionin Rust’s main function, assuming the programmerhas FC-ified the program around all calls to unsafe

foreign functions, the first use of fc_protect(var) istrusted to be from the Rust program. In the exam-ple shown in Figure 2, a new line of code is added tothe body of process_traffic (from Figure 1), whichperforms the trusted call. Upon receiving the call,FC’s kernel module disables mprotect and recordsthe address of the protected page and the trusted re-gion’s address in the protected address table. Thetrusted code region’s address is the address to whichsubsequent mprotect calls on the protected memorypages must return. The call to fc_normal perfromsa system call asking FC’s kernel module to first allowcalls to mprotect and then reverse page permissions.Before re-enabling mprotect, the kernel checks theinstruction pointer of the requesting task to verifythe return address from the call against the recordedaddress in the protected address table. When a ma-licious call to fc_normal is made, the instructionpointer has an invalid address, and FC’s kernel mod-ule’s policy policy is to kill the process (althoughother actions could be used for applications wherefail safety is important). We will further analyze thesecurity of FC’s kernel module and possible attacksin Section 2.5.

2.5 Security Analysis

We examine the security guarantees achieved by FCfrom an attacker’s perspective. According to ourthreat model, the attacker is only capable of a remoteattack, for example by crafting requests to a server.FC’s effectiveness depends on both the attackers goalsand capabilities, and how much data the program ex-poses to vulnerable unsafe components.

Protection Against Data Attacks The maingoal of FC’s design is to thwart attacks that dependon reading or modifying sensitive data in the pro-gram’s memory. The attacker’s gateway to the pro-gram’s memory is through unchecked memory bound-aries in the unsafe external library. Once exploited,the attacker can potentially search through all acces-sible memory to find the target data. The attacker’stask is easier when the calling frame from the trustedregion can be identified (for accessing stack data),and when a reference to memory allocated on heap ispassed to the unsafe function.

First, identifying the calling frame enables the at-tacker to access data allocated in the trusted region.This data is completely protected by FC, except forany data that is on the calling frame’s page. Pro-vided that the programmer does not violate FC’s in-tended usage (by not declaring sensitive data objectsin the calling frame), the attacker cannot manipulateor read data on the stack. Second, when the unsafefunction has a pointer to a memory allocated in theheap, the attacker can identify the region of the mem-ory that is likely to contain sensitive data. As heapis allocated on consecutive memory addresses, the at-tacker can attempt to access pages that may belongto the heap. FC protects memory in the heap as re-quested by the programmer. Thus, all memory pagesthat were designated to move to secure compartmentsare not accessible by the attacker, when executing inisolation mode. One limitation of FC is that theremay be heap memory allocated for libraries that arenot visible to the programmer. It is also importantto note that any data in memory that is used as indi-rect jump location or memory reference is potentiallysensitive; if such references are exposed to the ad-versary, they may be corrupted to allow jumps thatbypass FC protections or to copy sensitive data intolocations that are not protected in a future unsafecall.

Bypassing FC’s protection Bypassing FC in-volves issuing calls to mprotect with a list of ad-dresses to be set to PROT_READ OR PROT_WRITE.First, the attacker is required to identify which mem-ory pages are of interest. Second, a separate call tomprotect is needed for each memory page. An alter-

Page 5

fc_immutable(&traffic); fc_protect(&traffic); let dec = unsafe { decrypt_traffic(&traffic worker.session_key); } fc_normal(&traffic);

Kernel

FCfc_protect

Trusted and immutable

code regionFC-ified Rust program

Protected Address Table

Trusted calls

Figure 2: The trusted region is an address (in the code of FC’s user space library) to which calls from mprotect

must return to, which is ensured by FC’s kernel module. The kernel module traps and monitor all calls tomprotect and maintains a list of page addresses in the protected address table for each thread.

native is that the attacker brute-forces the range ofall virtual memory addresses and sets the protectionfor all pages to PROT_READ OR PROT_WRITE.

FC’s primary line of defense is the kernel-level dis-cretionary access control based on code regions. Asexplained in Section 2.4, FC’s kernel module onlyallows mprotect to succeed on memory pages thatare not in the list of the task’s secure compartments.Also, FC will not allow such calls to succeed if the cur-rent task’s instruction pointer does not indicate theaddress of FC’s library function in the trusted region,which renders the attack unfeasible. There is, how-ever, a possibility of launching a return-oriented pro-gramming attack by chaining a set of gadgets withinthe trusted region to call mprotect and trick the ker-nel to releasing secure compartments. The limitationof this attack is that the only possibility of an exe-cution path is to re-execute the calling trusted func-tion to first release the secure compartments and thenmake a call to the unsafe library function allowing theattacker to continue execution within the unsafe func-tion. Such an attack is not possible as calls to FCshould always create the secure compartments firstand then release them, calling the unsafe function inbetween. Repeating this execution only trigger’s FC’skernel module to be cautious of the process and ter-minate it as this will involve multiple consequent callsto create secure compartments.

3 Case Studies

FC assumes a knowledgeable programmer who usesits core functions to perform the necessary pro-tections. In general, FC’s protections could behighly automated, but our current implementiononly automates stack protection. As automat-ing heap protection involves many low-level de-tails including implementing a custom memory al-

locator, we leave it for future work. We presenthere our experience in FC-ifying a server basedon Rust’s openssl crate (https://github.com/sfackler/rust-openssl); we also FC-ified a similarRust TLS server based on the hyper crate (https://github.com/ctz/hyper-rustls/) but since theexperience and results were similar we only describeopenssl in detail here. We show performance resultsfor both in Section 4.

FC-ifying openssl The openssl crate relies heav-ily on the foreign function interface to fully implementa TLS server based on the functionality provided inthe original openssl library implemented in C. Sim-ilar to hyper, the server using openssl can be FC-ified for protecting the acceptor object when handlinga client.

An acceptor object contains the server’s creden-tials. The acceptor is stored in a Mutex, which is inturn managed by a heap memory under an instanceof Arc. The objective of the FC-ified openssl im-plementation is to secure the heap page that con-tains the server’s private key. For each incomingconnection, at the time the client is served in adedicated thread, the program no longer needs ac-cess to the acceptor. Thus, after the thread re-ceives a pointer to the acceptor (acceptor.clone),locks the Mutex (acceptor.lock), and finally ac-cepts the connection (acceptor.accept(stream)),a call to fc_protect(acceptor_addr) will resultin a secure compartment for the heap page onwhich the acceptor resides. This secures the pri-vate key which will be inaccessible to the un-safe code. The call to fc_auto_stack andthe corresponding call to fc_auto_stack_reverse

will automatically protect the stack pages, andthe call to kernel_disable! and kernel_enable!

(kernel_disable! and kernel_enable! are macrosto facilitate using FC’s interface with the kernelfc_protect as described in Section 2.4) that re-

Page 6

strict access to sys_mprotect. The call to a helpermacro, stack_padding! allocates auxiliary memoryon stack, ensuring that the data allocated prior to thecall remains on a separate virtual memory page.

1 fn openssl_listener() {

2 let acceptor = load_ssl_acceptor();

3 stack_padding!();

4 let acceptor = Arc::new(Mutex::new(acceptor));

5 let acceptor_ptr = acceptor.clone();

6 let acceptor_addr = memory_page_addr!(*acceptor_ptr);

7 let listener = TcpListener::bind(SERVER_IP).unwrap();

8 for stream in listener.incoming() {

9 match stream {

10 Ok(stream) => {

11 let acceptor = acceptor.clone();

12 let child = thread::spawn(move || {

13 let acceptor = acceptor.lock().unwrap();

14 let mut stream = acceptor.accept(stream).unwrap();

15 fc_private_u(acceptor_addr);

16 fc_auto_stack();

17 kernel_disable!(acceptor_addr as usize);

18 handle_client(&mut stream);

19 kernel_enable!();

20 fc_auto_stack_reverse();

21 fc_normal_u(acceptor_addr);

22 });

23 child.join().unwrap();

24 }

25 Err(_) => { /* connection failed */ }

26 }

27 break;

28 }

29 }

Figure 3: Starts a FC-ified server, while protectingthe server’s private key after a new client connectionis established. This function is thread-safe and doesnot cause segmentation fault for accessing the pro-tected acceptor.

Attacks Thwarted FC can be used in variousways depending on the needs. The main goal isto prevent exposing the server’s credentials and theprogram’s state (mainly on stack) when interactingwith untrustworthy clients and executing unsafe code.For complex servers, the handling function may in-clude references to commodity unsafe libraries thatcan jeopardize the security guarantees of the pro-gram, causing arbitrary data leak. FC guaranteesthat the explicitly protected data remains unreach-able when such an attack occurs. FC releases the se-cure compartments, when called through fc_normal,after completing a client processing. This ensures

that the server’s credentials are only accessible whenthe server can make sure a malicious client is nolonger connected. A slight limitation of this imple-mentation is when serving clients for long periods,which can cause long delays for concurrently con-necting clients (as the acceptor object is locked whileserving the client). One can prevent this limitationby generating multiple copies of the acceptor with apool of worker threads model. Each thread would useFC to protect its own copy.

4 Performance

This section reports on our experiments to evaluatethe run-time cost of FC, first showing results on a setof microbenchmarks, and then reporting application-level performance measurements on the openssl andhyper case study applications from Section 3.

4.1 Microbenchmarks

For the microbenchmarks, our goal is to understandthe cost of each of the operations involved in usingFC. We use Rust’s benchmarking interface to measurethe cost for creating secure compartments, launchingand processing a plain openssl request, and launch-ing and processing a FC-ified openssl request, as im-plemented in Figure 3.

Table 1 reports the cost of using FC’s main inter-faces. The Base benchmark is an empty closure formeasuring the benchmarking interface’s cost. ThePadding benchmark is a closure which isolates twomemory pages using 512 × 64 byte arrays. Creat-ing Secure Compartment is a closure that introducespadding, modifies the isolated page’s permissions,communicates the page address to the kernel module,and reverses page permissions and requests the kernelto re-enable access to the specified page address.

The last two rows in Table 1 compare the time re-quire to launch an openssl-based HTTP server andprocessing a single client using a plain Rust imple-mentation and a FC-enabled implementation. Theresults shown are the average over multiples of multi-iteration tests, in which some of the tests did notdistinguish the difference at all. FC’s cost is neg-ligible relative to the overall cost of openssl. Thecost of FC is slightly more noticeable in the systembenchmarks presented next. We performed the testson a vmware Workstation virtual machine on a localdisk with Ubuntu 15 as the host system, configuredto use two of the available four cores and two GBs ofmemory.

Page 7

Experiment Time (ms)Base 0.000024Padding 0.0054Creating Secure Compartment 0.0105openssl server 14.342FC-ified openssl server 14.385 (0.29% increase)

Table 1: Microbenchmark results.

4.2 System Benchmarks

In the second and third settings, we test the per-formance of unsafe operations in TLS-based HTTPservers, which either use unsafe operations for invok-ing openssl operations or make calls to Rust’s ring

library, which in turn makes unsafe calls to crypto-graphic functions. We perform a time comparisonbetween a plain HTTP server that doesn’t use FCand an FC-ified HTTP server. FC’s latency includesthe small overhead of the kernel module, which is dis-abled in experiments without FC.HTTP server Measuring the throughput of a

openssl-based and a hyper-based HTTP server re-quired implementing benchmarking tools for a precisemeasuring of the contribution of each thread in pro-cessing the requests. In the openssl-based server, foreach request three secure compartments are createdprior to handling the process and one secure com-partment is created for handling the process. Wemeasure the throughput by running the test for 60seconds while automatically sending HTTP requestsin intervals of 10 milliseconds, collecting the numberof successfully processed requests at the end of theexperiment. In each iteration, all four secure com-partments are created and destroyed. FC maintainsmodest overhead even when handling 128 simultane-ous request threads, reaching a pick decrease of 5%in the number of requests processed in 60 seconds.FC’s overhead was noticeable when every request in-volved 50 calls to ring for computing a file’s digest,with an average decrease of 13.69% processed requestsoccurred. Finally, when using a duration of 30–100seconds with a fixed number of 16 simultaneous re-quests processed in intervals of 10 milliseconds, theaverage decrease in the number of requests processedwas 8.30% (Figure 4.

In the hyper-based server, the throughput wasmeasured similar to the benchmark of Figure 4 inwhich a growing number of threads send simultane-ous requests. All requests were implemented as echorequests. In the FC-enabled version, during the timethe client is served, the server’s private key is totallyisolated and is unusable. To serve multiple clients,for each request, the private key is cloned and once

30 40 50 60 70 80 90 100Processing interval (secs)

1000

2000

3000

4000

5000

6000

7000

Req

uest

s pr

oces

sed

Multithreaded HTTPS throughput with 50 digests

RustFC

Figure 4: Throughput of an openssl HTTP in 30–100 seconds. Each request has 50 calls to ring’s di-gest. In the FC-enabled server, each digest iterationcreates and releases a secure compartment.

the shared key is established, the cloned private keywill be kept in a separate secure compartment. Theresult of the experiment is in Figure 5, showing anaverage decrease of 1.38% in the number of requestsprocessed.

1 2 4 8 16 32 64 128Number of clients

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

Req

uest

s pr

oces

sed

104 Hyper+TLS throughput

RustFC

Figure 5: Throughput of a Hyper server using RustTLS in 60 seconds. Requests are sent locally using 2k

threads for k ∈ [0,7].

5 Related Work

The goal of isolating code components (also referredto as application compartmentalization) is a long-standing goal of our community and has been thefocus of extensive research in operating systems, pro-gramming languages, and architecture. This is thefirst work to focus on isolating unsafe code within asingle address space for Rust programs.

In terms of our memory model and isolation tech-niques, previous works most similar to FC include Na-tive Client [30], Codejail [29], HideM [9], SeCage [15],Shred [7], lwC [14], and SpaceJMP [12]. We discussthese next, followed by a brief account of classical

Page 8

work on software fault isolation and the principle ofleast privilege.

Sandboxing Libraries Native Client [30] pro-vided memory sandboxing for libraries, allowing lim-ited interaction with the trusted program using re-mote procedure calls. Fidelis Charm and NativeClient (NaCI) share the main objective of limitingexternal libraries, however, differ in approach andapplicability. In contrast with NaCI’s approach forloading libraries in limited containers, FC containsthe trusted memory when interacting with unsafe li-braries. Codejail [29] proposed an enhancement ofthe idea by sandboxing a library by disallowing writeaccess to the program’s data, making the sensitivememory read-only. The program would selectivelyallow write access to its data, when a tight inter-action with the library is needed. Codejail sharesFC’s goals in (i) proposing a secure memory shar-ing model that does not require modifications to thelibrary and (ii) supporting tight and limited interac-tions with an untrustworthy library. Aside from fo-cusing on Rust, FC is distinguished in that the mem-ory of the trusted program is the sandbox, instead ofthe library, and unless a data object must be sharedwith the library, the entire memory allocated eitheron stack or heap of the trusted Rust code is inacces-sible to the library. This key difference in memorysandboxing model overcomes Codejail’s limitation oflibraries within a specific memory regions.

Various techniques have been developed for confin-ing and limiting processes or groups of processes (e.g.,[19, 8, 1, 13, 10, 23, 18]), aiming for isolating vulner-able software from critical system resources. In thedesign of FC, a sandbox would avoid incuring unnec-essary latency as our goal is to isolate code at the finelevel of (often frequent) unsafe calls. That said, sand-boxing using Intel enclaves [13] can provide improvedsecurity for FC’s kernel module.

Memory partitioning Shared memory amongdistrustful threads within a single process is ad-dressed by Arbiter [27], which proposes to use mem-ory permission bits to protect a thread’s data fromanother. Arbiter uses to a policy manager that un-derstands programmer-annotated privileges and en-forces them. FC uses memory page padding to sep-arate and isolate data objects based on a simple bi-nary permission system (immutable or private). Thekernel component in FC only enforces FC’s integrityand does not need to enforce application-level poli-cies. HideM [9] takes a radical approach by usingsplit-TCB to show different contents of a page for dif-ferent CPU operations, hiding data when a specifiedcode region should not have access to it. In contrast,FC temporarily hides the actual memory page and

isolates the data when an explicit interaction with anuntrustworthy library function occurs. SeCage [15]is similar to HideM in providing different views ofthe memory according to separated privileges. Us-ing hardware virtualization, SeCage targets a strongadversary model in which the operating system doesnot need to be trusted.

Shred [7], light-weight context (lwC) [14], andSpaceJMP [12] are new methods for splitting the vir-tual address space into multiple distinct sections aim-ing for isolating untrustworthy code. Shreds and lwCintroduce abstractions similar to threads. To protectdata across a program, Shred provides a set of pro-gramming interfaces to request creating and movingdata into separate Shreds. At runtime, Shreds areimplemented using Intel’s memory protection keys.A lwC, in contrast, does not work with memory per-missions directly but creates separate address spaces,when the programmer requests that using the pro-gramming interfaces. SpaceJMP is similar to lwCs inusing multiple virtual address space, differing fromlwCs in that SpaceJMP enables memory sharingacross multiple processes. While FC is similar tothese in that it provides an interface for isolatingmemory regions, it does not propose a new operatingsystem model and does not require switches betweenaddress spaces.

Other related work but farther away have pro-vided interesting contributions for application com-partmentalization, mainly with narrow and specificapplicability. For example, Mimosa specifically tar-gets cryptographic keys and uses hardware transac-tional memory to ensure that no process, other thanMimosa, can access the keys. Mimosa uses encryp-tion to hide the keys, when the system is idle. Al-though FC and Mimosa agree on a high level goal ofhiding data objects in memory, FC differs in that ituses memory isolation without the need for hardwaretransactional memory and works with arbitrary data.DataShield [5] protects data in C++ programs by dis-allowing pointer dereferencing based on programmerannotations. Song et al. propose a data-flow integrityapproach to infer and enforce correct flow of sensitivedata in kernel space [24]. Lastly, SOAAP [11] is areasoning tool for assisting programmers in using ap-plication compartmentalization to avoid security andcorrectness errors. We envision a similar tool for ourfuture work to support programmers with FC andautomate the task of locating unsafe regions and thedata objects that must be isolated.Software fault isolation has consistently re-

ceived attention during the past decades. Simple so-lutions such as placing the faulty code, or the un-safe code, in a separate address space seem viable, al-

Page 9

though for merely a call to an unsafe function, the un-necessary context switches are too much of a burden.Wahbe et al. pioneered the design of logically sepa-rated fault domains within a single address space [26],which was followed by an effort to isolate addressabil-ity from accessibility in Opal, a single-address-space64-bit architecture [6]. The work in [26] describeda model in which fault domains are separated basedon the code region through restricting the executionof one to jump to another. This model inspired ourkernel-centric code region discrimination design, witha fundamental difference; FC would not require anRPC interface to enable cross-domain interactions. Infact, FC imposes no particular paradigm on the pro-gram and automates code region separation througha kernel extension.

The principle of least privilege [22] is thetheme of a number of previous work that promisedleast privilege isolation. With resource contain-ers [2] separating access control from execution, isola-tion progressed further towards decoupling schedulingfrom security requirements, which were a fundamen-tal design issue with process management in modernmonolithic kernels. The work by Provos et. al seta clear goal: privilege separation within an applica-tion forbids programming errors in the lower privi-leged code from abusing higher privileged code [20].However, privilege separation was a return back tothe use of processes as basic blocks for isolation,reusing UNIX per-process protection domains. Priv-trans [4] automated privilege separation using pro-grammer annotations, partitioning a program into amonitor and a slave program, continuing the efforts ofisolation at the process level. Sthreads in Wedge [3]introduced default-deny compartments within a sin-gle monolithic program, spawning new threads, notentire processes, for isolating parts of the program.Sthreads enjoy programmer tagged memory accessrights, which are enforced at runtime. Programmerannotated privileges were also introduced for isolat-ing kernel modules [17] from core kernel services toprevent privilege escalation. Similarly, Trellis [16] al-lows code-annotated privileges (mainly for memoryallocations), which are enforced by the kernel at run-time. CHERI [28], a hardware extension relying on acapability co-processor, supports compartmentaliza-tion for in-address-space memory isolation targetingthe C language.

Finally, RustBelt [21] develops a subet of Rust,namely λRust, and uses to prove the safety of Rustprograms. An important result provided by RustBeltis verification of safety while using unsafe interactionswith linked libraries. Rustbelt verifies a λRust pro-gram with unsafe code has safely encapsulated the

externally linked library using within Rust wrappers.

6 Conclusion

Rust provides strong memory guarantees using zero-cost abstractions, but any non-trivial Rust programtoday includes unsafe code and most fall back on us-ing libraries in unsafe languages. A long-range goalshould be to eliminate the need for any unsafe code—developing native Rust libraries when possible, andwhen arbitrary memory operations are needed usingmore powerful formal methods to prove the safetyof code that cannot be proven safe by Rust’s com-piler. A practical path to dramatic improvements inprogram safety and reliability, however, requires com-bining safe and unsafe code. Incorporating any unsafecode into a Rust program, however, abandons all ofthe safety guarantees. Fidelius Charm provides a steptowards safe incorporation of unsafe code by isolatingsensitive data in memory from the unsafe code. Weachieved a high level of isolation, without requiringany compiler changes or complex abstractions, andin a way that can be applied to any Rust programwhen interacting with any unsafe library function.

References

[1] H. M. J. Almohri, D. Yao, and D. G. Kafura.Process authentication for high system assur-ance. IEEE Trans. Dependable Secur. Comput.,11(2):168–180, Mar. 2014.

[2] G. Banga, P. Druschel, and J. C. Mogul. Re-source containers: A new facility for resourcemanagement in server systems. In Proceedingsof the Third Symposium on Operating SystemsDesign and Implementation, pages 45–58, 1999.

[3] A. Bittau, P. Marchenko, M. Handley, andB. Karp. Wedge: Splitting applications intoreduced-privilege compartments. In Proceedingsof the 5th USENIX Symposium on NetworkedSystems Design and Implementation, NSDI’08,pages 309–322, 2008.

[4] D. Brumley and D. X. Song. Privtrans: Au-tomatically partitioning programs for privilegeseparation. In USENIX Security Symposium,2004.

[5] S. A. Carr and M. Payer. Datashield: Config-urable data confidentiality and integrity. In Pro-ceedings of the 2017 ACM Asia Conference onComputer and Communications Security, ASIA

Page 10

CCS ’17, pages 193–204, New York, NY, USA,2017. ACM.

[6] J. S. Chase, H. M. Levy, M. J. Feeley, and E. D.Lazowska. Sharing and protection in a single-address-space operating system. ACM Trans.Comput. Syst., 12:271–307, 1994.

[7] Y. Chen, S. Reymondjohnson, Z. Sun, and L. Lu.Shreds: Fine-grained execution units with pri-vate memory. In Proceedings of the 2016 IEEESymposium on Security and Privacy (SP), pages56–71, May 2016.

[8] B. Ford and R. Cox. Vx32: Lightweight user-level sandboxing on the x86. In USENIX 2008Annual Technical Conference, ATC’08, pages293–306, Berkeley, CA, USA, 2008. USENIX As-sociation.

[9] J. Gionta, W. Enck, and P. Ning. Hidem: Pro-tecting the contents of userspace memory in theface of disclosure vulnerabilities. In Proceed-ings of the 5th ACM Conference on Data andApplication Security and Privacy, CODASPY’15, pages 325–336, New York, NY, USA, 2015.ACM.

[10] A. Gollamudi and S. Chong. Automatic en-forcement of expressive security policies usingenclaves. In Proceedings of the 2016 ACMSIGPLAN International Conference on Object-Oriented Programming, Systems, Languages,and Applications, OOPSLA 2016, pages 494–513, New York, NY, USA, 2016. ACM.

[11] K. Gudka, R. N. M. Watson, J. Anderson,D. Chisnall, B. Davis, B. Laurie, I. Marinos,P. G. Neumann, and A. Richardson. Clean ap-plication compartmentalization with soaap. InACM Conference on Computer and Communi-cations Security, 2015.

[12] I. E. Hajj, A. Merritt, G. Zellweger, D. S. Milo-jicic, R. Achermann, P. Faraboschi, W. meiW. Hwu, T. Roscoe, and K. Schwan. Space-jmp: Programming with multiple virtual addressspaces. In ASPLOS, 2016.

[13] D. Kuvaiskii, O. Oleksenko, S. Arnautov,B. Trach, P. Bhatotia, P. Felber, and C. Fetzer.Sgxbounds: Memory safety for shielded execu-tion. In EuroSys, 2017.

[14] J. Litton, A. Vahldiek-Oberwagner, E. Elnikety,D. Garg, B. Bhattacharjee, and P. Druschel.Light-weight contexts: An os abstraction for

safety and performance. In Proceedings of the12th USENIX Conference on Operating SystemsDesign and Implementation, OSDI’16, pages 49–64, Berkeley, CA, USA, 2016. USENIX Associa-tion.

[15] Y. Liu, T. Zhou, K. Chen, H. Chen, andY. Xia. Thwarting memory disclosure with effi-cient hypervisor-enforced intra-domain isolation.In ACM Conference on Computer and Commu-nications Security, 2015.

[16] A. Mambretti, K. Onarlioglu, C. Mulliner,W. Robertson, E. Kirda, F. Maggi, andS. Zanero. Trellis: Privilege separation for multi-user applications made easy. In InternationalSymposium on Research in Attacks, Intrusions,and Defenses, pages 437–456. Springer, 2016.

[17] Y. Mao, H. Chen, D. Zhou, X. Wang, N. Zel-dovich, and M. F. Kaashoek. Software faultisolation with api integrity and multi-principalmodules. In SOSP, 2011.

[18] E. Pattuk, M. Kantarcioglu, Z. Lin, and H. Ulu-soy. Preventing cryptographic key leakage incloud virtual machines. In USENIX SecuritySymposium, 2014.

[19] D. S. Peterson, M. Bishop, and R. Pandey. Aflexible containment mechanism for executinguntrusted code. In USENIX Security Sympo-sium, 2002.

[20] N. Provos, M. Friedl, and P. Honeyman. Pre-venting privilege escalation. In USENIX SecuritySymposium, 2003.

[21] R. K. D. D. Ralf Jung, Jacques-Henri Jourdan.RustBelt: Securing the foundations of the rustprogramming language. In Proceedings of the45th ACM SIGPLAN Symposium on Principlesof Programming Languages, POPL 2018, NewYork, NY, USA, 2018. ACM.

[22] J. H. Saltier and M. P. Schroeder. The protectionof information in computer systems. IEEE CSITNewsletter, 3(12):19–19, Dec 1975.

[23] R. Sinha, M. Costa, A. Lal, N. P. Lopes, S. K.Rajamani, S. A. Seshia, and K. Vaswani. A de-sign and verification methodology for secure iso-lated regions. In PLDI, 2016.

[24] C. Song, B. Lee, K. Lu, W. Harris, T. Kim, andW. Lee. Enforcing kernel security invariants withdata flow integrity. In NDSS, 2016.

Page 11

[25] L. Szekeres, M. Payer, T. Wei, and D. X. Song.Sok: Eternal war in memory. In Proceedingsof the 2013 IEEE Symposium on Security andPrivacy, SP ’13, pages 48–62, Washington, DC,USA, 2013. IEEE Computer Society.

[26] R. Wahbe, S. Lucco, T. E. Anderson, and S. L.Graham. Efficient software-based fault isolation.In SOSP, 1993.

[27] J. Wang, X. Xiong, and P. Liu. Between mu-tual trust and mutual distrust: Practical fine-grained privilege separation in multithreaded ap-plications. In USENIX Annual Technical Con-ference, 2015.

[28] R. N. M. Watson, J. Woodruff, P. G. Neumann,S. W. Moore, J. Anderson, D. Chisnall, N. H.Dave, B. Davis, K. Gudka, B. Laurie, S. J.Murdoch, R. Norton, M. Roe, S. D. Son, andM. Vadera. Cheri: A hybrid capability-systemarchitecture for scalable software compartmen-talization. 2015 IEEE Symposium on Securityand Privacy, pages 20–37, 2015.

[29] Y. Wu, S. Sathyanarayan, R. H. C. Yap, andZ. Liang. Codejail: Application-transparent iso-lation of libraries with tight program interac-tions. In ESORICS, 2012.

[30] B. Yee, D. Sehr, G. Dardyk, J. B. Chen,R. Muth, T. Ormandy, S. Okasaka, N. Narula,and N. Fullagar. Native client: A sandboxfor portable, untrusted x86 native code. 200930th IEEE Symposium on Security and Privacy,pages 79–93, 2009.

Page 12