Experimental Computer Systems Lab A Binary Rewriting Defense Against Stack-based Buffer Overflow Attacks Manish Prasad, Tzi-cker Chiueh SUNY Stony Brook

Experimental Computer Systems Lab

A Binary Rewriting Defense Against Stack-based Buffer Overflow Attacks

Manish Prasad, Tzi-cker ChiuehSUNY Stony Brook


Roadmap Binary Translation Buffer Overflow and Return

Address Defense (RAD) Static BT and RAD Experimental Results Conclusion


Binary Translation Motivation

To do things without access to source Traditional Applications

Legacy Code Migration (HP Aries, UQBT) Program Optimization (Etch, IBM BOA)

Approaches (and their limitations) Static – Lacks transparency, accuracy Dynamic – Less efficient


Binary Translation and Software Security Legacy applications from outside

vendor


Binary Translation and Software Security

Application AVendor X

NO SOURCE CODE

CERT Advisory


VULNERABILITY !!!


Binary Translation and Software Security

CERT Advisory


VULNERABILITY !!!


Has it been done yet ?? Dynamic Translation

DynamoRIO [MIT] LibVerify [Bell-Labs]

Hardware Support [UIUC]


Why Static Binary Translation (SBT) ? Inferring legacy program behavior

Application-specific security policies Use Static Analysis if you can

Butler Lampson, Hints for Computer System Design


Goals How far can we go with pure static

BT ? Buffer Overflow protection

Widespread Simple yet covers most SBT issues

Foundation for general Win32/PE instrumentation framework


Contributions Comprehensive Treatment of Static BT

Why and where it fails ? Prototype implementation incorporating

static BT state-of-the-art High disassembly precision Exhaustive experiments with several

commercial grade Windows applications


Buffer Overflow Attack 0

4G

StackGrowth

Stack Evolution on a Function Call

Buffer Overflow and RAD


Buffer Overflow Attack

Function Arguments

caller



0

4G

StackGrowth



Function Arguments

Return Address CALL



0

4G

StackGrowth



Function Arguments

Return Address

Old Frame Pointer

Local Variables

Local Buffer

Other Local Variables

callee



0

4G

StackGrowth



Function Arguments

Return Address

Old Frame Pointer

Local Variables

Local Buffer


target



0

4G

StackGrowth



Function Arguments

Return Address

Old Frame Pointer

Local Variables

Local Buffer


A

A

A

A

Exp

AttackBuffer

A = addr of exploitExp = Exploit

Code

Unbounded Buffer Copy


0

4G

StackGrowth



Exploit Code

Address of Exploit

Address of Exploit

Address of Exploit

Address of Exploit


return

Overflow Aftermath


0

4G

StackGrowth


Return Address Defense (RAD)

Argumentscaller

Return Addr CALL

Stack

Ret Addr Copy

Return Address Repository(RAR)

PrologueReturn Address Save




Arguments

Return Addr

Stack

Ret Addr Copy

Return Address Repository

LocalVariableSpace

callee




Ret addr corrupted

Stack

Ret Addr Copy


AddressOf

Exploit Code

UnsafeBufferCopy




Ret addr corrupted

Stack

Ret Addr Copy


EpilogueReturn Address Check




Ret addr corrupted

Stack

Ret Addr Copy


Attack !!



Static BT & RAD – Central Issues Disassembly Code Instrumentation

Static BT and RAD

Binary File

Disassembler Instrumentation


Disassembly Core component for static analysis

of binaries Principal Approaches

Linear Sweep Recursive Traversal

Static BT and RAD





First Byte valid inst

valid inst

valid inst

valid inst

invalid inst

Static BT and RAD





Entry Point

CALL fn

fn

RET

Static BT and RAD




Linear Sweep Recursive Traversal RET

Entry Point

CALL fn

fn

RET

Static BT and RAD




Recursive Traversal Linear Sweep RET

Entry Point

CALL fn

fn

RET

Static BT and RAD


Disassembly – Impediments Code/Data distinction Variable x86 instruction size Indirect Branches Functions without explicit CALL PIC

Static BT and RAD



0x0F 0x85 0xC0 0x0F 0x85 ….

JNE offset

0x0F // dataTEST eax, eax

JNE offset

0x0F 0x85 0xC0 0x0F 0x85 0x0F 0x0F 0x85 …..

Static BT and RAD



0x0F 0x85 0xC0 0x0F 0x85 ….

JNE offset

0x0F // dataTEST eax, eax

JNE offset

0x0F 0x85 0xC0 0x0F 0x85 0x0F 0x0F 0x85 …..

Static BT and RAD


Disassembly – Impediments Code/Data distinction Variable x86 instruction size Indirect Branches Functions without explicit

CALL PIC

Static BT and RAD


Disassembler – Approach Recursive Traversal

Accurately code/data identification Linear Sweep

Reach areas not covered by recursive traversal

Compiler-independent heuristics Recover from errors

Static BT and RAD


Disassembly Accuracy

Static BT and RAD


Code Instrumentation Add desired functionality Preserve original program

semantics

Static BT and RAD


Code Instrumentation – RAD Add buffer overflow protection Preserve original program

semantics

Static BT and RAD


What to instrument ? Function boundary identification

Prologue and epilogue must be both instrumented/uninstrumented

False alarms Pattern matching (for ‘interesting’

functions) Stack frame allocation and

deallocation

Static BT and RAD


What to instrument ?

Binary File

DisassemblerCore Binary

Rewrite Engine

RAD

Each Instruction If ‘interesting’ function

Prototype


Inserting Checking Code JMP at prologue and epilogue to RAD

code Replace 5 byte worth instructions Should not disturb branch targets Stack frame allocation (at prologue) >=

5 bytes Stack frame deallocation possible in 2 – 4

bytes Return address check in INT 3 handler

Static BT and RAD

Prologue

Epilogue

JMP (save ret addr)

JMP (check ret addr)


Inserting Checking Code JMP at prologue and epilogue to RAD

code Replace 5 byte worth instructions Should not disturb branch targets Stack frame allocation (at prologue) >=

5 bytes Stack frame deallocation possible in 2 – 4

bytes Return address check in INT 3 handler

PUSH EBP // 1 byte

MOV EBP, ESP // 2 byte

SUB ESP, x // 3-6 bytes

ADD ESP, x // 3-6 bytesPOP EBP // 1 byte

RET // 1 byte

MOV ESP, EBP // 2 bytesPOP EBP // 1 byte

RET // 1 byte

LEAVE // 1 byteRET // 1 byte

INT 3 // 1 byte

Static BT and RAD


INT 3 Statistics

Static BT and RAD


Experimental Evaluation Goals of experiments

Effect on program correctness Resilience to buffer overflow attacks Performance and space overhead

Experimental Evaluation


Space Overhead

Micro-Benchmark

Macro-Benchmark



Execution Time Overhead

Overhead = Execution Time with RAD – Execution Time without RADExecution Time without RAD



Resilience to Buffer Overflow Attack Windows Help (Winhlp32.exe) Windows NT 4.0 with Service Pack 4.0 Content file (.CNT) with long heading

string Published exploit code

Resists Attack !!



Known Limitations Disassembly Limitations Hand-crafted Assembly RAD Limitations Multi-Threaded Applications Self-Modifying Code


Known Limitations Disassembly Limitations

Indirect branches Hand-crafted Assembly RAD Limitations Multi-Threaded Applications Self-Modifying Code


Indirect Branches and Control Flow Analysis


Known Limitations Disassembly Limitations Hand-crafted Assembly

Inter-procedural jumps, multiple function entry points

RAD Limitations Multi-Threaded Applications Self-Modifying Code


Known Limitations Disassembly Limitations Hand-crafted Assembly RAD Limitations

Memory Pointer corruption Multi-Threaded Applications Self-Modifying Code


Known Limitations Disassembly Limitations Hand-crafted Assembly RAD Limitations Multi-Threaded Applications

Per-thread RAR needed Self-Modifying Code


Known Limitations Disassembly Limitations Hand-crafted Assembly RAD Limitations Multi-Threaded Applications Self-Modifying Code

Beyond static analysis


Conclusions First study to implement Static BT

state-of-the-art into a working system High disassembly accuracy Demonstrated effectiveness on

commercial grade Windows applications

Qualified success as a security tool Basis for a general Win32/PE

Instrumentation framework


Looking Ahead Support for DLLs Dynamic translation to ameliorate

static BT deficiencies Automatic application-specific

security policy generation Copy and Tamper resistant

software


Project Page http://www.ecsl.cs.sunysb.edu/brew.htm Disassembler Sources

http://www.ecsl.cs.sunysb.edu/disassembler.tgz


Questions ??


… and I’m looking for work !!!


Disassembler – Passes DATA

DATA

DATA

:

:

CALL 0x40011C

JMP 0x400110

:

RET

MOV EAX, 0x40012E; PUSH 0x400100

CALL EAX

RET

MOV EAX, ECX

CALL 0X400140

0x400100

0x400110

0x40011C

RET

0x40012E

Static BT and RAD




DATA

DATA

:

:

CALL 0x40011C

JMP 0x400110

:

RET


CALL EAX

RET

MOV EAX, ECX

CALL 0X400140

0x400100

0x400110

0x40011C

RET

0x40012E

Pass 1RecursiveTraversal

Entry Point

Inter-ProceduralCall Graph

Static BT and RAD



DATA

DATA

:

:

CALL 0x40011C

JMP 0x400110

:

RET


CALL EAX

RET

MOV EAX, ECX

CALL 0X400140

0x400100

0x400110

0x40011C

RET

0x40012E


Entry Point

Inter-ProceduralCall Graph

FunctionBody

Function ControlFlow Graph

BackwardEdge

Static BT and RAD



DATA

DATA

:

:

CALL 0x40011C

JMP 0x400110

:

RET


CALL EAX

RET

MOV EAX, ECX

CALL 0X400140

0x400100

0x400110

0x40011C

RET

0x40012E


Entry Point

FunctionBody

Static BT and RAD


DATA

DATA

DATA

:

:

CALL 0x40011C

JMP 0x400110

:

RET


CALL EAX

RET

MOV EAX, ECX

CALL 0X400140

0x400100

0x400110

0x40011C

RET

0x40012E

Pass 2Linear Sweep

FunctionBody

Legal inst.Bytes as code

Disassembler – Passes

Static BT and RAD



DATA

DATA

:

:

CALL 0x40011C

JMP 0x400110

:

RET


CALL EAX

RET

MOV EAX, ECX

CALL 0X400140

0x400100

0x400110

0x40011C

RET

0x40012E

Pass 3Error RecoveryHeuristics

FunctionBody

Code sequencenot ending withJMP/RET=> not code

Static BT and RAD


Execution Time Micro-benchmark


Penalty = Additional RAD Overhead Original Run Time


WattProbe Linux-based experimental platform for

low-power computing Power vs. Performance measurements

Fine Grained High Resolution Across Multiple Resources

Power characterization of interactive applications Application-specific low power modes


Caching Infrastructure for Stackable File Systems (FAST ’03 WIP)

Cache coherence support Transparent to native and stackable

file systems Minimal changes to the page cache

code Hooks into a cache manager

Prototype Linux implementation


Communication and Membership Management for Linux Clusters

Low-latency link-level semi-reliable transport layer (Springer LNCS, HiPC ’02) No hardware support

Membership Management Cluster-wide consistent membership

view Scalable failure detection


Thanks




Disassembly Limitations False Negatives (Functions Missed) False Positives (Falsely Identified

Functions)

Prototype


Disassembly Limitations False Negatives (Functions Missed)

Partly/fully misidentified as data Identified fully as code

False Positives (Falsely Identified Functions Prologue

Epilogue

Data

FunctionBody

Uninstrumented

Prologue

Identified as Data

Instrumented

Security HoleRAR Overflow

Prototype



Partly/fully misidentified as data Identified fully as code

False Positives (Falsely Identified Functions Prologue

Epilogue

Identified as Data

Fn1

Fn2

InstrumentedRAR overflow

UninstrumentedSecurity Hole

Prototype



Partly/fully misidentified as data (pop up)

Identified fully as code False Positives (Falsely Identified

Functions)Data

Interesting Prolog

Function Body

Identified as CodeIdentified as CodeSecurity Hole

Prototype


Disassembly Limitations False Negatives (Functions Missed) False Positives (Falsely Identified

Functions) Code following

data/unconditional branch Missed attacks RAR overflow

Indirect Jump

Data

Jump TargetFunction Entry Pt

Prototype


Indirect Branches and Control Flow Analysis

Prototype


RAD Limitations Attacks resisted

Return Address Frame Pointer

Does not protect Memory pointer attacks

Function pointer corruption Import table overwrites

Prototype


Multi-Threaded Applications RAR shared between threads Possible Solution

Thread Information Block (TIB) access using FS register

Thread Local Storage (TLS) slots in TIB Separate RAR for each thread Thread-specific RAR addresses in TLS

slots

Prototype


Hand-Crafted Assembly Quirks

No interesting prolog

No interesting epilog

Interesting prolog

Interesting epilog

Fn 1

Fn 2

JNE label

label

Inter-procedural jump in MSAccess

Fn1 enteredReturn address not saved

Exit from Fn2Return address check made

False Alarm

Prototype


Software Architecture & Flow of Control

Binary File

DisassemblerCore Binary

Rewrite Engine

RADPE-specificcomponent

Initialization

New PESection

Set up RAD mine zones

Prototype

Instrumentation

Documents

Experimental Computer Systems Lab A Binary Rewriting Defense Against Stack-based Buffer Overflow Attacks Manish Prasad, Tzi-cker Chiueh SUNY Stony Brook