ProbeDroid: Crafting Your Own

Dynamic InstrumentTool on Android

for App Behavior Exploration

ZongXian Shen

About Me

www.zsshen.org

• Passionate Security Researcher and Developer

• Earned Master in CS from NCTU, Taiwan

• Involved in DSNS Lab

andy.zsshen@gmail.com

ZSShen

@AndyZSShen

Disclaimer

• A research project run in my spare time

• Still a lot of features to enhance

• Slide is the design memo and hack notes

Outline

• Intro to dynamic binary instrumentation

• Intro to ProbeDroid instrument kit

• Design and implementation of ProbeDroid

• App analysis with custom instrument tools

Intro to Dynamic Binary Instrumentation

Dynamic Binary Instrumentation

• Technique that inserts code into a process to get

runtime information or change process behavior

without modifying original program binary

Definition Source: Intel PIN manual

DBI Application

• Performance tuning and bug hunting

• Cache performance, memory access footprint

• Behavior tracing and data logging

• API call sequence, code block control flow relation

• Changing process behavior on the fly

• Execution path exploration, product hacking

DBI for Android App

• Current Android app is mainly coded with Java

and run on custom Java runtime named ART

• Due to semantic gap, prefer Java level DBI rather

than instrumentation directly on native ISA

• Class field and object content inspection

• Method call sequence and parameter profiling

• Changing Java artifacts on the fly

Inspiring Work

• ADBI and DDI introduced by Collin Mulliner

• https://github.com/crmulliner/adbi

• https://github.com/crmulliner/ddi

• DBI framework based on Dalvik runtime

• Demonstrate how to hook interested Java method

and manipulate class field

And now ProbeDroid, targeting on ART

runtime with enhanced user interface

Intro to ProbeDroidInstrument Kit

ProbeDroid Instrument Kit

• Programmable instrumentation

• Code your own instrument tools with Java practice

• Flexible APIs for you to

• Hook interested library or app defined methods

• Customize instrument gadgets for different analysis purposes

• Modify method in/output to hack app at runtime

• Succinct deployment

• Only ProbeDroid engine and instrument tools are required

• No need to customize Android framework

https://github.com/ZSShen/ProbeDroid

ProbeDroid Usage

• Source building

• Compile launcher, engine, and exported jar

• Import jar to Android Studio project for tool creation

• Play and hack

• Push launcher, engine and tool to experiment device

• Run launcher to inject engine to target app

ProbeDroid Project Wiki

Sample Tool

Signature of to be

instrumented method

Create custom

instrument gadget

Register the gadget to ProbeDroid engine

Sample Tool

• Manipulate the trapped input parameters

• Do hacks before entering the hooked method

• Manipulate the trapped return value

• Do hacks after leaving the hooked method

• Be injected into the target app process

• Work as a mini runtime

• Load and execute the instrument tool

• Hook the specified methods and install gadgets

• Marshal control flow for hooked methods and gadgets

ProbeDroid Tasks

Currently support X86 & ARM ISAs and Lollipop 5.0

ProbeDroid Design and Implementation

ProbeDroid Overview

Inject Engine Library

Launcher Run

Library Inited

Load Instrument Tool

Get Runtime Utilities

Hook Specified Methods

Method Entered

Call Pre-Method Gadget

Call Original Method

Call Post-Method Gadget

3. Play Music

2. Compose Gadget

1. Deploy Engine1 & 2 Before App Execution

3 During App Execution

Stage: Deploy Engine

libART

Android & Java

Base Classes

libProbeDroid

Launcher

App Process

libART

Android & Java

Base Classes

Zygote Process

Attach and

Wait for child

Control the

process

Fork process

• Library injection

• Catch the newly forked

app process

• Force the app process

to load engine library

Engine binaryLoad library

Library Injection

• No convenient APIs like WriteProcessMemory()

and CreateRemoteThread() dedicated for Windows

• Manually crafting ptrace() operation sequence to

simulate the procedure on Android

Injection Procedure

1. Attach to Zygote and wait for target app to be forked

2. Attach to app process and release Zygote

3. Resolve the address of mmap() and dlopen() in app process

4. Force app to execute mmap() for stashing library pathname

5. Force app to execute dlopen() for loading engine library

Known technique with proper modification

Resolve Function Address

/system/lib/libc.so

… …

… …

mmap

… …

/system/bin/linker

… …

… …

dlopen

… …

• Resolve /proc/pid/maps for library base address in both processes

• Use dlopen() and dlsym() for symbol address in launcher process

• Use relative offset to resolve symbol address in app process

base_linker

Launcher Process

addr_dlopen

base_libc

addr_mmap

/system/lib/libc.so

… …

… …

mmap

… …

/system/bin/linker

… …

… …

dlopen

… …

App Process

(addr_dlopen-

base_linker)+

base_linker’

base_linker’’

base_libc’’

(addr_mmap-

base_libc)+

base_libc’’

Remote Function Call

0

0

4096

PROT_READ|WRITE

MAP_ANONYMOUS

0

0

base_stack

Invalid return

address

esp base_stack

eip mmap()

eax addr_libpath

0

4096

PROT_READ|WRITE

MAP_ANONYMOUS

0

0

base_stack

ARM_sp base_stack

ARM_pc mmap()

ARM_r0 addr_libpath

ARM_lr 0Invalid return

address

0

addr_libpath

RTLD_NOW

Invalid return

address

base_stack

esp base_stack

eip dlopen()

eax handle_lib

ARM_lr 0

addr_libpath

RTLD_NOW

Invalid return

address

base_stack

ARM_sp base_stack

ARM_pc dlopen()

ARM_r0 handle_lib

• Invoke mmap() to allocate space for library path

• Invoke dlopen() to load the specified library

Force SIGSEGV

for wait()

Stage: Compose Gadget

libART

Android & Java

Base Classes

App Classes

Instrument

Classes

libProbeDroid

Instrument

Tool APK

App Oat

App Process

• Bootstrapping

• Acquire ART context to

start instrument world

• Hooks installation

• Load instrument tool

• Modify ART artifacts to

hook interested methods

Bootstrapping

• JNI exports helpful utilities for native code to

access Java runtime features

• ProbeDroid must acquire JNI interface pointer

for all the instrument related tasks

JNI Interface Pointer

jint JNI_GetCreatedJavaVMs

(JavaVM** vms, jsize size, jsize* vm_count);

Return Java vm instance

pointer of current runtime

• First retrieve JavaVM pointer and stash it for subsequent tasks

• Then apply JavaVM pointer to acquire per-thread JNIEnv pointer

jint AttachCurrentThread

(JavaVM* vm, JNIEnv** p_env, void* thr_args);

Return JNI interface pointer

bound to current thread

Install Hooks

• Load instrument gadgets defined in tool APK

• Modify ART artifacts to divert the control flow to

our space when the specified methods are called

ART Metadata

Class A

Class B

Class Z

……

DexCache

ArtMethodA

ArtMethodB

ArtMethodZ

……

ArtFieldA

ArtField B

ArtField Z

……

ClassLoader

dex_class_def_idx

dex_type_idx

Class

entry_point_from_interpreter

entry_point_from_portable_compiled_code

entry_point_from_quick_compiled_code

dex_code_item_offset

dex_method_index

ArtMethod

push {r5,r6,r7,lr}

sub sp, sp, #16

mov r7, r0

str r0, [sp, #0]

……

blx lr

code_item

ins_sizeouts_size

insns_sizeinsns

invoke-super {v1}, AB

iget_object v0, v1

if-eqz v0, +3

const/4 v0, #+1

……

native code

DEX bytecode

App Compiled Code

……

0x0a: invoke-virtual {v6}, java.lang.ClassLoader android.content.Context.getClassLoader()

0x0d: move-result-object v3

0x0e: invoke-virtual {v3, v7}, java.lang.Class java.lang.ClassLoader.loadClass(java.lang.String)

0x11: move-result-object v0

……

DEX bytecode

dex PC: 0x000a

……

mov r7, r0

mov r1, r7

mov r2, r11

ldr r0, [r1, #0]

dex PC: 0x000e

ldr.w r0, [r0, #484]

ldr.w lr, [r0, #40]

blx lr

……

native code

Get ClassLoader object

Get loadClass() ArtMethod pointer

Get compiled code function pointer

Branch and link to the callee

Compiled Code Pointer

Normal method call

flow in most cases

Get ArtMethod

pointer

Get quick compiled

code pointer

Indirect jump to

method code

Modify quick compiled code pointer

to let it point to hook trampoline

Get ArtMethod

pointer

Get quick compiled

code pointer

Indirect jump to

trampoline

Call pre-method

gedget

Call original

method

Call post-method

gedget

ArtMethod Pointer

jmethodID (*GetMethodID) (JNIEnv*, jclass, const char*, const char*);

jmethodID (*GetStaticMethodID)(JNIEnv*, jclass, const char*, const char*);

jclass (*FindClass)(JNIEnv*, const char*);

Get loaded class id by

specifying class name

Get method id by specifying class

id, method name and signature

• jmethodID is actually the pointer to ArtMethod class

• Refer to art/runtime/art_method.h for ArtMethod class definition

and resolve member offset for entry_point_from_quick_compiled_code

About Install Timing

• Must be done before the execution of app code

• How early is that moment ?

App Init Flow

ZygoteInit .

runSelectLoop()

Wait for forking task

from ActivityManager

ActivityManagerService.

attachApplicationLocked()

Handle app initialization and

register the relevant metadata

ZygoteConnection .

handleChildProc()

Prepare to specialize the

process runtime

Call Zygote.

forkAndSpecialize()

ActivityThread. main()

Create an event handler to serve

the requests from ActivityManager

ActivityThread$H. handleMessage() callActivityThread. handleBindApplication()

Serve requests in event handler:

• Load the app Application class

• Call its Application.onCreate()

Zygote Process App Process

ActivityManager Process

The base runtime context is ready and the

lifecycle management for ContentProvider,

Activity, and Service is now started

Install Timing

• Before ActivityThread executes Application.onCreate()

• The declaring classes of the to be instrumented methods

may not be loaded at that early moment

• Must intercept the ClassLoader used by ActivityThread to

load non-system classes containing app defined methods

Intercept ClassLoader

Deploy a special hook for ClassLoader.loadClass()

ActivityThread

calls ClassLoader.loadClass()

Load app

Application class

ActivityThread

calls ClassLoader.loadClass()

Intercept app

ClassLoader

Load declaring classes

of target methods

Install hooks for target

methods

Load app

Application class• Restore work of ActivityThread

• Resolve ArtMethod pointer

• Substitute native code pointer

• Patch instrument gadget

Stage: Play Music

libART

Android & Java

Base Classes

App Classes

Instrument

Classes

libProbeDroid

App Process • Method in/output

• Extract different types of

parameters and return value

• Gadget marshalling

• Deliver modifiable parameters

• Original method invocation

• Handle generic method call

for different signature types

Call method

Trap to

ProbeDroid

Call gadget

Call original

method

Extract In/Output

• Control flow is trapped into trampoline

• Extract input parameters for gadget and pass them to

the original method

• Return from original method

• Extract return value for gadget and pass it to caller

• How to extract correct data on different ISAs?

Calling Convention

Method Register EAXEntry Spill

1st Chunk ECX2nd Chunk EDX3rd Chunk EBXOthers stored on stack

Return Register

Float and Double XMM0Long EAX_EDXOther types EAX

Method Register R0Entry Spill

1st Chunk R12nd Chunk R23rd Chunk R3Others stored on stack

Return Register

Double and Long R0_R1Float R0Other types R0

X86 ISA ARM ISA

• Defined in art/compiler/jni/quick/.* source files

• Trampoline must follow the conventions defined on different ISAs

Entry Spill

R1 ObjPtr

R2 Char

R3 Long (Hi)

Stk Long (Lo)

Int

Virtual Method

Signature Func1(CJI)V

1st param

4th param

3rd param

2nd param

R1 Byte

R2 Double(Hi)

R3 Double(Lo)

Stk ObjPtr

Float

Static Method

Signature Func2(BD[SF)V

1st param

4th param

3rd param

2nd param

• Parameter index is determined by data type and is not always

equal to chunk index

• Trampoline must be smart enough to handle this

Marshal Gadgets

• Deliver modifiable method in/output for gadgets

• Manage object pointer and reference in trampoline

Box In/Output

• Trampoline applies JNIs to invoke gadgets

• To make primitive in/output modifiable for

gadgets, we can box them in wrapper classes

byte Byte

short Short

int Integer

long Long

float Float

double Double

boolean Boolean

char Character

• Gadgets manipulate wrapper objects and trampoline must

unbox objects for record update when gadgets return

Object Pointer and Reference

• Non-primitive data is compiled and managed as

object pointer in native code

• Object pointer in JNI is boxed and managed as

indirect reference that maps to the real pointer

jstring str = env->NewStringUTF(“jni\0”);

env->CallVoidMethod(receiver, method_id, str);

Shuffle

Function

str

ObjPtr

receiver

ObjPtr

Indirect

Reference Table

Ref 1

Idx 1

Ref 2

Idx 2

• Receiver object and non-primitive in/output

trapped by trampoline are object pointers

• Since JNI only accepts reference, it is necessary to

• Wrap object pointers as references to call gadgets

• Unwrap reference to object pointer when gadgets return

• No exported interface, so manually resolve ART functions

relevant to reference management

Object Pointer and Reference

Indirect Reference Table

struct JNIEnvExt : public JNIEnv

{

… …

void* thread_;

void* jvm_;

uint32_t local_ref_cookie_;

void* local_refs_table_;

… …

};

Actual definition of JNI interfacestruct JNIEnvExt : public JNIEnv

{

… …

Thread* const self;

JavaVMExt* vm;

// Cookie used when using the local

// indirect reference table.

uint32_t local_ref_cookie;

// JNI local references.

IndirectReferenceTable locals;

… …

}

Our mirrored structure

• Extract the indirect reference table defined in JNIEnvExt structure

• Craft a mirrored structure for space layout resolution

• Cast JNIEnv pointer to this type for member field access

IndirectRef IndirectReferenceTable::

Add(uint32_t cookie, mirror::Object* obj)

bool IndirectReferenceTable::

Remove(uint32_t cookie, IndirectRef iref)

Indirect Reference Table

mirror::Object*

Thread::DecodeJObject(jobject obj)

Insert an object pointer and

the reference is returned

Remove a table entry with the

specified reference

Get the object pointer with

the specified reference

• Use dlsym() to resolve ART functions relevant to table manipulation

• To fulfill C++ calling convention, pass the table pointer extracted

from the casted JNIEnvExt structure as this pointer

Call Original Method

• Trampoline never knows the signature type of to

be called method until runtime

• Impossible to enumerate all the combinations of

parameter list and return type with C/C++

• Solution is to make generic function call with

assembly programming that fits native ISA

LibFFI

• Portable Foreign Function Interface library

• Generic C interface to various ISAs that allows users to

call functions with signatures specified at runtime

• Users apply it to bridge the interpreted and natively

compiled code including CPython, OpenJDK, DalvikVM

• ProbeDroid uses it to reduce native programming effort

LibFFI Project Page

LibFFI Usage

ffi_status

ffi_prep_cif (ffi_cif *cif,

ffi_abi abi,

unsigned int nargs,

ffi_type *rtype,

ffi_type **argtypes)

void

ffi_call (ffi_cif *cif,

void *fn,

void *rvalue,

void **avalues)

FFI context

ABI to use

Number of parameters

Return type specification

Parameter types specification

FFI context

Function pointer

To be filled with return value

Parameter values to be passed in

• Aim to specify function signature

• Aim to make generic function call

LibFFI API Reference

Generate Function Call at Runtimeboolean

String.startsWith(String, int)

JniEnv*

ObjPtr*

jmethodID*

ObjPtr*

Int*

ffi_type_pointer

ffi_type_pointer

ffi_type_pointer

ffi_type_pointer

ffi_type_sint32

Mini Interpreter

FFI parametervalue array

Craft FFI call payload

bool*ffi_type_

uint8

FFI parametertype array

return value return type

App Analysis with Custom Tools

(To be fine tuned with more demo and samples)

Google Maps Demo

Forensics for the strings converted from StringBuffer and

StringBuilder buffer with tool

Click the Picture for demo link

StringInspector

Obfuscated App

•Detect dynamic code loading and intercept the payload

DexClassLoader<init>(String dexPath, String optimizedDirectory,

String librarySearchPath, ClassLoader parent)

More relevant methods … …

Field Class.getDeclaredField(String name)Method Class.getDeclaredMethod(String name)

Object Field.get(Object object)Object Method.invoke(Object receiver, Object[] args)

More relevant methods … …

• Resolve the method and field mutated via Java reflection

Instant Messaging App

javax.crypto.Cipher.* relevant methods

• Useful to analyze crypto algorithm

• Resolve app credential (evil ?)

android.net.http.* relevant methods

• Useful to analyze authentication protocol

• Resolve message content (evil ?)

And more app analysis result …