PTS 2019 CANDIDATE PDF · start app hook trigged hook shows stack trace hook gathers method info. dynamic update of « xref from »with invoked methods dynamic update of xref with

DEXCALIBURAUTOMATE YOUR ANDROID APP REVERSE

Or hooking for dummies

https://github.com/FrenchYeti/dexcalibur.git


WHO AM I ?

GEORGES-B. MICHEL

▸ @FrenchYeti

▸ [email protected]

▸ Software Security Evaluator at Thales

▸ Day : Reverse engineering (Android + TEE) apps

▸ HCE Payment applications, Trusted Applications, ARM binaries

▸ Night : Develop reverse / pentest / appsec tools

▸ Frida addict

Aka @FrenchYeti

EXAMPLE OF AN OBFUSCATED ANDROID APPLICATION

MOTIVATION

LET’S IMAGINE AN OBFUSCATED MULTI-DEX APPLICATIONPACKER

CLASS LOADER DEX LOADER

APP CLASSES & METHODS

Clear .dex file & JNI libs

Ciphered secondary .dex file

DECIPHER & LOAD

NATIVEFUNCTIONS

MOTIVATION






DECIPHER & LOAD

INVOKE BY REFLECTION

NATIVEFUNCTIONS

MOTIVATION






DECIPHER & LOAD


Ciphered JNI lib

DECIPHER & LOAD

NATIVEFUNCTIONS

WHITE BOXCRYPTO

MOTIVATION






DECIPHER & LOAD


Ciphered JNI lib

DECIPHER & LOAD

Class loaded from the network(NetworkClassLoader)

DOWNLOAD,DECIPHER & LOAD

JNI FUNCTIONS

NATIVEFUNCTIONS

WHITE BOXCRYPTO

MOTIVATION






DECIPHER & LOAD


Ciphered JNI lib

DECIPHER & LOAD



JNI FUNCTIONS

NATIVEFUNCTIONS

WHITE BOXCRYPTO

MOTIVATION

PACKER CLASS LOADER

DEX LOADER



DECIPHER


Ciphered JNI lib

DECIPHER & LOAD



JNI FUNCTIONS

YOU CAN HOOKONLY WHAT YOU SEE

WHAT CAN I HOOK ?


NATIVEFUNCTIONS

MOTIVATION

PACKER CLASS LOADER

DEX LOADER


Clear .dex file


DECIPHER


Ciphered JNI lib

DECIPHER & LOAD



JNI FUNCTIONS

IT REQUIRES SEVERAL HOOKING SESSIONS


NATIVEFUNCTIONS

WHAT IS INTERESTING TO HOOK ?

MOTIVATION

THE IDEA

▸ Deobfuscate waste of time

MOTIVATION

THE IDEA


▸ Manage hooks not so easy

MOTIVATION

THE IDEA



▸ Manual tasks can be automated (start App, …)

MOTIVATION

THE IDEA




▸ Several devices hooked simultaneously

MOTIVATION

THE IDEA




▸ Several devices hooked simultaneously

▸ Application size explore bytecode/libs is boring

MOTIVATION

THE IDEA

▸ Show functions invoked dynamically as « xrefs »

▸ Discover automatically classes & bytecode loaded dynamically (DexFile ..)

▸ Generate hook with a single click on the function

▸ Debug a single hook while others are active

▸ Enable/disable hook without lose or pollute the source code

CHRISTMAS WISH LIST 1/2 :

THE IDEA

▸ Multi-user : share the same instrumentation with my friends

▸ Instrumente several devices and merge hook logs (Workflow / IoT)

▸ Be able to run with rooted & non-rooted devices

▸ Offer user-friendly GUI and API,

▸ Free & open-source ! ( license APACHE 2 )

CHRISTMAS WISH LIST 2/2 :

WHAT IS DEXCALIBUR ?

NOT JUST A TOOLBOX

DEX DISASSEMBLER Baksmali


NOT JUST A TOOLBOX

FILE IDENTIFIERS & PARSERS



NOT JUST A TOOLBOX

STATIC BYTECODE ANALYZER

DYNAMIC BYTECODE ANALYZER




NOT JUST A TOOLBOX





INSTRUMENTATION TOOL


NOT JUST A TOOLBOX




MODULAR HEURISTIC & SEARCH ENGINE




NOT JUST A TOOLBOX





DEVICE MANAGER & FRIDA UTILS




NOT JUST A TOOLBOX





WEB SERVER & UI


CONTROLS & CUSTOMIZE




NOT JUST A TOOLBOX





WEB SERVER & UI


IMPROVES ATRUNTIME CONTROLS &

CUSTOMIZE




NOT JUST A TOOLBOX





WEB SERVER & UI


IMPROVES ATRUNTIME CONTROLS &

CUSTOMIZE

DEXCALIBUR





POWERED BY …

ANDROID SDK

APKTOOL +

BAKSMALI

Today

NATIVE HOOK CANNOT BE GENERATED NO BYTECODE SYMBOLIC EXEC

Functions contained into JNI/native libscan be hooked, but decompilers/analyzersdont support it. So, native hook cannot be generated.

NICE TOOLS :-)


POWERED BY …

ANDROID SDK

APKTOOL +

BAKSMALI

Today

NATIVE HOOK CANNOT BE GENERATED NO BYTECODE SYMBOLIC EXEC

Functions contained into JNI/native libscan be hooked, but decompilers/analyzersdont support it. So, native hook cannot be generated.

ADD NATIVE LIBRARIES SUPPORT SMALI SYMBOLIC EXEC

NICE TOOLS :-)

LIEFR2 LIEF

Tomorrow

RetDec

SMALI VM Z3 SOLVERAND MORE !

DEMO #1

HOW IT WORKS ?

HOW IT WORKS ?

1) START PHASE - FILE ANALYSIS

UNCOMPRESS APKAPK FILE

DEVICE

FILE ANALYZER

Files identified & categorized:key stores, libs, properties, xml,shared pref, cache, …

Pull Application data/data/data/xxx …

Undetected / high entropy files are tagged

notify1

3

4

Parse APK content2

HOW IT WORKS ?

1) START PHASE - ANDROID API ANALYSIS


ANDROID API/STUB

ApplicationGraph

Statically builtDEX

DISASSEMBLER SAST

FILE ANALYZER

3

1 2

Create appgraph

DEVICE

HOW IT WORKS ?

1) START PHASE - APPLICATION BYTE CODE ANALYSIS


ANDROID API/STUB

DEX DISASSEMBLER

notify

SAST

ApplicationGraph

Statically builtDEX

DISASSEMBLER SAST

FILE ANALYZER

12

4

3Update appgraph

DEVICE

HOW IT WORKS ?

2) INSTRUMENTATION PHASE - BEFORE RUN

notify

Categorized Files

Application+Android APIGraph

Statically built

DYNAMIC LOADER

BYTE ARRAY CLASSIFIER FILE ACCESS KEY STORES

…

1 MODULAR HEURISTIC ENGINE

HOW IT WORKS ?

notify

Categorized Files


Statically built

DYNAMIC LOADER

NATIVE LIB / JNI

FILE ACCESS DESCRIPTORS

STREAMS

Search pattern &method

Correlate static filesBind a file to a method

KEY STORE

…

1

2

2’

MODULAR HEURISTIC ENGINE


HOW IT WORKS ?

Categorized Files


Statically built

DYNAMIC LOADER

NATIVE LIB / JNI


STREAMSKEY STORE

…

HOOK MANAGER

Get methodsignature

ASK FOR INSTRUMENTATION

Generatefrida code

HOOKS

3

45



HOW IT WORKS ?

2) INSTRUMENTATION PHASE - RUNTIME


Statically built

DYNAMIC LOADER

NATIVE LIB / JNI


STREAMSKEY STORE

…

HOOK MANAGER DEVICE

HOOKS

Starts app &deploys

Hook data : args, return, this, …

6

7Correlate graph &intercepted data8


HOW IT WORKS ?


Statically built

DYNAMIC LOADER

NATIVE LIB / JNI


STREAMSKEY STORE

…

HOOK MANAGER DEVICE

HOOKS

Starts app &deploys

Hook data : args, return, this, …

6

7Correlate intercepted data8

Push discovered elements & tag node9


2) INSTRUMENTATION PHASE - RUNTIME

« HEY ! GIVE ME THE MOST COMPLETE PICTURE OF THE APPLICATION »

DRAW A COMPLETE PICTURE OF THE APPLICATION

MIX * ANALYSIS WITH INSTRUMENTATION RESULTS

GRAPHSSTATIC

ANALYSIS

ANDROIDINTERNALS

CALLSSTATIC VALUES


GRAPHSSYMBOLIC

VALUESSTATIC

ANALYSIS

DYNAMIC ANALYSIS

ANDROIDINTERNALS

CALLSSTATIC VALUES

SOLVE CONSTRAINT

…



GRAPHSSYMBOLIC

VALUESSTATIC

ANALYSIS

DYNAMIC ANALYSIS

ANDROIDINTERNALS

CALLSSTATIC VALUES

FILE ANALYSIS

KEYSTORES

PROPERTIES

LIBS & DEX

…

STRUCTURESSOLVE CONSTRAINT

…



GRAPHSSYMBOLIC

VALUES

PARAMS & RETURNS

VALUES

STATIC ANALYSIS

DYNAMIC ANALYSIS

DYNAMIC INSTRUMENTATION

ANDROIDINTERNALS

CALLSSTATIC VALUES

DATA READ/WRITE SECONDARY

DEX & LIBS

STACK TRACERUNTIME CONTEXT

FILE ANALYSIS

KEYSTORES

PROPERTIES

LIBS & DEX

…

STRUCTURESSOLVE CONSTRAINT

…


CASE #1 DYNAMIC UPDATE OF XREF WITH INVOKED METHODS

METHOD INVOKED DYNAMICALLY

‣ Method.invoke()

‣ Class.getMethod()

From a static point-of-view only two methods are called :

Smali code

DYNAMIC UPDATE OF « XREF FROM »WITH INVOKED METHODS


DYNAMIC UPDATE OF XREF WITH INVOKED METHODS

GRAPHS

PARAMS & RETURNS

VALUES

STATIC ANALYSIS


ANDROIDINTERNALS

CALLSSTATIC VALUES


DEX & LIBS




GRAPHS

PARAMS & RETURNS

VALUES

STATIC ANALYSIS


ANDROIDINTERNALS

CALLSSTATIC VALUES


DEX & LIBS


REFLECTION API INSTRUMENTED



GRAPHS

PARAMS & RETURNS

VALUES

STATIC ANALYSIS


ANDROIDINTERNALS

CALLSSTATIC VALUES


DEX & LIBS


REFLECTION API INSTRUMENTEDSTART APP



GRAPHS

PARAMS & RETURNS

VALUES

STATIC ANALYSIS


ANDROIDINTERNALS

CALLSSTATIC VALUES


DEX & LIBS



HOOK TRIGGED



GRAPHS

PARAMS & RETURNS

VALUES

STATIC ANALYSIS


ANDROIDINTERNALS

CALLSSTATIC VALUES


DEX & LIBS



HOOK TRIGGED

HOOK GATHERS METHOD INFO



GRAPHS

PARAMS & RETURNS

VALUES

STATIC ANALYSIS


ANDROIDINTERNALS

CALLSSTATIC VALUES


DEX & LIBS



HOOK TRIGGED

HOOK SHOWS STACK TRACE




GRAPHS

PARAMS & RETURNS

VALUES

STATIC ANALYSIS


ANDROIDINTERNALS

CALLSSTATIC VALUES


DEX & LIBS



HOOK TRIGGED

HOOK SHOWS STACK TRACE


HEURISTIC ENGINE UPDATE DB

BEFORERUNTIME

DYNAMIC UPDATE OF THE CALL GRAPH


BEFORERUNTIME

AFTER RUNTIME



UPDATE OF THE CALL GRAPH

Green nodes are internal Android or Java methods

Pink node are invoked dynamically and not discovered statically

Gray nodes have been discovered statically


DEMO #2

DYNAMIC UPDATE OF XREFS WITH INVOKED METHODS


CASE #2 ANALYZE DEX FILE LOADED DYNAMICALLY

ANALYZE DEX FILE LOADED DYNAMICALLY


PARAMS & RETURNS

VALUES



DEX & LIBS


FILE ANALYSIS

LIBS & DEX

CLASS GRAPH

STATIC ANALYSIS

ANDROIDINTERNALS

CALLS



PARAMS & RETURNS

VALUES



DEX & LIBS


FILE ANALYSIS

LIBS & DEX

CLASS GRAPH

STATIC ANALYSIS

ANDROIDINTERNALS

CALLS

DEX LOADING API INSTRUMENTEDSTART APP



PARAMS & RETURNS

VALUES



DEX & LIBS


FILE ANALYSIS

LIBS & DEX

CLASS GRAPH

STATIC ANALYSIS

ANDROIDINTERNALS

CALLS

DEX LOADING API INSTRUMENTED

DEXFILE CONSTRUCTORS TRIGGED

START APP



PARAMS & RETURNS

VALUES



DEX & LIBS


FILE ANALYSIS

LIBS & DEX

CLASS GRAPH

STATIC ANALYSIS

ANDROIDINTERNALS

CALLS



START APP

HOOKS ASK IF DEX FILES ARE ALREADY KNOWN

Dex File already analyzed ?



PARAMS & RETURNS

VALUES



DEX & LIBS


FILE ANALYSIS

LIBS & DEX

CLASS GRAPH

STATIC ANALYSIS

ANDROIDINTERNALS

CALLS



START APP



COPY OR GET DEX FILE



PARAMS & RETURNS

VALUES



DEX & LIBS


FILE ANALYSIS

LIBS & DEX

CLASS GRAPH

STATIC ANALYSIS

ANDROIDINTERNALS

CALLS



START APP



COPY OR GET DEX FILEDECOMPILE DEX & UPDATE DB


CASE #3 BYTECODE CLEANER

BYTECODE CLEANER

BYTE CODE CLEANER : REMOVE NOP

BEFORE

BYTECODE CLEANER

BYTE CODE CLEANER : REMOVE NOP

BEFORE AFTER

REMOVE USELESS GOTO

BEFORE

BYTECODE CLEANER

REMOVE USELESS GOTO

BEFORE AFTER

BYTECODE CLEANER

DEXCALIBUR - NEXT STEPS

IMPROVEMENTS

‣ Use my own customizable Dex Decompiler (or use LIEF)?

‣ Add r2 binding and native hooks

‣ HTTP communications & Intent grabbing

‣ Bytecode & native symbolic exec (Z3) ?

‣ Bytecode emulation (SmaliVM @CalebFenton)?

‣ Offers native instruction hooking (QBDI)?

‣ And fuzz (afl-fuzz params + feedback given by hooking)?

DEXCALIBUR

Thanks

Q&A

ANNEXES

HOW TO INSTALL ?

HOW TO INSTALL ?

‣ Ensure you have the requirements (Frida, NodeJS, apktool)

‣ Or install from DockerHub

git clone https://github.com/FrenchYeti/dexcalibur.git

cd dexcalibur

npm install

docker pull frenchyeti/dexcalibur

docker run -it \

-v <workspace>:/home/dexcalibur/workspace \

-p 8080:8000 —dev=<device> \

frenchyeti/dexcalibur


DEXCALIBUR - WHAT IS IT ?

SEARCH BYTE ARRAY

Documents

PTS 2019 CANDIDATE PDF · start app hook trigged hook shows stack trace hook gathers method info. dynamic update of « xref from »with invoked methods dynamic update of xref with