Michael Bond Varun Srivastava Kathryn McKinley Vitaly Shmatikov University of Texas at Austin

Efficient, Context-Sensitive Detection of Real-World Semantic AttacksMichael BondVarun SrivastavaKathryn McKinleyVitaly ShmatikovUniversity of Texas at Austin

Real Semantic Exploits&

Efficient, Context-Sensitive Detection

classLoader.loadClass(“java.util.HashSet”);

How an applet loads a class

loadClass(name) { ... if (name.lastIndexOf(‘.’) != -1) securityManager.checkPackageAccess(name); ... super.loadClass();}

classLoader.loadClass(“java.util.HashSet”);

loadClass(name) { ... if (name.lastIndexOf(‘.’) != -1) securityManager.checkPackageAccess(name); ... super.loadClass();}

SecurityManager.checkPackageAccess()

classLoader.loadClass(“java.util.HashSet”);

Access-control security

loadClass(name) { ... if (name.lastIndexOf(‘.’) != -1) securityManager.checkPackageAccess(name); ... super.loadClass();}

ClassLoader.loadClass():341

walkPathComponents() { ...121: { ... if (file.exists()) ... } ...139: { ... if (file.exists()) ... } ...}

FileURLLoader.getResource():73

SecurityManager.checkPackageAccess()

classLoader.loadClass(“java.util.HashSet”);

loadClass(name) { ... if (name.lastIndexOf(‘.’) != -1) securityManager.checkPackageAccess(name); ... super.loadClass();}

ClassLoader.loadClass():341

walkPathComponents() { ...121: { ... if (file.exists()) ... } ...139: { ... if (file.exists()) ... } ...} SecurityManager.checkRead()

File.checkRead():1485

File.exists():268

FileURLLoader.getResource():73

SecurityManager.checkPackageAccess()

classLoader.loadClass(“java.util.HashSet”);

classLoader.loadClass(“sun/applet/AppletClassLoader”);

Sun Java Virtual Machine 1.3

loadClass(name) { ... if (name.lastIndexOf(‘.’) != -1) securityManager.checkPackageAccess(name); ... super.loadClass();}

classLoader.loadClass(“sun/applet/AppletClassLoader”);

loadClass(name) { ... if (name.lastIndexOf(‘.’) != -1) securityManager.checkPackageAccess(name); ... super.loadClass();}

SecurityManager.checkPackageAccess()

classLoader.loadClass(“sun/applet/AppletClassLoader”);

loadClass(name) { ... if (name.lastIndexOf(‘.’) != -1) securityManager.checkPackageAccess(name); ... super.loadClass();}

ClassLoader.loadClass():341

walkPathComponents() { ...121: { ... if (file.exists()) ... } ...139: { ... if (file.exists()) ... } ...} SecurityManager.checkRead()

File.checkRead():1485

File.exists():268

FileURLLoader.getResource():73

SecurityManager.checkPackageAccess()

classLoader.loadClass(“sun/applet/AppletClassLoader”);

loadClass(name) { ... if (name.lastIndexOf(‘.’) != -1) securityManager.checkPackageAccess(name); ... super.loadClass();}

ClassLoader.loadClass():341

walkPathComponents() { ...121: { ... if (file.exists()) ... } ...139: { ... if (file.exists()) ... } ...} SecurityManager.checkRead()

File.checkRead():1485

File.exists():268

FileURLLoader.getResource():73

SecurityManager.checkPackageAccess()

classLoader.loadClass(“sun/applet/AppletClassLoader”);

Semantic exploit

Examples:• Omitted security check• Untrusted code executes in wrong

context• Misconfigured security policy

loadClass(name) { ... if (name.lastIndexOf(‘.’) != -1) securityManager.checkPackageAccess(name); ... super.loadClass();}

ClassLoader.loadClass():341

walkPathComponents() { ...121: { ... if (file.exists()) ... } ...139: { ... if (file.exists()) ... } ...} SecurityManager.checkRead()

File.checkRead():1485

File.exists():268

FileURLLoader.getResource():73

SecurityManager.checkPackageAccess()

classLoader.loadClass(“sun/applet/AppletClassLoader”);

How to detect this exploit?

Infeasible path detection?

Does not violate semantics

(e.g., type & memory safety,

control-flow integrity)

loadClass(name) { ... if (name.lastIndexOf(‘.’) != -1) securityManager.checkPackageAccess(name); ... super.loadClass();}

ClassLoader.loadClass():341

walkPathComponents() { ...121: { ... if (file.exists()) ... } ...139: { ... if (file.exists()) ... } ...} SecurityManager.checkRead()

File.checkRead():1485

File.exists():268

FileURLLoader.getResource():73

SecurityManager.checkPackageAccess()

classLoader.loadClass(“sun/applet/AppletClassLoader”);

How to detect this exploit?

Check against specification?

No specification available

loadClass(name) { ... if (name.lastIndexOf(‘.’) != -1) securityManager.checkPackageAccess(name); ... super.loadClass();}

ClassLoader.loadClass():341

walkPathComponents() { ...121: { ... if (file.exists()) ... } ...139: { ... if (file.exists()) ... } ...} SecurityManager.checkRead()

File.checkRead():1485

File.exists():268

FileURLLoader.getResource():73

SecurityManager.checkPackageAccess()

classLoader.loadClass(“sun/applet/AppletClassLoader”);

How to detect this exploit?

Infer specification from dynamic behavior?

loadClass(name) { ... if (name.lastIndexOf(‘.’) != -1) securityManager.checkPackageAccess(name); ... super.loadClass();}

ClassLoader.loadClass():341

walkPathComponents() { ...121: { ... if (file.exists()) ... } ...139: { ... if (file.exists()) ... } ...}

File.exists():268

FileURLLoader.getResource():73

classLoader.loadClass(“sun/applet/AppletClassLoader”);

Which dynamic behavior?

SecurityManager.checkPackageAccess()

SecurityManager.checkRead()

File.checkRead():1485

loadClass(“java.util.HashMap”);… SecurityManager.checkPackageAccess()…… FileURLLoader.getResource():73 walkPathComponents() :121

File.exists()

loadClass(“sun/applet/AppletClassLoader”);… SecurityManager.checkPackageAccess()…… FileURLLoader.getResource():73 walkPathComponents() :121

File.exists()

loadClass(“MyClass”);… SecurityManager.checkPackageAccess()…… FileURLLoader.getResource():73 walkPathComponents() :139

File.exists()

loadClass(“java.util.HashMap”);… SecurityManager.checkPackageAccess()…… FileURLLoader.getResource():73 walkPathComponents() :121

File.exists()

loadClass(“sun/applet/AppletClassLoader”);… SecurityManager.checkPackageAccess()…… FileURLLoader.getResource():73 walkPathComponents() :121

File.exists()

loadClass(“MyClass”);… SecurityManager.checkPackageAccess()…… FileURLLoader.getResource():73 walkPathComponents() :139

File.exists()

Train

Deploy

classLoader.loadClass(“MyClass”);

loadClass(name) { ... if (name.lastIndexOf(‘.’) != -1) securityManager.checkPackageAccess(name); ... super.loadClass();}

classLoader.loadClass(“MyClass”);

loadClass(name) { ... if (name.lastIndexOf(‘.’) != -1) securityManager.checkPackageAccess(name); ... super.loadClass();}

SecurityManager.checkPackageAccess()

classLoader.loadClass(“MyClass”);

loadClass(name) { ... if (name.lastIndexOf(‘.’) != -1) securityManager.checkPackageAccess(name); ... super.loadClass();}

ClassLoader.loadClass():341

walkPathComponents() { ...121: { ... if (file.exists()) ... } ...139: { ... if (file.exists()) ... } ...} SecurityManager.checkRead()

File.checkRead():1485

File.exists():268

FileURLLoader.getResource():73

SecurityManager.checkPackageAccess()

classLoader.loadClass(“MyClass”);

loadClass(“java.util.HashMap”);… SecurityManager.checkPackageAccess()…… FileURLLoader.getResource():73 walkPathComponents() :121

File.exists()

loadClass(“sun/applet/AppletClassLoader”);… SecurityManager.checkPackageAccess()…… FileURLLoader.getResource():73 walkPathComponents() :121

File.exists()

loadClass(“MyClass”);… SecurityManager.checkPackageAccess()…… FileURLLoader.getResource():73 walkPathComponents() :139

File.exists()

loadClass(“java.util.HashMap”);… SecurityManager.checkPackageAccess()…… FileURLLoader.getResource():73 walkPathComponents() :121

File.exists()

loadClass(“sun/applet/AppletClassLoader”);… SecurityManager.checkPackageAccess()…… FileURLLoader.getResource():73 walkPathComponents() :121

File.exists()

loadClass(“MyClass”);… SecurityManager.checkPackageAccess()…… FileURLLoader.getResource():73 walkPathComponents() :139

File.exists()

loadClass(“java.util.HashMap”);… SecurityManager.checkPackageAccess()…… FileURLLoader.getResource():73 walkPathComponents() :121

File.exists()

loadClass(“sun/applet/AppletClassLoader”);… SecurityManager.checkPackageAccess()…… FileURLLoader.getResource():73 walkPathComponents() :121

File.exists()

loadClass(“MyClass”);… SecurityManager.checkPackageAccess()…… FileURLLoader.getResource():73 walkPathComponents() :139

File.exists()

Train

Train

Deploy

(Sampled & Reproduced)Real Semantic Exploits

Context sensitivity needed?

History sensitivity needed?

SlashPathMistakenly omitted security check

Yes Yes

XSLTUntrusted code executes in wrong (application) security context

Yes No

LiveConnectUntrusted code executes in wrong (applet) security context

No No

OperaPolicyMisconfigured security policy

No No

ClassLoader.loadClass():312

ClassLoader.loadClass():341

SecurityManager.checkRead()

File.checkRead():1485

File.exists():268

FileURLLoader.getResource():73

FileURLLoader.walkPathComponents():121

More

con

text

sen

siti

vit

y

ClassLoader.loadClass():312

ClassLoader.loadClass():341

SecurityManager.checkRead()

File.checkRead():1485

File.exists():268

FileURLLoader.getResource():73

FileURLLoader.walkPathComponents():121

Fals

e n

eg

ati

ves

Fals

e p

osi

tives

More

con

text

sen

siti

vit

y

ClassLoader.loadClass():312

ClassLoader.loadClass():341

SecurityManager.checkRead()

File.checkRead():1485

File.exists():268

FileURLLoader.getResource():73

FileURLLoader.walkPathComponents():121

Fals

e n

eg

ati

ves

Fals

e p

osi

tives

More

con

text

sen

siti

vit

y

Overh

ead

loadClass(name) { ... if (name.lastIndexOf(‘.’) != -1) securityManager.checkPackageAccess(name); ... super.loadClass();}

ClassLoader.loadClass():341

walkPathComponents() { ...121: { ... if (file.exists()) ... } ...139: { ... if (file.exists()) ... } ...}

SecurityManager.checkRead()

File.checkRead():1485

File.exists():268

FileURLLoader.getResource():73

SecurityManager.checkPackageAccess()

classLoader.loadClass(“java.util.HashSet”);

loadClass(name) { ... if (name.lastIndexOf(‘.’) != -1) securityManager.checkPackageAccess(name); ... super.loadClass();}

ClassLoader.loadClass():341

walkPathComponents() { ...121: { ... if (file.exists()) ... } ...139: { ... if (file.exists()) ... } ...}

SecurityManager.checkRead()

File.checkRead():1485

File.exists():268

FileURLLoader.getResource():73

SecurityManager.checkPackageAccess()

classLoader.loadClass(“java.util.HashSet”);

S ← walkStack ()check ( S )

S ← walkStack ()check ( S )

Costs of Context Sensitivity

antlr

char

tec

lipse fop

hsql

dbjy

thon

luin

dex

luse

arch

pmd

xala

nps

eudo

jbb

geom

ean

0%

50%

100%

150%

200%

Walk stack (k = inf)Walk stack (k = 3)

Overh

ead

Costs of Context Sensitivity

antlr

char

tec

lipse fop

hsql

dbjy

thon

luin

dex

luse

arch

pmd

xala

nps

eudo

jbb

geom

ean

0%

50%

100%

150%

200%

Walk stack (k = inf)Walk stack (k = 3)

Overh

ead

Proportional todepth & security

calls

High overhead at security calls

Efficient,Depth-Limited

Context Sensitivity

Probabilistic Calling Context[Bond & McKinley ’07]

Represent calling context asprobabilistically unique integer

Probabilistic Calling Context[Bond & McKinley ’07]

Compute value at every callUse value at security calls

Probabilistic Calling Context[Bond & McKinley ’07]

Compute value at every callUse value at security calls

Always-available contextLow overhead at security calls

loadClass(name) { ... if (name.lastIndexOf(‘.’) != -1) securityManager.checkPackageAccess(name); ... super.loadClass();}

ClassLoader.loadClass():341

walkPathComponents() { ...121: { ... if (file.exists()) ... } ...139: { ... if (file.exists()) ... } ...}

SecurityManager.checkRead()

File.checkRead():1485

File.exists():268

FileURLLoader.getResource():73

SecurityManager.checkPackageAccess()

classLoader.loadClass(“java.util.HashSet”);

loadClass(name) { ... if (name.lastIndexOf(‘.’) != -1) securityManager.checkPackageAccess(name); ... super.loadClass();}

ClassLoader.loadClass():341

walkPathComponents() { ...121: { ... if (file.exists()) ... } ...139: { ... if (file.exists()) ... } ...}

SecurityManager.checkRead()

File.checkRead():1485

File.exists():268

FileURLLoader.getResource():73

SecurityManager.checkPackageAccess()

classLoader.loadClass(“java.util.HashSet”);

V1 ← f ( V0 , cs1 )

V0 ← 0

loadClass(name) { ... if (name.lastIndexOf(‘.’) != -1) securityManager.checkPackageAccess(name); ... super.loadClass();}

ClassLoader.loadClass():341

walkPathComponents() { ...121: { ... if (file.exists()) ... } ...139: { ... if (file.exists()) ... } ...}

SecurityManager.checkRead()

File.checkRead():1485

File.exists():268

FileURLLoader.getResource():73

SecurityManager.checkPackageAccess()

classLoader.loadClass(“java.util.HashSet”);

V1 ← f ( V0 , cs1 )

V2 ← f ( V0 , cs2 )

V0 ← 0

loadClass(name) { ... if (name.lastIndexOf(‘.’) != -1) securityManager.checkPackageAccess(name); ... super.loadClass();}

ClassLoader.loadClass():341

walkPathComponents() { ...121: { ... if (file.exists()) ... } ...139: { ... if (file.exists()) ... } ...}

SecurityManager.checkRead()

File.checkRead():1485

File.exists():268

FileURLLoader.getResource():73

SecurityManager.checkPackageAccess()

classLoader.loadClass(“java.util.HashSet”);

V1 ← f ( V0 , cs1 )

V2 ← f ( V0 , cs2 )V3 ← f ( V2 , cs3 )

V4 ← f ( V3 , cs4 )

V5 ← f ( V4 , cs5 )V6 ← f ( V5 , cs6 )

V7 ← f ( V6 , cs7 )

V0 ← 0

loadClass(name) { ... if (name.lastIndexOf(‘.’) != -1) securityManager.checkPackageAccess(name); ... super.loadClass();}

ClassLoader.loadClass():341

walkPathComponents() { ...121: { ... if (file.exists()) ... } ...139: { ... if (file.exists()) ... } ...}

SecurityManager.checkRead()

File.checkRead():1485

File.exists():268

FileURLLoader.getResource():73

SecurityManager.checkPackageAccess()

classLoader.loadClass(“java.util.HashSet”);

V1 ← f ( V0 , cs1 )check ( V1 )

V2 ← f ( V0 , cs2 )V3 ← f ( V2 , cs3 )

V4 ← f ( V3 , cs4 )

V5 ← f ( V4 , cs5 )V6 ← f ( V5 , cs6 )

V7 ← f ( V6 , cs7 )check ( V7 )

V0 ← 0

loadClass(name) { ... if (name.lastIndexOf(‘.’) != -1) securityManager.checkPackageAccess(name); ... super.loadClass();}

ClassLoader.loadClass():341

walkPathComponents() { ...121: { ... if (file.exists()) ... } ...139: { ... if (file.exists()) ... } ...}

SecurityManager.checkRead()

File.checkRead():1485

File.exists():268

FileURLLoader.getResource():73

SecurityManager.checkPackageAccess()

classLoader.loadClass(“java.util.HashSet”);

V1 ← f ( V0 , cs1 )check ( V1 )

V2 ← f ( V0 , cs2 )V3 ← f ( V2 , cs3 )

V4 ← f ( V3 , cs4 )

V5 ← f ( V4 , cs5 )V6 ← f ( V5 , cs6 )

V7 ← f ( V6 , cs7 )check ( V7 )

V0 ← 0

Historysensitivity

f ( V , cs ) ≡ 3V + cs (mod 232)

PCC Function

f ( V , cs ) ≡ 3V + cs (mod 232)

Motivated by MPI data-type hashing [Langou et al. ’05] [Gropp ’00]

PCC Function

f ( V , cs ) ≡ 3V + cs (mod 232)

Encodes entire calling context

PCC Function

f ( V , cs ) ≡ 2 32/k V + cs (mod 232)

Encodes last k call sites

PCC Function

f ( V , cs ) ≡ 2 32/k V + cs (mod 232)

Cheap to compute

PCC Function

f ( V , cs ) ≡ 2 32/k V + cs (mod 232)

Cheap to compute Composition cheap to compute

PCC Function

f ( V , cs ) ≡ 2 32/k V + cs (mod 232)

Cheap to compute Composition cheap to compute Non-commutative

PCC Function

f ( V , cs ) ≡ 2 32/k V + cs (mod 232)

Cheap to compute Composition cheap to compute Non-commutative Probabilistically unique (?)

PCC Function

Costs of Context Sensitivity

antlr

char

tec

lipse fop

hsql

dbjy

thon

luin

dex

luse

arch

pmd

xala

nps

eudo

jbb

geom

ean

0%

50%

100%

150%

200%

Walk stack (k = inf)Walk stack (k = 3)PCC

Overh

ead

Costs of Context Sensitivity

antlr

char

tec

lipse fop

hsql

dbjy

thon

luin

dex

luse

arch

pmd

xala

nps

eudo

jbb

geom

ean

0%

50%

100%

150%

200%

Walk stack (k = inf)Walk stack (k = 3)PCC

Overh

ead

Not proportionalto depth

Low overhead at security calls

Detect all exploitswithout many false positives

Context sensitivity: 3History sensitivity: 1

Evaluating Accuracy

Real Semantic Exploit

Context sensitivity needed?

History sensitivity needed?

SlashPathMistakenly omitted security check

Yes Yes

XSLTUntrusted code executes in wrong (application) security context

Yes No

LiveConnectUntrusted code executes in wrong (applet) security context

No No

OperaPolicyMisconfigured security policy

No No

Evaluating False Positives

Leave-one-out cross-validation on

12 benign applets8 benign XSLT inputs

Evaluating False Positives

Leave-one-out cross-validation on

12 benign applets8 benign XSLT inputs

Depth-limited context sensitivity needed

Related Work

Context and history sensitivityfor unsafe languages [Forrest et al., Feng et

al.]

Context sensitivityfor anomalous paths [Inoue et al.]

Context & history sensitivityactually neededfor real exploits

Context & history sensitivityactually neededfor real exploits

Tension between false positives & negatives

Backup

Always-Available Calling Context

check(V) { H = h(V, lastV); checkHelper(H); lastV = V; }

classLoader.loadClass(“java.util.HashSet”)

SecurityManager.checkRead()SecurityManager.checkPackageAccess()

How an Applet Loads a Class

classLoader.loadClass(“java.util.HashSet”)

How an Applet Loads a Class

classLoader.loadClass(“sun/applet/AppletClassLoader”)

SecurityManager.checkPackageAccess() SecurityManager.checkRead()

SecurityManager.checkRead()SecurityManager.checkPackageAccess()

classLoader.loadClass(“java.util.HashSet”)

Anomaly-Based Intrusion Detection

classLoader.loadClass(“sun/applet/AppletClassLoader”)

SecurityManager.checkPackageAccess() SecurityManager.checkRead()

SecurityManager.checkRead()SecurityManager.checkPackageAccess()

Train: observe behavior

Deploy: detect new behavior

classLoader.loadClass(“java.util.HashSet”)

Anomaly-Based Intrusion Detection

classLoader.loadClass(“sun/applet/AppletClassLoader”)

SecurityManager.checkPackageAccess() SecurityManager.checkRead()

SecurityManager.checkRead()SecurityManager.checkPackageAccess()

classLoader.loadClass(“MyClass”);

SecurityManager.checkRead()SecurityManager.checkPackageAccess()

classLoader.loadClass(“java.util.HashSet”)

classLoader.loadClass(“sun/applet/AppletClassLoader”)

SecurityManager.checkPackageAccess()

SecurityManager.checkPackageAccess()

classLoader.loadClass(“MyClass”);

SecurityManager.checkPackageAccess()

SecurityManager.checkRead()...FileURLLoader.walkPathComponents():139...

SecurityManager.checkRead()...FileURLLoader.walkPathComponents():121...

SecurityManager.checkRead()...FileURLLoader.walkPathComponents():121...

Context Sensitivity Needed!

classLoader.loadClass(“java.util.HashSet”)

classLoader.loadClass(“sun/applet/AppletClassLoader”)

SecurityManager.checkPackageAccess()

SecurityManager.checkPackageAccess()

classLoader.loadClass(“MyClass”);

SecurityManager.checkPackageAccess()

SecurityManager.checkRead()...FileURLLoader.walkPathComponents():139...

SecurityManager.checkRead()...FileURLLoader.walkPathComponents():121...

SecurityManager.checkRead()...FileURLLoader.walkPathComponents():121...

Train: observe behavior

Deploy: detect new behavior

Train: observe behavior

Context Sensitivity Needed!

Detecting Real Attacks

C3 H1

Context sensitivity History sensitivity

Detecting Real Attacks

C3 H1

Context sensitivity History sensitivity(applets only)

Detecting Real Attacks

C0 H0 C0 H1

C1 H0 C1 H1

C3 H0 C3 H1

CH0 CH1

SlashPath

Anomalies

(All)

C0H00 (35)

C1H00 (54)

C3H00 (110)

CH00 (194)

Anomalies

(All)

C0H10 (59)

C1H11 (90)

C3H12 (145)

CH12 (222)

XSLT

Anomalies

(All)

C00 (20)

C10 (40)

C32 (42)

C222 (1,573)

Evaluating False Positives

Leave-one-out cross-validation on

12 benign applets8 benign XSLT inputs

Evaluating False Positives

ArcTest

AtomViewer

CardTest

DiffEq

DitherTest

DrawTest

C3H0 0 0 0 0 4 0

C3H1 1 9 0 1 7 0

CH0 32 113 0 125 77 10

CH1 40 61 10 131 94 5Euler Gas Matrix Puzzle ReflFrame StringWav

e

C3H0 2 0 0 0 4 0

C3H1 6 1 0 1 6 0

CH0 46 14 56 10 74 9

CH1 101 28 73 12 93 0

Evaluating False Positives

ui resume testcase testcase2

C0 0 0 0 0

C1 1 0 0 2

C3 0 0 1 2

C 15 3 63 1,409

testcase3

testcase4

testcase5

testcase6

C0 0 0 0 0

C1 0 1 0 0

C3 0 1 0 0

C 6 2 49 1