View
215
Download
1
Embed Size (px)
Citation preview
Outline
I. Metamorphic software What is it? Good and evil uses
II. Metamorphic virus construction kitsIII. How effective are metamorphic engines?
How to compare two pieces of code? Similarity of viruses/normal code
IV. Can we detect metamorphic viruses? Commercial virus scanners HMMs and similarity index
V. Conclusion
What is Metamorphic Software?
Software is metamorphic provided All copies do the same thing Internal structure differs
Today almost all software is cloned “Good” metamorphic software…
Mitigate buffer overflow attacks “Bad” metamorphic software…
Avoid virus/worm signature detection
Metamorphic Software for Good?
Suppose program has a buffer overflow If we clone the program
One attack breaks every copy Break once, break everywhere (BOBE)
If instead, we have metamorphic copies Each copy still has a buffer overflow One attack does not work against every copy BOBE-resistant Analogous to genetic diversity in biology
A little metamorphism does a lot of good!
Metamorphic Software for Evil?
Cloned virus/worm can be detected Common signature on every copy Detect once, detect everywhere (DODE?)
If instead virus/worm is metamorphic Each copy has different signature Same detection may not work against every copy Provides DODE-resistance? Analogous to genetic diversity in biology
Effective use of metamorphism here is tricky!
Crypto Analogy
Consider WWII ciphers German Enigma
Broken by Polish and British cryptanalysts Design was (mostly) known to
cryptanalysts Japanese Purple
Broken by American cryptanalysts Design was (mostly) unknown to
cryptanalysts
Crypto Analogy
Cryptanalysis break a (known) cipher Diagnosis determine how an unknown
cipher works (from ciphertext) Which was the greater achievement,
breaking Enigma or Purple? Cryptanalysis of Enigma was harder Diagnosis of Purple was harder
Can make a reasonable case for either…
Crypto Analogy
What does this have to do with metamorphic software?
Suppose the good guys generate metamorphic copies of software
Bad guys can attack individual copies Can bad guys attack all copies?
If they can diagnose our metamorphic generator, maybe
But that’s a diagnosis problem…
Crypto Analogy
What about case where bad guys write metamorphic code? Metamorphic viruses, for example
Do good guys need to solve diagnosis problem? If so, good guys are in trouble
Not if good guys “only” need to detect the metamorphic code (not diagnose)
Not claiming the good guys job is easy Just claiming that there is hope…
Virus Evolution
Viruses first appeared in the 1980s Fred Cohen
Viruses must avoid signature detection Virus can alter its “appearance”
Techniques employed encryption polymorphic metamorphic
Virus Evolution - Encryption
Virus consists of decrypting module (decryptor) encrypted virus body
Different encryption key different virus body signature
Weakness decryptor can be detected
Virus Evolution – Polymorphism
Try to hide signature of decryptor Can use code emulator to decrypt
putative virus dynamically Decrypted virus body is constant
Once (partially) decrypted, signature detection is possible
Virus Evolution – Metamorphism
Change virus body Mutation
techniques: permutation of
subroutines insertion of
garbage/jump instructions
substitution of instructions
Virus Construction Kits – PS-MPC
According to Peter Szor:“… PS-MPC [Phalcon/Skism Mass-Produced Code generator] uses a generator that effectively works as a code-morphing engine…… the viruses that PS-MPC generates are not [only] polymorphic, but their decryption routines and structures change in variants…”
Virus Construction Kits – G2
From the documentation of G2 (Second Generation virus generator):
“… different viruses may be generated from identical configuration files…”
Virus Construction Kits – NGVCK
From the documentation for NGVCK (Next Generation Virus Creation Kit):
“… all created viruses are completely different in structure and opcode…… impossible to catch all variants with one or more scanstrings.…… nearly 100% variability of the entire code”
Oh, really?
How We Compare Two Pieces of Code
Opcode sequences Score
0 call1 pop2 mov3 sub
… m-1 m-1
… score = n-1 jmp average
% match
0 push 0 n-1 0 n-11 mov2 sub3 and
…
…m-1 retn
Program X
Graph of real matches
Program Y Program Y
(lines with length > 5)(matching 3 opcodes)Assembly programs
Program X
Graph of matches
Program X
Program Y
Virus Families – Test Data
Four generators, 45 viruses 20 viruses by NGVCK 10 viruses by G2 10 viruses by VCL32 5 viruses by MPCGEN
20 normal utility programs from the Cygwin bin directory
Similarity within Virus Families – Results
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0 50 100 150 200
Comparison number
Similarity score
NGVCK viruses
Normal files
Similarity within Virus Families – Results
NGVCK G2 VCL32 MPCGEN Normalmin 0.01493 0.62845 0.34376 0.44964 0.13603max 0.21018 0.84864 0.92907 0.96568 0.93395average 0.10087 0.74491 0.60631 0.62704 0.34689
Minimum, maximum, and average similarity scores
Similarity within Virus Families – Results
Size of bubble = average similarity
NGVCK
G2VCL32 MPCGENNormal
0
0.2
0.4
0.6
0.8
1
1.2
-0.2 0 0.2 0.4 0.6 0.8
Minmum similarity score
Maximum similarity score
NGVCK
G2
VCL32
MPCGEN
Normal
NGVCK Similarity to Virus Families
NGVCK versus other viruses 0% similar to G2 and MPCGEN viruses 0 – 5.5% similar to VCL32 viruses (43
out of 100 comparisons have score > 0) 0 – 1.2% similar to normal files (only 8
out of 400 comparisons have score > 0)
NGVCK Metamorphism/Similarity
NGVCK By far the highest degree of
metamorphism of any kit tested Virtually no similarity to other viruses
or normal programs Undetectable???
Commercial Virus Scanners
Tested three virus scanners eTrust version 7.0.405 avast! antivirus version 4.7 AVG Anti-Virus version 7.1
Each scanned 37 files 10 NGVCK viruses 10 G2 viruses 10 VCL32 viruses 7 MPCGEN viruses
Commercial Virus Scanners
ResultseTrust and avast! detected 17
(G2 and MPCGEN)AVG detected 27 viruses (G2,
MPCGEN and VCL32)none of NGVCK viruses detected
by the scanners tested
Virus Detection with HMMs
Use hidden Markov models (HMMs) to represent statistical properties of a set of metamorphic virus variants Train the model on family of
metamorphic viruses Use trained model to determine
whether a given program is similar to the viruses the HMM represents
Virus Detection with HMMs – Data
Data set 200 NGVCK viruses (160 for training, 40
for testing) Comparison set
40 normal exes from Cygwin 25 other “non-family” viruses (G2,
MPCGEN and VCL32) 25 HMM models generated and
tested
Virus Detection with HMMs – MethodologyTraining:
(1)Training set(160 files) (2) Training (4)
Threshold
(3)
Data Set
(1) Test set Normal programs(40 files) (40 files)
Other viruses(25 files)
Comparison Set
Classifying:
(3) Scoring
(1) Scoring LLPO > Threshold ?
HMM
Scores (LLPO) virus0 -2.0 virus1 -2.3 :
:
random0 -11.3 :
other0 -8.9
HMMProgram A
Virus Detection with HMMs – Results
Test set 0, N = 2
-160
-140
-120
-100
-80
-60
-40
-20
0
0 10 20 30 40
File number
Score (LLPO)
family viruses
normal files
Virus Detection with HMMs – Results
Detect some other viruses “for free”
Test set 0, N = 3
-180
-160
-140
-120
-100
-80
-60
-40
-20
0
0 10 20 30 40
File number
Score (LLPO)
familyviruses
non-familyviruses
normalfiles
Virus Detection with HMMs
Summary of experimental results All normal programs distinguished VCL32 viruses had scores close to
NGVCK family viruses With proper threshold, 17 HMM models
had 100% detection rate and 10 models had 0% false positive rate
No significant difference in performance between HMMs with 3 or more hidden states
Virus Detection with HMMs – Trained Models
Converged probabilities in HMM matrices may give insight into the features of the represented viruses
We observe opcodes grouped into “hidden” states most opcodes in one state only
What does this mean? We are not sure…
Detection via Similarity Index
Straightforward similarity index can be used as detector To determine whether a program belongs
to the NGVCK virus family, compare it to any randomly chosen NGVCK virus
NGVCK similarity to non-NGVCK code is small
Can use this fact to detect metamorphic NGVCK variants
Detection via Similarity Index
Threshold determination:Pairwise comparison Scoring
Virus V Subset of D (randomly (randomly
chosen) chosen)Virus 0Virus 1
Data set D :Virus X
Classifying:Scoring
Similarity score > Threshold ? Yes => family virus
Virus V No => not family virus
Similarity scores Virus 0 0.035 Virus 1 0.041 : : Virus X 0.189
Program A
Detection via Similarity Index
Experiment compare 105 programs to one
selected NGVCK virus Results
100% detection, 0% false positive Does not depend on specific
NGVCK virus selected
Conclusion
Metamorphic generators vary a lot NGVCK has highest metamorphism
(10% similarity on average) Other generators far less effective (60%
similarity on average) Normal files 35% similar, on average
But, NGVCK viruses can be detected! NGVCK viruses too different from other
viruses and normal programs
Conclusion
NGVCK viruses not detected by commercial scanners we tested
Hidden Markov model (HMM) detects NGVCK (and other) viruses with high accuracy
NGVCK viruses also detectable by similarity index
Conclusion
All metamorphic viruses tested were detectable because High similarity within family and/or Too different from normal programs
Effective use of metamorphism by virus/worm requires A high degree of metamorphism and
similarity to other programs This is not trivial!
The Bottom Line
Metamorphism for “good” Buffer overflow mitigation, BOBE-
resistance A little metamorphism does a lot of good
Metamorphism for “evil” For example, try to evade virus/worm
signature detection Requires high degree of metamorphism
and similarity to normal programs Not impossible, but not easy…
The Bottom Bottom Line
All-too-often in security, the advantage lies with the bad guys
For metamorphic software, perhaps the inherent advantage lies with the good guys
References X. Gao, Metamorphic software for buffer overflow
mitigation, MS thesis, Dept. of CS, SJSU, 2005 P. Szor, The Art of Computer Virus Research and
Defense, Addison-Wesley, 2005 M. Stamp, Information Security: Principles and
Practice, Wiley InterScience, 2005 M. Stamp, Applied Cryptanalysis: Breaking Ciphers
in the Real World, Wiley, 2007 W. Wong, Analysis and detection of metamorphic
computer viruses, MS thesis, Dept. of CS, SJSU, 2006
W. Wong and M. Stamp, Hunting for metamorphic engines, Journal in Computer Virology, Vol. 2, No. 3, 2006, pp. 211-229
Hidden Markov Models (HMMs)
state machines transitions between states have fixed
probabilities each state has a probability distribution for
observing a set of observation symbols states = features of the input data transition and the observation probabilities
= statistical properties of features can “train” an HMM to represent a set of
data (in the form of observation sequences)
HMM Example – the Occasionally Dishonest Casino
1: 1/6 0.05 1: 1/100.95 2: 1/6 2: 1/10 0.9
3: 1/6 3: 1/104: 1/6 4: 1/105: 1/6 5: 1/106: 1/6 0.1 6: 1/2
Fair Loaded
HMM Example – the Occasionally Dishonest Casino
2 states: fair/loaded The switch between dice is a Markov
process Outcomes of a roll have different
probabilities in each state If we can only see a sequence of rolls, the
state sequence is hidden want to understand the underlying
Markov process from the observations
HMMs – the Three Problems
1. Find the likelihood of seeing an observation sequence O given a model , i.e. P(O | )
2. Find an optimal state sequence that could have generated a sequence O
3. Find the model parameters given a sequence O
There exist efficient algorithms to solve the three problems
HMM Application – Determining the Properties of English Text
Given: a large quantity of written English text
Input: a long sequence of observations consisting of 27 symbols (the 26 lower-case letters and the word space)
Train a model to find the most probable parameters (i.e., solve Problem 3)
HMM Application - Results
Observation probabilities converged, each letter belongs to one of the two hidden states
The two states correspond to consonants and vowels
Can use trained model to score any unknown sequence of letters to determine whether it corresponds to English text. (i.e. Problem 1)
Note: no a priori assumption was made HMM effectively recovered the statistically
significant feature inherent in English
HMM Application - Results
Probabilities can be sensibly interpreted for up to n = 12 hidden states
Trained model could be used to detect English text, even if the text is “disguised” by, say, a simple substitution cipher or similar transformation
HMMs – The Trained Models
popretnpushjbrcljnbjadivadcrorshrrol
addsar
boundcmpsbretfmovxordecnotimul
movsbstosdlodswlodsdlodsbin
repemovsdfnstenv
cmcjns jle clc rcr fildout
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
observation probability
opcode
state 0
state 1
state 2
HMMs – Run Time of Training Process
5 to 38 minutes, depending on number of states N.
0
500
1000
1500
2000
2500
1 2 3 4 5 6 7
Number of states N
Training time (seconds)
500 iterations
800 iterations
HMMs – Run Time of Classifying Process 0.008 to 0.4 milliseconds, depending on N and number of opcodes
T .
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0 500 1000 1500
Length of observation sequence T
Scoring time (milliseconds)
N = 2
N = 3
N = 4
N = 5
N = 6