ICT Security: Testing methodology for Targeted Attack defence tools

ICT Security: Testing methodology forTargeted Attack defence tools

Luca MellaRelatore: Prof. Franco Callegati

Correlatore: Dott. Ing. Marco Ramilli

Tesi di Laurea in Progetto di Reti di Telecomunicazioni LMUniversity of Bologna, Scuola di Ingegneria ed Architettura

Campus di CeSeNA

March 27, 2014

Modern Threats

Modern Threats (in a Nutshell)

Cyber Attacks before 2005

I Pranksters, lone wolves

I Disruption, DoS, DDoS, . . .

I Identity theft, Cyber-crime

Modern Cyber Attacks

I Organized groups, heavyautomation, black markets

I Business models:I Pay Per Install (PPI)I Exploit as a Service (EAAS)

I Cyber-espionage, TargetedAttacks, APTs

Cyber-warfare and Cyber-espionage

I Steal secrets, intellectual propriety, projects

I Surveillance

I Sabotage

Luca Mella (University of Bologna) TA Test Method March 27, 2014 2 / 16

Modern Threats OAs and TAs

OAs and TAs

Opportunistic Attacks

I “Non-targeted-target policy”

I Steal accounts, passwords, credit cards, Bitcoin wallets, . . .

I Drive by downloads

Targeted Attacks

I ReconnaissanceI Gather information about target

I Weaponization and DeliveryI Prepare the weaponized malware and deliver.

I Installation and Command and ControlI Ensure access to target infrastructure

I ActionsI Lateral movement for achieving the goals of the attack.

. . . Similar process can be found in Cyber-Warfare taxonomies


Defense approaches against Modern Threats

Defense approaches against Modern Threats

An holistic approach. . .

I Traditional defensesI AVsI FW/NGFI Proxy, WAF

I Network and Host probesI Sandboxes

I “a way to separating running programs, contain their execution through a fullycontrollable environment”

I Eg. virtualization and emulation basedI Automated dynamic analysis

I Security Information and Event Management (SIEM)I DashboardI Alerting and ReportingI RetentionI Aggregation and CorrelationI Aid forensic analysis

I Computer Security Incident Response Team (CSIRT)Luca Mella (University of Bologna) TA Test Method March 27, 2014 4 / 16

Security System Testing

Security System Testing

Anti Malware Products Testing

I Anti Malware Testing Standard Organization (AMTSO)I Static tests

I EICAR test fileI Known sample-set

I Dynamic testsI Performance of the product is determined by the behaviour of the sampleI Execution environment determines sample behaviourI Dependency from external resourcesI Loss of reproducibility

Cloud-Based Products TestingI Continuous updates

I Reputation data

I Black-list, White-lists

I Threat data correlation

Cannot be frozen duringa test session!


Security System Testing Summing Up

Security System Testing: Summing Up

Summing Up

I EICAR test files is merely an “installation test file”

I Samples that “resemble” a malicious program are useless

I Known sample-sets

I Focus on malicious programs detection

Gap between what is tested andwhat is actually tackled

Part of the InfoSec community is aware of this problem, eg:

http://www.fireeye.com/blog/corporate/2013/10/

be-the-change-test-methodologies-for-advanced-threat-prevention-products.html


http://www.fireeye.com/blog/corporate/2013/10/be-the-change-test-methodologies-for-advanced-threat-prevention-products.html

http://www.fireeye.com/blog/corporate/2013/10/be-the-change-test-methodologies-for-advanced-threat-prevention-products.html

Testing Security Systems Against TAs

Testing Security Systems Against TAs

Vision

Test systems against modern threats

Goal

Lower the gap between what is tested and what is actually tackled

Mainstays

I Several sub-tests based on the TA kill-chain modelI ReconnaissanceI Weapon Delivery and Command and ControlI Actions

I Testing SystemsI Emphasis on Products w.r.t. System point of viewI Expectations from each component of the TA detection system to test

I Comparative TestsI Interferences


Testing Security Systems Against TAs Testing Reconnaissance

Testing Security Systems Against TATesting Reconnaissance

Testing Reconnaissance

I Test the ability to detect information gathering phase of the attack

I Need care when defining the sample-set

An example:I Want to network monitor product or web-analytics with some detection eng.

I Samples as network traffic samples

I Malicious samples via real information gathering

I Benign samples via traffic replay, traffic models, or real-user traffic


Testing Security Systems Against TAs Testing Reconnaissance

Testing Security Systems Against TATesting Delivery and Command and Control

Testing Delivery and Command and Control

I Detection of the act of spreading malware with the purpose ofcompromise hosts

I Observable delivery and unobservable deliveryI Samples Execution is required

I Unknown nature of the threatI Side effects in the sample-set definition

I Consider the sample creation


Testing Security Systems Against TAs Testing Actions

Testing Security Systems Against TATesting Actions and Lateral Movements

Testing Actions

I Detect lateral movements performed by the attacker after a successfulinfection

I Local actions performed inside the host

I Network actions involve network, eg. telnet sessions, loginbrute-force, network share access, RDP sessions, . . .

I Malicious Samples: sequence of actions with malicious intent.I Watch for the context of the host!

I Consider the expectations formulated in preliminary analysis


Testing Security Systems Against TAs Workflow

Testing Security Systems Against TA: Workflow











Case Study Preliminary and Test-Bed Activities

Case StudyPreliminary and Test-Bed Activities

Testing Two Systems

I Preliminary AnalysisI Formulate expectations

for each componentI Channels (HTTP)I Artifacts (malware

download, maliciouscomm.)

I Sub-tests to enable

I Test-bed ActivitiesI Deploy test-bedI Report possible

interferences, eg. If asystem might blockfurther communications

Figure : Test-bed architecture


Case Study Sample and Test Activities

Case StudySample and Test Activities

Sample Activities

I Collect SamplesI Two phase sample collection

strategyI Same week: collect, validateI Few hours before: re-collect,

find latest version of previouslycollected samples.

I Create SamplesI Real malicious program:

SpyWareI Evasion techniques

Test ActivitiesI Proceed with delivery phase

I Monitor each componentI PASS/FAIL/NA judgment for each

component

I Proceed with command andcontrol phase

I Monitor each componentI PASS/FAIL/NA judgment for each

component


Case Study Analysis Activities

Case StudyAnalysis Activities

Delivery Results

Command and Control Results

Analysis Activities

I Take into account expectationsand possible interferences.

I Formulate considerations fromSystem point of view

I All systems have not detected thecompletely new attack

I SYSTEM-1 performed better incommand and control detection

I Formulate considerations fromComponent point of view

I APT-DET-1 provides valuablecontribution especially in commandand control detection.


Case Study Analysis Activities

Conclusions

I Lower gap between what is tested and what is actually tackledI Modern, advanced, targeted threats in mindI Support real-word test-beds, context has central role

I Comparison test between systemsI Enable formulation of consideration from both system and product

point of view

I Complete and GeneralI Covers relevant attack stepsI Can also be applied in Gray-Box tests on already deployed security

systems

Further WorkI Specializations and further analysis of particular scenarios (eg.

drive-by, phishing mails, data-leak, . . . )

I Extension with Incident Response testing


:(){:|: &}; : :(){:|: &}; :

GRAZIE PERL’ATTENZIONE!


Technology

ICT Security: Testing methodology for Targeted Attack defence tools