Cortex Program Status Brief Christopher Geib July 12th 2005

Cortex Program Status Brief

Christopher GeibJuly 12th 2005

2 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt

Self Regeneration is not enough

• Systems must be designed that are capable of response and regeneration after deliberate attack and catastrophic failures.

• However, response and regeneration must be sensitive to…- the mission that is being executed,

What tools and services are critical for the current mission goals? What tools and services are not critical for the current mission goals?

- and the lessons learned from the previous failure. What features of the protocol were exploited in the attack? Have features of the domain changed?

• Are there new kinds of connections that should be blocked?• Are some kinds of attacks more common than we thought?

Mission Aware Planning and Learning


Program Technical Objectives

• Prove the viability of automatically synthesizing a meta-level controller for model-based, system response and regeneration.

• Such regeneration control systems must have two critical capabilities:- planning that is sensitive to the system mission,

How much of the systems resources should be committed or held in reserve?

Planning conditional responses to known threats. Trading off commitments for mission critical objectives.

- and learning to prevent similar failures in the future. Patch the services that were exploited Modify the mission model to capture changes to the world.

Meta Level Planning and Learning


Existing Practice

• Automated system rebooting will bring elements of the system back on line after an attack, but these are limited:- Simple scripts that are not sensitive to the mission model.

- Can’t make trade offs between competing needs.

- Single system reboot.

• Learning of exploits and diagnosing root causes of the failure is handled offline by system experts.- Conclusions are not captured at the mission model level.

- Often performed somewhere else.

- Slow and requires significant expertise.

Done By Hand If At All


Talk Outline

• What is the mission? - Mission model

- Conops for Cortex within this Model

• What is our technical approach?- Current system design and use cases

- Thrust areas Learning Planning

• What progress have we made?• How are we performing against our metrics?• Conclusions

What Is The Mission?


Mission Model Motivation

• Critical Problem Features- Real world applicability and importance.

- Military relevance clear.

- Challenging complexity exceeds simple solutions.

- Multiple mission phases with differing objectives requiring differing responses to attack.

- Easy to find cyber attack methods within the literature.

• Daily mission planning cycle - Based on DARPA Cyber Panel Grand Challenge Problem

- Defense of an operations critical MySQL database for mission planning. Direct access Apache/php mediated access


Daily Mission Cycle

1: High Level Planning by Commander

2: Detailed Planning by

Naval Operations

3: Scheduling & Task Orders by

Naval Operations

4: Task OrdersSent to Ships

5: ScheduledMaintenance

Ongoing: BattleDamage Assessment

from other ships

17:00-19:45 20:00-22:10

08:00-11:20

12:00-12:05

15:00-16:00


Database and Web Server Host

Mission Domain System Architecture

Apache WebServer

PHPinterpreter

MySQLPlanning dB

StaticContent(.html)

DynamicContent(.php)

SQL

https

http

MySQL_BRP_Plan

radiolink

(Blueridge LAN)

(Planning LAN)

FW


Cortex Role in this Domain

• Protect the MySQL database:- Guarantee availability- Guarantee data integrity

• Making use of:- Redundant “taste tester” architecture- CIRCA planning for response and resource use- Learning based on active experimentation

• Cortex control capabilities:- Control query replication.- Manage tasters (which is lead-taster, building new ones)- Control access to DB (block known exploits)- Invoke learning module

What Is Our Technical Approach?


New Technology

• What is truly new?- Automated synthesis of meta-level controllers based on

decision theoretic tradeoffs among mission requirements for response and regeneration.

- Active experimentation based learning to prevent zero-day exploits and extend our mission model.

• How much will it improve?- New capability in some cases

Mission-based cost benefit trade offs and dynamic reallocation of resources after system failure.

- In others automating a process currently done by hand. Automated learning/generalization of exploits can’t increase the

runtime.


CIRCA Planning: Generalized Semi-Markov Decision Processes

• Most existing decision-theoretic planning systems are based on the Markov Decision Processes and have difficulty handling multiple, asynchronous events.

• GSMDP provides the most natural framework for this purpose.

• CIRCA (Cooperative Intelligent Real-Time Control Architecture) is the first GSMDP planner.

Uses the decision-theoretic principle of maximizing expected utility (e.g. to trade off performance against safety).

Uses rich stochastic models of world, transitions, actions, and time to construct best defense plans.

Does not hand-build, but automatically synthesizes plans and can thus adapt to defend against combinations/mutations of existing exploits based on the mission model.


Exploratory Experiments With New Heuristics

1870

1880

1890

1900

1910

1920

1930

1940

1950

100 1000 10000 100000 1e+06 1e+07

Exp

ect

ed

Utilit

y

Time (ms)

OriginalNew

Optimal plan found inonly 1% time.


Active Learning Approach

• We know an attack succeeded, but we don’t know why.- Need an educated guess as to culprits, then validate

• We want to explore the possibilities.- Model normal mission traffic according to several axes of

variability Score each attack according to these axes Experiment for most suspicious values

Build Model of Normal Traffic

Use model to identify suspicious elements

ExperimentHistorical(Normal)Queries

Attack Query

Learner

Blocking Rules

Model of

Normal


Axes of Variability (Definition)

• The different ways an attack (e.g. to MySQL) might be formulated.- Provided a priori by a domain expert

• Query content- Word order (e.g. some permutations cause MySQL to crash)

- Binary machine instructions

- Unusual payload (e.g. unix commands, registry keys, database administrative commands)

• Query length (single/multiple terms)• Resource consumption patterns• Probing (e.g. password guessing)• Session-wide (multiple queries)


Axes of Variability (detectors)

• We currently detect: - Text Queries:

String length: overall query length SQL parser: term length (table, col, row), number of terms (table, row,

col) Word order: if there are no other suspicious elements in the attack

- Packet Content (Using hex values of the packet): Command type Joint probability: each command has different expected byte patterns

• Plausible to design new or better detectors:- String content (e.g. look for hex)- Unusual payload (parser, keyword filter, content-filtering)- Session-wide (frequent patterns analysis)- Word order (patterns analysis)- More Joint probability distributions (e.g. strings may generally

be long, but passwords are short)


Modeling Mission Traffic

• Model normal traffic- Domain expert creates measures for each axis of variability (or combinations

thereof)- Currently stored as histograms of the computed values- e.g.

• Compare attack to the model- Compute a “suspicion score”: how unusual is this attack for this axis of

variability- Score = (value – μ) / σ- e.g. add “32” to above histogram;

score is 12.6

• Experiment (Active Learning)- Sort the suspicion scores.- Experiment in the “most” suspicious axis first.

Test hypothesis of culprit Discover the boundary conditions


TastersTastersTasters

Replicator

Delete tasters

Create tasters

Switch Tasters

Replicate queries

Heartbeat Status

RTSReplicate

Switch Tasters

Rebuild Tasters

Send to Learning

.

AMP

CSM

Once per phase

Proxy (Dexter)Block known bad queries

Taste test

Log results

Master DBQuery

LearnerRead Training Data

Experiment

Generate Rules

Normal Query

Cortex Demo Architecture and Use Cases

New AbilitySince January Demo


TastersTastersTasters

Replicator

Delete tasters

Create tasters

Switch Tasters

Replicate queries

Heartbeat Status

RTSReplicate

Switch Tasters

Rebuild Tasters

Send to Learning

.

AMP

CSM

Proxy (Dexter)Block known bad queries

Taste test

Log results

Master DBQuery

LearnerRead Training Data

Experiment

Generate Rules

Attack gets throughAttack is blocked

Cortex Demo Architecture and Use Cases


Risks and Mitigations

• Search space for the controller is large.- Mitigation: Our work is focused on heuristics that are

solving these problems without covering the whole space.

- Mitigation: Plans are built offline before system commission.

- Mitigation: Possible to build plans that provide safe reduced functionality states to move to while regenerating.

• Identification of a covering set of axes of variability and experimentation methods.- Mitigation: Fixed protocols have limited degrees of freedom

making it easier to enumerate.

- Mitigation: Axis only has to be identified once .

• “Binary poisons”- Mitigation: Don’t eat the second half of the poison.

- Mitigation: Use program diversity tools to push some code level violations that would be binary poisons into our space.

What Progress Have We Made?


Current Demo

• Complete system protecting MySQL with normal background traffic for our mission model.

• All three end-to-end use cases.- Integration of CIRCA planning for single mission phase.- Learning via active experimentation.

• Two different attacks in Mission Model Phase 1- Both kill MySQL 3.23.49 server but are very different.

Password buffer-overflow (BID 8590)• Exploits the lack of bounds-checking on the password field for a user.

Binary attack against COM_TABLE_DUMP command (BID 6368)• Exploits a casting vulnerability of signed integer values. • Sends a negative value as one of the string length parameters to the

command, and corrupts internal MySQL memory.


Major Accomplishments Since Last Meeting

• Infrastructure- Mission model defined.- 2 Specific attacks identified and implemented.- Background traffic generators built.- Improved & new GUIs developed.- Extending to Apache/PHP server (in progress).- Identifying Apache exploits (in progress).

• Learning - Design and Implementation of learning architecture and active-

experimentation approach. Identification of axes of variability. Traffic modeling.

- Successful testing against 2 very different attacks.

• Planning- Successful exploratory experiments on new planning heuristics.

How Are We Performing Against Our Metrics?


Program Level Metrics

• Correctly diagnosing root cause 10% of attacks.- System correctly diagnoses all of the attacks covered by an

identified axis of variability.

- Question becomes one of coverage of the axes of variability relative to the set of attacks.

• Correctly responding to 5% of diagnosed attacks- CORTEX plans controllers that respond to all (100%) of the

possible attacks enumerated within the mission model.

- Learning blocking rules for all (100%) of the diagnosed attacks.


Project Metrics

• Time to synthesize optimal controller (measured)- Experiments have cut the time to find the optimal plan for

this problem by >90%. Continuing experiments in other problems to identify efficacy of the method.

• Accuracy of the rules learned (measured)- Anecdotal analysis suggests no false positives; performing

measurements against traffic model.

• Overhead added to normal processing (measured)- Anecdotal evidence suggests this is not too bad, but

awaiting final system configuration to measure.

• Improved throughput from increased mission specific system availability. (measured)


Expected Major Achievements

• Prove the viability of automatically synthesizing a meta-level controller for model-based system response and regeneration.- Limited impact on current users

- Over all improved system throughput from increased system mission sensitive availability.

- Demonstrate system scalability in two mission phases

• Demonstrate system planning for at least two different mission phases with significantly different requirements and responses.

• Demonstrate the viability of learning zero-day exploits within well understood protocols for at least two different protocols.

Significant Research Results


SCHEDULE

CORTEX – Mission-Aware Closed-Loop Cyber Assessment and Response

• System Reference Model drives intrusion assessment, diagnosis, and response.

• Automatically search for response policies that optimize tradeoff of security against mission ops.

• “Taste-tester” server redundancy supports robustness and learning from new attacks.

• High confidence intrusion assessment and diagnosis.

• Pre-planned automatic responses to contain and recover from faults and attacks.

• Automatic tradeoffs of security vs. service level & accessibility.

• Learns to recognize and defeat novel attacks.

Computing services

Active Security ControllerExecutive

Controller Synthesis ModuleNetworks, Computers

Attacks, intrusions

IMPACT

NEW IDEASSecurity Tradeoff Planner

Scyllarus Intrusion

Assessment

Demos:Thin slice

demo

Jan 05 Jul 05

Learning Demo

Dec 05

Mission Aware Learning and

Response Demo

May 05

PI Meeting Demo

End

Come See the Demo

Documents

Cortex Program Status Brief Christopher Geib July 12th 2005