Passive Network Discovery Systems

Martin Roesch

The Current State of Intrusion Detection

3

What is NIDS?What is NIDS?

A network intrusion detection system monitors traffic in real time and alerts when suspicious activity is detected

4

Why is NIDS Important?Why is NIDS Important?

Access control (firewalling) is only part of the security solution, you need network monitoring technology (Defense in Depth) to secure your enterprise effectively

5

Complementary Security MeasuresComplementary Security Measures

Network IDS complements and augments firewalls and other security infrastructure Provides “assurance” in case firewall is bypassed or

misconfigured Protects against insider threats Affords forensic analysis against changing

environments and threat vectors

6

What’s Wrong with NIDS?What’s Wrong with NIDS?

IDS is not working as well as hoped Industry has been its own worst enemy for years, over-hyped and

under delivered What are intrusion detection systems really for?

Awareness - How is my network working? How is my security infrastructure working?

Analysis - When things go wrong, what happened and how can I prevent it from happening again?

Classic IDS does not protect networks, it allows people to understand how/if their protection is working and what happened when it fails

7

Problems With IDS ImplementationsProblems With IDS Implementations

Implementational Issues Some assembly required

IDSes traditionally require a great deal of tuning for the environment they’re monitoring

Most NIDS solutions are lacking a credible data management solution

Tuning is an ongoing process “What do you mean you don’t know IP?!”

Proper training is required to get value from an IDS Interpreting the output from an IDS requires a great

deal of expertise System policy management

Managing the distributed sensor detection configuration is a manual process

8

Problems With IDS ImplementationsProblems With IDS Implementations

Conceptual Problems Detection Failures

Ptacek & Newsham paper, classic guide on how to defeat IDS by taking advantage of ambiguities that IDS cannot resolve

– Fundamental problem with the approach used by many (all?) IDSes Data management

Once I’ve got my IDS tuned and my staff trained, I run into the next problem: data management

IDS generates huge amounts of information, this information must be managed

Data management is a very hard problem as well (on the order of difficulty with IDS in the first place)

Data coming from IDSes is subjective for a variety of reasons, users are left to add context

The Missing Link

10

Intrusion detection systems operate in a contextual vacuum No knowledge of the network topology

No knowledge of the network’s assets

No knowledge regarding asset criticality

Effective prioritization is impossible without context Priority is in the eye of the beholder

Automated response is extremely risky

100% Effective detection is impossible without context IDS must guess about network topology and composition, making

assumptions frequently

Mistaken assumptions lead to false positives or false negatives

If the attacker has more information about the target than the NIDS, this can be leveraged

What you don’t know can kill youWhat you don’t know can kill you

11

Example: The Linux web server cannot be vulnerable to CodeRed There was a valid attack on the wire but it wasn’t critical or relevant in

this context This isn’t a false positive or false negative but it gets assigned a default

priority (e.g. critical) for the event type instead of in context with the target that was attacked (to coin a term, “nontextuals”)

Thousands of these a day dilute the value of the of the data from IDS Remember: usability of the information is the key to a useful IDS

Linux WebServer

The Internet CodeRedAttack

•••IDS

CodeRedAttack!!

The Contextual Vacuum: PriorityThe Contextual Vacuum: Priority

12

Contextual Vacuum: Lack of Host ContextContextual Vacuum: Lack of Host Context Hosts (OS IP stacks) process packets differently Overlaps

Duplicates Re-transmissions Configuration options

If the attacker knows the OS being attacked and the NIDS doesn’t, evasion can result

COBEM NTEN T!I AIncoming overlapping packets:1. A hacker introduces an intentional overlap in the packet stream

AD

COBEM NTENT!I A ADAccept both

COBEM NTENT!I A DAccept first

COM NTENT!I A DBAccept neither

COADM NTENT!I A BAccept last

2. The IDS/IDP processes the packets applying a ‘general’ case that may differ dramatically from the target

With numerous possible interpretations:

13

Contextual Vacuum: Lack of Network ContextContextual Vacuum: Lack of Network Context

Target

The Internet

Firewall/IPS

•••Router

•••Router

Router•••

•••IDS

ANATOMYSTACK

ANATOMYSTACKTTL=3

ANATOMYSTACKTTL=2

ANATOMYSTACKTTL=0

OMYS

•••

TTL=1

AN ATTACK

TTL=1

Session content can change downstream TTL (Time-To-Live) expiration enable IDS/IDP evasion MTU (Maximum Transfer Unit) policy variations enable IDS/IDP evasion Knowledge of topology is critical for proper traffic analysis

14

How Can We Solve this Problem?How Can We Solve this Problem?

Context needs to be driven into network intrusion detection if it is going to get better

What elements of context are needed? Network context

Topology Host Context

Host OS Host Services

Exposure Context Vulnerability classes available against the network

15

Current Tools for Building Context Current Tools for Building Context

Active scanners Intermittent picture of network profile

Laptops are frequently disconnected from the network Many machines run more than one operating system Compromised servers are easily hidden from active scanners

Limited scope Not all protocols Not all ports Not all assets

Strong potential for service disruption Consumption of network bandwidth Conclusions are binary in accuracy, either 100% right or

100% wrong

Host-based technologies Cannot detect the unknown host or service Impose significant administrative burdens

16

The Ideal for Building ContextThe Ideal for Building Context

Passive network discovery systems (PNDS) are the only workable approach All network participants are observed

All protocols All ports All assets

Information is persistent Real-time All of the time

Many techniques can be leveraged and combined Packet analysis Flow analysis Protocol analysis Confidence model

No disruption of network operations Minimal ‘moving parts’

17

Vulnerability AnalysisVulnerability Analysis

VA by inference Knowledge about the host and its profile is

immediately associated with knowledge about vulnerabilities, exploits, and remediation processes

No packets are used to probe targets on the network, purely passive

Passive approach allows for constant vulnerability monitoring

Necessary to understand the exposure context Confidence model is more appropriate to

improving NIDS

18

Real-time Change DetectionReal-time Change Detection

New network assets (and vulnerabilities) Laptops Servers Rogue devices

Wired Wireless

Unauthorized users New network services (and vulnerabilities)

Ports Protocols Services

Policy violations Devices Protocols Operating systems Services Applications

Essential for understanding possible impact of attacks

Benefits of Passive Network Discovery

Systems

20

IDS: Without ContextIDS: Without Context

21

IDS: With ContextIDS: With Context

Provide host and network context to the IDS Target-based IDS!

PNDS

22

Event->Vulnerability/Change CorrelationEvent->Vulnerability/Change Correlation

Prioritization based on potential impact Events that correlate to nothing are not that interesting Events correlating to vulnerabilities are more interesting Events correlating to vulnerabilities and then affecting

change are highly interesting

Tiered prioritization Relevance Vulnerability Asset Sensitivity Attack Effectiveness

23

Automated TuningAutomated Tuning

Dynamic implementation of security policies Protocols Operating systems Services Applications

Protect the network instead of just trying to detect random attacks!

24

Eliminate False Positives/NegativesEliminate False Positives/Negatives

Model traffic in the IDS/IPS in exactly the same way as the end host.

HostProfiles

RNAEvents

RNARepository

TCP StateMachine (stream

reassembly)

OS/Version n0OS/Version

IPDefragmentation

TCP StateMachine (stream

reassembly)

OS/Version n1OS/Version

IPDefragmentation

…

Multi-ProtocolSession

Acquisition

NetworkTraffic

(packets

ProtocolDecoding

ProcessMethod

Rules-BasedInspection

Network Hosts

=

25

Enable Contextual ResponseEnable Contextual Response

IDP technologies have many alternatives for response Alert only Update policy (firewall, router, etc.) Block Session Block Traffic (in-line filtering)

Context allows target-specific response(s)

Web Server

Commerce Server

EmployeeDatabase

The Internet

AlertUpdate

AlertOnly

AlertUpdateBlock

Target?

Response Processing Module

Conclusions

27

The Concept of NID Needs to EvolveThe Concept of NID Needs to Evolve

Algorithms are not enough False positive picture has not improved

dramatically in the past 10 years Protecting the packets/protocols is a broken

model

28

PNDS Are the Right AnswerPNDS Are the Right Answer

Vulnerability scanners still solve problems, they just don’t solve this one very well

We cannot expect to provide accurate intrusion detection in environments where attackers have better information about the targets than the defenders

PNDS address all the problems of context generation in a way that is appropriate for large, highly changeable environments

First commercial PNDS will be available in December (from Sourcefire)

Questions & Answers