Network Security:Network Security:Intrusion Detection Intrusion Detection

and Protectionand Protection

Photiou SavvasPhotiou Savvas

University of Cyprus

What is What is computer computer security ?security ?Security requirements of different system are different.Security requirements of different system are different.

University:University:

Papers, discoveries and work are available to public.Papers, discoveries and work are available to public. Protects the integrity and confidentiality of grades and other Protects the integrity and confidentiality of grades and other

data.data. Its shared resources must be open via the internet.Its shared resources must be open via the internet.

Military research organization:Military research organization:

All the work within the organization must remain secret.All the work within the organization must remain secret. Emphasizes confidentiality over integrity.Emphasizes confidentiality over integrity. None of its resources must be available over the internet.None of its resources must be available over the internet.

Security Security ComponentsComponents

RequirementsRequirements: “What do you want security to do for you?”: “What do you want security to do for you?”

Policy: Policy: “What steps do you take to reach the goal set out “What steps do you take to reach the goal set out above?”above?”

Mechanisms:Mechanisms: “What tools and procedures do you use to “What tools and procedures do you use to ensure the above steps are followed?”ensure the above steps are followed?”

FirewallsFirewallsThe role of a firewall is to deny or permit access to a The role of a firewall is to deny or permit access to a network based on the enforced policy.network based on the enforced policy.

Packet Filtering Firewalls.Packet Filtering Firewalls.Applies packet filters based on protocol type, on source and Applies packet filters based on protocol type, on source and destination address and on source and destination ports.destination address and on source and destination ports.

Application Gateway Firewalls.Application Gateway Firewalls.Every connection to a host outside of the internal network is Every connection to a host outside of the internal network is made through an application program called a proxy. made through an application program called a proxy.

Stateful Inspection Firewalls.Stateful Inspection Firewalls.Track the state of communication sessions and dynamically Track the state of communication sessions and dynamically open and close ports based on access policies.open and close ports based on access policies.

Therefore a firewall can implement policies that Therefore a firewall can implement policies that concern the perimeter of the protected network.concern the perimeter of the protected network.

The Role of an Intrusion Detection The Role of an Intrusion Detection System (IDS).System (IDS).

“ “ With so much advancement in hacking, if attackers try hard With so much advancement in hacking, if attackers try hard enough, they will eventually succeed in infiltrating the system. enough, they will eventually succeed in infiltrating the system. This makes it important to monitor what is taking place on a This makes it important to monitor what is taking place on a system and look for suspicious behavior. Intrusion detection system and look for suspicious behavior. Intrusion detection systems do just that. “systems do just that. “ A false positive occurs when the IDS reports an event of A false positive occurs when the IDS reports an event of

legitimate network activity as an intrusion.legitimate network activity as an intrusion. Likewise a false negative occurs when the IDS fails to detect Likewise a false negative occurs when the IDS fails to detect malicious network activity.malicious network activity.

As we employ heavier rules in the IDS we can detect more As we employ heavier rules in the IDS we can detect more variances of intrusion attempts but more false positives are variances of intrusion attempts but more false positives are probable.probable.If we employ lighter rules, we have less false positives but the If we employ lighter rules, we have less false positives but the system is easier to penetrate.system is easier to penetrate.

The security policy for the specific system must specify how The security policy for the specific system must specify how the IDS would perform.the IDS would perform.

Definition of IDSDefinition of IDS

An Intrusion detection system is a system that is used to An Intrusion detection system is a system that is used to detect inappropriate, incorrect or anomalous activity.detect inappropriate, incorrect or anomalous activity.

Can be host based or network based.Can be host based or network based.

Malicious activity can be classified as misuse if it originates Malicious activity can be classified as misuse if it originates from the internal network or intrusion if it originates from the from the internal network or intrusion if it originates from the external network.external network.

Most common approaches are pattern matching detection Most common approaches are pattern matching detection and statistical anomaly detection.and statistical anomaly detection.

Pattern Matching Pattern Matching DetectionDetection

Looks for a fixed sequence of bytes within each packet. Looks for a fixed sequence of bytes within each packet. To To filter traffic inspection the pattern is also usually associated filter traffic inspection the pattern is also usually associated with a particular service and source or destination port.with a particular service and source or destination port.

For example it looks for IPv4 packets that use TCP protocol, For example it looks for IPv4 packets that use TCP protocol, have destination port of 27015 and contain the string “abc” in have destination port of 27015 and contain the string “abc” in the payload.the payload.

Is straightforward and easy to deploy but…Is straightforward and easy to deploy but… Many attacks and protocols don’t always use well known Many attacks and protocols don’t always use well known ports.ports. If the matching pattern isn’t so unique a large number of If the matching pattern isn’t so unique a large number of false positivesfalse positives can occur.can occur.

Stateful Pattern Stateful Pattern MatchingMatching

Stateful packet matching adds to pattern matching by Stateful packet matching adds to pattern matching by searching for unique sequences that might be distributed searching for unique sequences that might be distributed across several packets within a stream.across several packets within a stream.

Is more specific that pattern matching but…Is more specific that pattern matching but… Is still vulnerable to false positives if the pattern isn’t unique Is still vulnerable to false positives if the pattern isn’t unique enough.enough. Slight modification of an attack can avoid detection.Slight modification of an attack can avoid detection.

Statistical Anomaly Statistical Anomaly DetectionDetection

Statistical anomaly detection detects activity that deviates Statistical anomaly detection detects activity that deviates from “normal” activity. It depends on the statistical definition of from “normal” activity. It depends on the statistical definition of normal and because of that is usually prone to a large number normal and because of that is usually prone to a large number of false positives.of false positives.

Intrusion detection based Intrusion detection based on Hidden Markov Modelon Hidden Markov Model

The Hidden Markov Model is a finite set of states each of The Hidden Markov Model is a finite set of states each of which is associated with a probability distribution. Transitions which is associated with a probability distribution. Transitions among the states are governed by a set of probabilities called among the states are governed by a set of probabilities called transition probabilities. transition probabilities.

In a particular state, an outcome or observation can be In a particular state, an outcome or observation can be generated according to the associated probability distribution. generated according to the associated probability distribution.

It is the outcome, not the state visible to an external observer It is the outcome, not the state visible to an external observer and therefore the states are “hidden” to the outside.and therefore the states are “hidden” to the outside.

Building a Hidden Markov Building a Hidden Markov ModelModel

The biggest challenge is to select the states that best The biggest challenge is to select the states that best characterize the system’s activity.characterize the system’s activity.

Usual observable outcomes are login events and system calls.Usual observable outcomes are login events and system calls.

Transtition Matrix

The IDS knows the initial state of the system. Then it The IDS knows the initial state of the system. Then it calculates the possible transitions and observable outcomes for calculates the possible transitions and observable outcomes for a series of steps.a series of steps.

If a series of observable outcomes matches the predicted If a series of observable outcomes matches the predicted behavior of the model, then the behavior is considered as behavior of the model, then the behavior is considered as normal, else it is considered abnormal.normal, else it is considered abnormal.

Weaknesses of an IDS Weaknesses of an IDS implementationimplementation

The IDS does not know the full range of behavior allowed by The IDS does not know the full range of behavior allowed by a particular protocol.a particular protocol.

The IDS does not know the exact expected behavior of each The IDS does not know the exact expected behavior of each host.host.

The IDS does not know the topology of the internal network.The IDS does not know the topology of the internal network.

These ambiguities can be exploited by an attacker to trick the These ambiguities can be exploited by an attacker to trick the IDS into assuming different activity that the actual. IDS into assuming different activity that the actual.

By manipulating the TTL field in the IP header the IDS By manipulating the TTL field in the IP header the IDS does not know which packet actually arrives at the end does not know which packet actually arrives at the end

hosthost

The IDS does not know how the end The IDS does not know how the end host would deal with the reception of host would deal with the reception of

overlapping packetsoverlapping packets

““The passive network intrusion detection The passive network intrusion detection systems can only effectively identify malicious systems can only effectively identify malicious flows when used in conjunction with an flows when used in conjunction with an interposed active mechanism”.interposed active mechanism”.

Traffic Normalization / Protocol Scrubbing

Active Networks

Traffic NormalizationTraffic Normalization

The normalizer’s job is to sit directly in the path of traffic into a site and patch up or normalize the packet stream to remove potential ambiguities so that the traffic seen by the intrusion detection system is guaranteed unambiguous.

How a normalizer treats How a normalizer treats some ambiguities of the IP some ambiguities of the IP

ProtocolProtocol

IPv4 HeaderIPv4 Header

Version:Version: A normalizer should only pass A normalizer should only pass packets with IP version fields which the NIDS packets with IP version fields which the NIDS understandsunderstands

Header Length:Header Length: It may be possible to send a It may be possible to send a packet with an incorrect header length field packet with an incorrect header length field that arrives at the end system and is that arrives at the end system and is accepted. However, other operating systems accepted. However, other operating systems or internal routers may drop the packet. If or internal routers may drop the packet. If the header length is less than 20 bytes or the header length is less than 20 bytes or exceeds the packet length it should be exceeds the packet length it should be discarded.discarded.

Don’t Fragment Flag:Don’t Fragment Flag: If DF is set and the If DF is set and the Maximum Transmission Unit (MTU) Maximum Transmission Unit (MTU) anywhere in the internal network is smaller anywhere in the internal network is smaller than the MTU on the access link to the site, than the MTU on the access link to the site, an attacker can deterministically cause some an attacker can deterministically cause some packets to fail behind the link. The packets to fail behind the link. The normalizer clears the DF flag.normalizer clears the DF flag.

Time To Live :Time To Live : The normalizer sets the TTL value The normalizer sets the TTL value greater than the largest path across the internal greater than the largest path across the internal site.site.

More Fragments / Fragment Offset:More Fragments / Fragment Offset: An An ambiguity arises if two incoming fragments ambiguity arises if two incoming fragments overlap each other and differ in their contents. overlap each other and differ in their contents. Internal hosts may resolve the ambiguity Internal hosts may resolve the ambiguity differently. The normalizer reassembles differently. The normalizer reassembles incoming fragments before forwarding them. If incoming fragments before forwarding them. If needed fragments them again.needed fragments them again.

Stealth port Scans: The normalizer transmits an Stealth port Scans: The normalizer transmits an ACK packet behind every RST packet it forwards ACK packet behind every RST packet it forwards out of the siteout of the site

Attacks On the Attacks On the Normalizer:Normalizer:

Stateholding attacks:Stateholding attacks: The attacker tries to The attacker tries to consume the normalizer’s memory by causing it consume the normalizer’s memory by causing it to instantiate too many states. Common to instantiate too many states. Common stateholding attacks are:stateholding attacks are:

SYN flooding: The attacker floods SYN packets SYN flooding: The attacker floods SYN packets so that the normalizer instantiates states for each so that the normalizer instantiates states for each connection.connection.

ACK flooding: If the normalizer restarted ACK flooding: If the normalizer restarted recently by receiving an ACK packet it recently by receiving an ACK packet it instantiates state because the packet might be instantiates state because the packet might be part of a connection that initiated before the part of a connection that initiated before the restart. restart.

Initial window flooding: The attacker sends a Initial window flooding: The attacker sends a SYN to an internal host, receives a SYN-ACK and SYN to an internal host, receives a SYN-ACK and then floods data without sending ACK. The then floods data without sending ACK. The normalizer would buffer that information to normalizer would buffer that information to prevent inconsistent retransmissions.prevent inconsistent retransmissions.

CPU overload attacks:CPU overload attacks: An attacker attempts to An attacker attempts to overloadoverload

The CPU on the normalizer. Such attacks can The CPU on the normalizer. Such attacks can cause thecause the

normalizer to forward packets at a slower rate normalizer to forward packets at a slower rate than itthan it

normally would, but cannot cause an ambiguity to normally would, but cannot cause an ambiguity to pass.pass.

Usual policy of the normalizer to withstand such attacks:

The normalizer knows whether or not it’s under attack by monitoring the amount of memory it is consuming. If it’s not under attack it can instantiate whatever state it needs to normalize correctly. If it believes it is under attack, it takes a more conservative strategy that is designed to allow it to survive, although some legitimate traffic will see degraded performance.

Active Network-Based Active Network-Based Intrusion Detection and Intrusion Detection and

Response systemsResponse systems Active networks carry executable code

within packets which is executed at network nodes such as hubs, bridges, switches, routers, gateways.

Communication is achieved using the Active Network Encapsulation Protocol (ANEP)

Active Network-Based Intrusion Detection System Design

ReferencesReferences

Network Intrusion Detection: Evasion, Traffic Normalization and End-to-End Network Intrusion Detection: Evasion, Traffic Normalization and End-to-End Protocol Semantics. Mark Handley and Vern Paxson.Protocol Semantics. Mark Handley and Vern Paxson.

Intrusion Detection Based On Hidden Markov Model. Qing-Bo Yin, Li-Ran Shen, Intrusion Detection Based On Hidden Markov Model. Qing-Bo Yin, Li-Ran Shen, Ru-Bo Zhang, Xue-Yao Li, Hui-Qiang Wang.Ru-Bo Zhang, Xue-Yao Li, Hui-Qiang Wang.

A Hidden Markov Models-Based Anomaly Intrusion Detection Method. Ye Du, A Hidden Markov Models-Based Anomaly Intrusion Detection Method. Ye Du, Huiqiang Wang and Yonggang Pang.Huiqiang Wang and Yonggang Pang.

What Is Computer Security. Matt BishopWhat Is Computer Security. Matt Bishop

Protocol Scrubbing: Network Security Through Transparent Flow Modification. Protocol Scrubbing: Network Security Through Transparent Flow Modification. David Watson, Matthew Smart, G. Robert Malan, Farnam Jahanian.David Watson, Matthew Smart, G. Robert Malan, Farnam Jahanian.

An Active Network–Based Intrusion Detection And Response Systems. Han-Pang An Active Network–Based Intrusion Detection And Response Systems. Han-Pang Huang, Chia-Ming Chang.Huang, Chia-Ming Chang.