Maturing and Specializing: Incident Response Capabilities ...malicious events from nonevents, and 45% cited lack of visibility into events across a variety of systems and domains as

A SANS SurveyWritten by Alissa Torres

Advisor: Jake Williams

August 2015

Sponsored by AlienVault, Arbor Networks, Bit9 + Carbon Black,

Hewlett-Packard, McAfee/Intel Security, and Rapid7

Maturing and Specializing: Incident Response Capabilities Needed

©2015 SANS™ Institute

Hackers used to break into a system, steal as much data as possible and get out,

without worrying about detection. Today, however, they have learned to be patient,

harvest more data, and cause significant security and financial effects. Because of this,

organizations must detect and respond to incidents as quickly, efficiently and accurately

as possible.

The length of dwell time (the time from the attacker’s initial entry into an organization’s

network to the time the intrusion is detected) correlates most closely to the total cost

of a breach. The longer an attacker has unfettered access on a network,

the more substantial the data loss, severity of customer data theft and

subsequent regulatory penalties.

Of the 507 respondents to qualify and take the SANS 2015 Incident

Response Survey, 37% cited their the average dwell time as less than

24 hours, while 36% of organizations took 24 hours or less to remediate

real breaches. However, 50% took two days or longer to detect breaches,

7% didn’t know how long their dwell time was, another 50% took two

days or longer to remediate and 6% didn’t know. This represents a slight

improvement over our 2014 survey, in which 30% remediated breaches

in 24 hours or less, while 17% took one to two days to remediate, 51%

took more than two days to remediate and 6% took three months or

longer.

These and other results of the 2015 survey show that incident response

(IR) and even detection are maturing. For example, although malware

is still the most common underlying reason for respondents’ reported

incidents, 62% said malware caused their breaches, down from 82% in

2014. Data breaches also decreased to 39% from 63% last year. Such results hint that

malware prevention and other security technologies are working in an increasingly

complex threat landscape.

The shrinking window of response time, along with more automated tools and—just

as important—the specialized job titles to support the IR function are all indicators

of this maturation. Now for the bad news: Organizations are short on the skills and

technologies they need for full visibility and integrated response.

SANS ANALYST PROGRAMMaturing and Specializing: Incident Response Capabilities Needed1

Executive Summary

report an average dwell time of 24 hours or less, with 23% reporting two to seven days

spend an average of 24 hours or less to remediate an incident, and 28% remediate in two to seven days

66% cited a skills shortage as an impediment to effective IR

45%

37%

lack visibility into events across a variety of systems and domains

are unable to distinguish malicious events from nonevents

36%

37%

Executive Summary (CONTINUED)


In the survey, 37% of respondents said that their teams are unable to distinguish

malicious events from nonevents, and 45% cited lack of visibility into events across

a variety of systems and domains as key impediments to effective IR. Together, these

answers suggest the need for more precise conditions for security information and event

management (SIEM) alerts, as well as the need for more specialized IR skills.

Skills, while in demand, are also hard to come by, with 66% of survey takers citing a

skills shortage as being an impediment to effective IR. Another 54% cited budgetary

shortages for tools and technology, 45% lack visibility into system or domain events,

41% lack procedural reviews and practice, and 37% have trouble distinguishing

malicious events from nonevents.

Immature IR teams do not have the time or expertise to identify the initial entry of an

attacker into the network nor fully scope the attack for successful remediation. This

points to a “cleaver-like” approach to response, with 94% of respondents using the wipe

and reimage method of remediation. Even this is not always effective. As the recently

discovered Duqu 2.0 attacks demonstrate,1 advanced attackers count on their ability to

reinfect machines at will. Wiping and reimaging individual machines without mitigating

the full compromised system is certain to be a losing strategy.

Overall, these results reveal an increasingly complex response landscape and the need

for automation of processes and services to provide both visibility across systems and

best avenues of remediation. These issues, along with best practices and advice, are

discussed in the following pages.

1 https://securelist.com/files/2015/06/The_Mystery_of_Duqu_2_0_a_sophisticated_cyberespionage_actor_returns.pdf

About the Survey Respondents


The organizations in this survey are diverse in industry type, geographic location and

size of employee base, providing an excellent cross-section of IR capabilities as they exist

in companies today.

Size and Regions

The respondent pool includes a varied distribution of company size: 26% work for

companies with more than 20,000 employees and contractor staff, and 20% are from

companies of 500 employees or less (Figure 1).

Most (81%) of respondents’ organizations have a presence in the United States, with

Europe being the second most cited region with 33%. Overall, respondents represented

14 regions and countries, with many coming from global organizations.

Type of Industry

Government, technology and financial services were the top three sectors represented

in this survey, with 20%, 19% and 17% of the response base, respectively. Education

and manufacturing were each represented by at least 7% of respondents, while

just under 6% came from health care/pharmaceuticals. Energy/utilities, retail and

telecommunications were each represented by less than 4% of respondents. “Other”

write-in responses include aerospace/defense, chemical engineering and fast food.

How large is your organization’s workforce, including both employees and contractor staff?

500–

1,99

9

10,0

00–1

4,99

9

Gre

ater

than

20,

000

100–

499

5,00

0–9,

999

15,0

00–1

9,99

9

Few

er th

an 1

00

2,00

0–4,

999

Figure 1. Organization Size

30%

25%

20%

15%

10%

5%

0%

About the Survey Respondents (CONTINUED)


Roles/Responsibilities

Only 5% of respondents identified themselves as belonging to an IR/forensics consulting

firm. This indicates more organizations are bringing these types of skills in-house,

particularly as we look at the progress made over the past year in organizations creating

a dedicated in-house IR team. Last year, 59% of respondents had a dedicated team, while

73% reported having a team this year.

Results also reveal growing specialization in IR-related titles. Just over 9% of respondents

consider themselves specifically as incident responders, with others calling themselves

intelligence analyst, CERT team leader, incident/problem manager, IT security architect

or engagement manager in the write-in responses under the “Other” option. This

suggests that professionals with highly specific skill sets are filling niche roles on IR

teams. Increased specialization is typically a sign of maturation of an industry, a strong

progressive indicator for the IR profession as a whole. See Figure 2.

In a cursory search through open job requisites for security analysts, descriptions of

duties and responsibilities varied widely, as did the level of required experience for the

position. From assigned duties, such as being a member of a Tier 1 security operations

center (SOC) with responsibility for continuous monitoring, documentation and

reporting of incidents, to being a highly specialized technical expert who develops

signatures and countermeasures based on adversary tactics, techniques and procedures

(TTPs), the security analyst title is used as a catchall in the industry to describe a role with

a variety of duties and responsibilities.

What is your primary role in the organization, whether as an employee or consultant?

Figure 2. Many Roles Involved in Response

Security analyst

Security manager/Director/CSO/CISO

IT manager/Director/CIO

Incident responder

Other

System administrator

Digital forensics specialist

Compliance officer/Auditor

Network operations

Security operations center (SOC) manager

Help desk agent/Technician

Investigator

Percentage of respondents’

organizations having a dedicated IR team

73%

TAKEAWAY:

With increased specialization,

it becomes more important

to understand what each

member of an IR team

does. When reviewing the

experience of an employee

or a candidate for a position,

don’t rely on the title the

individual had. Instead, look

at the specific duties he or she

performed.

The majority (84%) of respondents report that their organization has experienced at

least one incident over the past year, with 18% experiencing more than 100 incidents.

Of those, 50% resulted in at least one real data breach: 9% say their investigations

resulted in only one critical incident, 25% say that that detection led to actual breach

investigations in two to 10 instances, and 6% report their investigations resulted in 11–

25 breaches, with just over 10% finding more than 25 actual breaches. Interestingly, the

majority of those who experienced two to 10 breaches started with two to 10 incidents.

See Figure 3.


Eyes on the Ground

Figure 3. Incidents Detected Compared to Actual Breaches Experienced

Breaches in the last 12 months

8.9%

6.3%

3.4%

3.1%2.6%1.3%

24.7%

1

1 or More Breach

2–10

11–25

26–50

51–100101–500500+

Incidents responded to in the last 12 months

5.3%

13.9%

7.6%

7.4%

8.4%

9.7%

31.2%

8.2%

8.4%

1

1 or More Incidents

2–10

11–25

26–50

51–100

101–500

500+

None

Unknown

Number of Incidents that Resulted in 2–10 Breaches

1 26–502–10 51–10011–25 101–500 500+

40%

30%

20%

10%

0%

Eyes on the Ground (CONTINUED)


These percentages show a decrease in actual breaches compared to 2014 results. In

2014, 61% experienced a serious breach, 18% did not and 21% did not know whether

they had experienced a breach. This year, 34% said they had no breaches (as opposed to

18% last year), and there were fewer unknowns (16% as opposed to 21%).2

One possible explanation for the notable decrease in critical breach incidents could

be the increase in automated IR tools. As we will see in the review of IR technology

implementations, 42% of our respondents have fully integrated SIEM correlation and

analysis, compared to only 22% last year.

Breach Payloads

Just as in last year’s survey results, malware tops the list (62%) as the most common

underlying nature of incidents in the respondent’s enterprise, down nearly 20% from

last year’s results. The combination of denial-of-service options tied with unauthorized

access for the second most common category of critical incident, with 43% of

respondents reporting such incidents. Both fell this year, with unauthorized access

showing the most dramatic decrease, from 70% in 2014 to 43%. Data breach occurrences

were down as well, with only 39% of 2015 respondents experiencing such incidents

compared to 63% last year. See Table 1.

2 “Incident Response: How to Fight Back,” www.sans.org/reading-room/whitepapers/analyst/incident-response-fight-35342

Table 1. Year-Over-Year Comparison of Incident Types

Incident Type

Malware

Distributed denial of service

Distributed denial of service (DDoS) main attack

Distributed denial of service (DDoS) diversion attack

Unauthorized access

Data breach

Advanced persistent threat (APT) or multistage attack

Insider breach

Unauthorized privilege escalation

Destructive attack (aimed at damaging systems)

Other

False alarms

2014

81.9%

48.9%

70.2%

62.8%

55.3%

12.8%

66.0%

2015

62.1%

43.1%

27.6%

15.5%

42.5%

38.5%

33.3%

28.2%

21.3%

14.9%

1.7%



A contributing factor to the decrease in malware incidents and, in some regard, data

breaches, may be the growing implementation of more effective antivirus, edge

detection and endpoint protection products. Organizations are becoming more adept at

handling these infections with automated processes and may no longer consider them

incidents, as they previously might have.

However, with 33% selecting advanced persistent threat (APT) or multistage attacks,

the extrapolation that APTs are mostly malware-based means there is some overlap in

answers. Unauthorized access (43%) could also be included in malware infections.

Denial and Destruction

DDoS has increasingly become a means to disable a company or hide nastier payloads

inside the noise of the DDoS. Respondents saw more frequent use of attack methods

over the past year. According to 28% of respondents, DDoS was used as a primary attack

method, while 16% saw it used as a diversion attack. This is slightly less than last year’s

49% of respondents who experienced a DDoS attack, whether as a primary attack vector

or a diversionary attack. DDoS was also mentioned as an attack type for the first time in

the 2015 version of the Verizon Data Breach Investigations Report (DBIR).3

Another 15% of respondents cited intentional system damage as a method employed

in breaches their organization has experienced over the past year, which can also deny

service. Though in past years data destruction was seen largely in insider cases with

rogue or disgruntled employees targeting specific data, today we have seen recent

examples of nation-states employing these attacks as weapons in cyberwarfare,

for example in the Sony attack of November 2014, which is suspected to have been

perpetrated by North Korea,4 and in the Las Vegas Sands casino intrusion reported in

December 2014,5 which has been attributed to Iran. What used to be an infrequent

occurrence of an information warfare technique is now becoming more common in

attackers’ weapons arsenals.

Ransomware such as CrytoLocker, first seen in September 2013 and written in by one

respondent, is also considered an attack on availability. This type of malware represents

yet another category of attack that could be considered a serious breach and result in

lost access to sensitive or highly valuable data if IR teams do not have a planned set of

procedures for responding to such incidents.

3 “2015 Data Breach Investigations Report,” www.verizonenterprise.com/DBIR4 www.bloomberg.com/news/articles/2014-12-04/sony-hack-signals-emerging-threat-to-destroy-not-just-steal-data5 www.bloomberg.com/bw/articles/2014-12-11/iranian-hackers-hit-sheldon-adelsons-sands-casino-in-las-vegas

TAKEAWAY:

In response to the growing

prevalence of attacks involving

data destruction and system

disruption, IR teams must

prepare to contain, counter

and remediate by creating

specific procedures that

address this unique type of

attack.



Targeted Data Theft

In this year’s survey, employee information was the most common category of data

stolen, with 41% of participants citing employee data as the top target of their attackers.

Another 36% cited individual customer information, while 30% selected intellectual

property. The fourth most common category of stolen data is proprietary customer data

(27%), different from individual customer information due to its relation to the service

provided by the victim company. For example, proprietary customer data from an ISP

would include Internet usage, bandwidth and IP address assignment information for the

customer. Table 2 provides a comparison of 2014 and 2015 data exfiltration statistics.

‘

It’s estimated that 4 million records were compromised in a recent example of data theft

detected in April 2015 at the U.S. Office of Personnel Management (OPM).6 The financial

consequences of this breach can be used as a case study for the typical organization.

Based on estimates that the cost of each record lost is $154,7 we can determine that the

OPM compromise will have a minimum cost of $616 million, depending on the type of

data stolen, just on a cost-per-record basis. When you factor in the sensitivity of the data

that was stolen, the cost will likely be much higher. Such breaches should be avoided

with proper prevention; and their effects must be minimized if they can’t be avoided.

Table 2. Data Types Targeted 2014–2015

Data Type

Employee information

Individual consumer customer information

Intellectual property (source code, manufacturing plans, etc.)

Proprietary customer information

Legal data

PCI data (payment card numbers, CVV2 codes, track data)

PHI data (health information)

Other

Other regulated data (SOX, non-PHI personally identifiable information, etc.)

2014

36.4%

36.4%

31.8%

31.8%

12.1%

15.2%

2015

41.2%

35.8%

29.7%

26.7%

14.5%

13.9%

12.1%

11.5%

11.5%

6 www.opm.gov/news/releases/2015/06/opm-to-notify-employees-of-cybersecurity-incident7 http://securityintelligence.com/cost-of-a-data-breach-2015/#.VYl0u0a6L_0

Verizon’s 2015 DBIR8 reports that the average time required for an attacker to conduct a

breach is decreasing while the average time to detect a breach is increasing. In reviewing

the incidents occurring in 2014, they found that in 60% of the breaches investigated

attackers were able to compromise an organization within minutes. Considering what

IR teams are up against, automating the response processes and reducing the time

available to attackers are imperative.

Metrics

Whether working as part of a consulting service or internal team, establish a set of

metrics to measure improvements in IR process efficiency and effectiveness. The

core reason for tracking metrics is to demonstrate the value of their investment to

stakeholders. However, the metrics used by survey takers vary widely: 23% of our

respondents use well-defined metrics to help track, evaluate and update their plan,

whereas 37% measure improvements in accuracy, response time and reduction of attack

surface, as shown in Figure 4.

For metrics to be useful, they must be periodically compared against a baseline. Based

on the complexity of an intrusion and the sophistication of an attacker, detection and

remediation prove to be more complex in specifically targeted industries. Comparing

metrics across industries is not a useful guide for measuring in-house IR functional

progress. Instead, use resources such as the whitepaper, “An Introduction to the Mission

Risk Diagnostic for Incident Management Capabilities (MRD-IMC),”9 as a guide for

establishing internal metrics.


Key Elements for Successful Incident Response

How do you assess the effectiveness and maturity of your IR processes?

Figure 4. Measures of Improvement

We use well-defined metrics to help us track, evaluate and update our plan.

We measure improvements in accuracy, response time and reduction of attack surface.

We conduct incident response exercises on a routine basis.

Other

8 www.verizonenterprise.com/DBIR/20159 http://resources.sei.cmu.edu/library/asset-view.cfm?assetID=91452

Key Elements for Successful Incident Response (CONTINUED)


A core measure of IR effectiveness is the time from infection, or occurrence of incident,

to detection and remediation. In our survey, the single most selected average for time to

detection was two to seven days (23%), which was also the most selected (28%) answer

option for time to remediate. However, when aggregated, 37% of respondents reported

an average time to detection of less than 24 hours, while 36% remediated within 24

hours after detection. See Figure 5.

[

In contrast, 11% take more than one month to detect an incident, as well as remediate

an incident after detection.

Time to detection and remediation are difficult metrics on which to compare

organizations because some industries are more attractive, valuable targets for

sophisticated attackers. So it’s important to measure improvement in incident

handling within the organization rather than looking at averages across industries.

But is this 24-hour or less time frame cited for both detection and remediation

realistic? Based on other security trend reports, the average dwell time for a

financial services company is 98 days, and it’s 197 days for retail.10 How, then, can

our respondents report an average time frame to return business functions to fully

operational within two to 10 days of infection?

The answer lies in the varying definitions of detection and remediation. Often

organizations determine time to detection from when the system first exhibited

suspicious behaviors, thus explaining the short (less than 24 hours) time frame

between infection and detection.

Percentage of respondents citing

2–7 days as the time to detection and

time to remediation, respectively

23%

28%

On average, how much time elapsed between the initial compromise and detection (i.e., the dwell time)? How long from detection to remediation?

Please check both columns as they apply.

Time to detection Time to remediation

Figure 5. Time to Detection/Time to Remediation

30%

25%

20%

15%

10%

5%

0%

Unknown < 1 hr 1–5 hrs 6–24 hrs 2–7 days 8–30 days 1–3 mos 4–6 mos 7–12 mos > 1 yr

10 www.zdnet.com/article/businesses-take-over-six-months-to-detect-data-breaches



Remediation Practices

Without proper investigative skills or resources, the number of compromised systems

and accounts—and the amount of data stolen—is not properly quantified. In these

instances, detection to remediation is achieved quickly with a simple wipe and reimage.

Yet, best industry response practices also include signaturing malware and attacker

behavior based on the initial system(s) identified. Once these unique signatures, known

as indicators of compromise, are created, they are used to scan other systems in the

enterprise. In this way, all systems with active malware or similar artifacts of attacker

activity will be identified. In our survey, 88% of our respondents stated they were

conducting this type of identification and follow-up either manually or in an automated

fashion (see Table 3).

For all the practices listed here, respondents did more manually than through

automated processes. The critical thing to remember is that manual practices take more

time and are usually much less accurate than automated procedures.

Table 3. Remediation Practices What practices do you have in place for remediating incidents?

Indicate whether the process is conducted manually, through automated systems that are integrated, or a combination of both. Choose only those that apply to your organization.

Answer Options

Quarantine affected hosts

Shut down system and take it offline

Kill rogue processes

Remove rogue files

Reimage/Restore compromised machines from gold baseline image

Isolate infected machines from the network while remediation is performed

Block command and control to malicious IP addresses

Reboot system to recovery media

Identify similar systems that are affected

Remotely deploy custom content or signatures from security vendor

Update policies and rules based on IOC findings and lessons learned

Removing file and registry keys related to the compromise without rebuilding or reinstalling the entire machine

Boot from removable media and repair system remotely

Other

Manual

44.8%

67.0%

52.1%

43.4%

60.4%

63.2%

38.5%

64.2%

49.7%

33.7%

59.4%

52.4%

58.7%

6.3%

Automated

22.2%

7.6%

11.1%

12.5%

11.1%

8.7%

18.8%

7.3%

10.8%

21.2%

7.3%

8.3%

6.9%

2.8%

Both

29.9%

20.5%

31.6%

38.2%

22.2%

21.5%

35.8%

18.4%

27.8%

33.0%

18.4%

23.3%

16.7%

5.2%

Total Response

96.9%

95.1%

94.8%

94.1%

93.8%

93.4%

93.1%

89.9%

88.2%

87.8%

85.1%

84.0%

82.3%

14.2%

TAKEAWAY:

Automating remediation

processes will speed time to

remediation and reduce the

workload assigned to IR staff.



What Works

Organizations are still automating what they can in their processes, which SANS defines

as integrating functions across ecosystems. Traditional anti-malware/edge protection,

logs and behavior-based scanning are the most integrated, according to results.

Detection

The three most popular detection technologies, as indicated by being either fully or

partially integrated into respondents’ IR capabilities, are IPS/IDS/firewall and unified

threat management (UTM) alerts (89%), log analysis (81%), and network-based scanning

agents for signatures and detected behavior (81%). See Table 4.

Table 4. Detection Technologies Does your organization use any of the following capabilities to identify impacted systems?

If so, please indicate how integrated each is with your overall incident response ecosystem. Select only those that apply.

Answer Options

IPS/IDS/Firewall/UTM alerts

Log analysis

Network-based scanning agents for signatures and detected behavior

User notification/Complaints

Network packet capture or sniffer tools

SIEM correlation and analysis

Endpoint Detection and Response (EDR) capabilities

Network flow and anomaly detection tools

Third-party notifications and intelligence

Network traffic archival and analysis tools

Intelligence and analytics tools or services

Host-based intrusion detection (HIDS) agent alerts

Endpoint controls (e.g., NAC or MDM)

Home-grown tools for our specific environment

SSL decryption at the network boundary

Browser and screen-capture tools

Third-party tools specific for legal digital forensics

Other

Total Integrated

88.5%

81.4%

81.0%

78.0%

75.9%

74.9%

74.2%

69.5%

69.5%

67.1%

65.8%

65.4%

61.0%

58.0%

55.9%

50.8%

49.5%

8.5%

Partially Integrated

32.9%

41.0%

34.2%

38.0%

40.7%

32.5%

38.6%

35.9%

41.7%

32.2%

38.3%

32.5%

33.9%

36.6%

29.5%

27.5%

26.8%

4.1%

Highly Integrated

55.6%

40.3%

46.8%

40.0%

35.3%

42.4%

35.6%

33.6%

27.8%

34.9%

27.5%

32.9%

27.1%

21.4%

26.4%

23.4%

22.7%

4.4%



There is a strong correlation with the top three automated capabilities between our

2015 and 2014 surveys (although the questions were worded differently in 2014). In our

2015 survey, IPS/IDS/Firewall and UTM alerts were integrated by 89% of respondents,

81% integrated log analysis in their response practices, and 81% integrated network-

based scanning agents for signatures and detected behavior. These categories were also

among the most highly used processes in 2014—whether automated, manual or both.

IPS/IDS/Firewall and UTM alerts were used by 91%, log analysis by 85%, and network-

based scanning agents for signatures and detected behavior by 96%.

Most respondents (44%) felt their SOCs were immature and unable to respond well

to events, with 25% believing their SOCS were maturing, 14% feeling their SOCs were

mature and the rest unsure of their status.

In correlating the maturity of IR capabilities within an organization with the technologies

and resources deployed, mature SOCs had the greatest integration of technologies

such as endpoint-detection-and-response (EDR) capabilities, network packet capture

implementation, and SIEM correlation and analysis.

Intelligence

The incorporation of cyberthreat intelligence (CTI) and analytics tools and services was

also more prevalent in organizations with mature SOCs. By correlating threat intelligence

and analytics, IR teams can detect and respond to threats based on past incidents and

those included in CTI feeds. In fact, in the 2015 SANS Cyberthreat Intelligence Survey,11

75% of respondents cited CTI as important to security. Yet only 66% of respondents

to this 2015 survey on IR report high or partial integration of intelligence with their IR

processes. They do, however, use intelligence provided either internally or through third-

party sources. Specifically, respondents use intelligence in the following ways:

• 96% tie intelligence to IP addresses. This was the most commonly implemented

type of CTI data when including internal and third-party sources.

• 93% tie traffic to known suspicious IPs.

• 91% track endpoint security data and logs.

• 91% incorporate signatures and heuristics from previous events.

11 “Who’s Using Cyberthreat Intelligence and How?” www.sans.org/reading-room/whitepapers/analyst/who-039-s-cyberthreat-intelligence-how-35767



For the full breakdown of in-house and third-party capabilities for IR processes, see

Figure 6.

As indicated earlier, most of the respondents to this survey work internally for their organizations, so it makes sense that the primary outsourced functions include heuristics, reputation data and adversary/attacker data attributes. Tor node IP addresses would fit into the reputation and attacker data categories as well. Many organizations will not expect to receive legitimate traffic from Tor exit nodes, but because the exit nodes change frequently, they need automated processes to effectively block attacks from Tor exit nodes.

It’s clear that significant security implementations are present within our respondents’ networks. However, their full functionality cannot be achieved without automation in analysis, correlation and reporting. A notable 42% of respondents have fully integrated, and 33% have partially integrated SIEMs into their IR ecosystems for analytics during response. Some may also be relying on their CTI tools or services to do the analytics for them, with 26% fully integrating and 28% partially integrating CTI within their functions. The 13% of organizations not currently integrating analytics, such as a SIEM, into their response should consider this a top priority to mature their SOC and IR processes.

What kind of threat intelligence are you using? Please indicate what is being delivered through third parties and what is developed internally?

Select only those that apply.

Susp

icio

us fi

les,

host

flow

an

d ex

ecut

able

s

Net

wor

k hi

stor

y da

ta

Endp

oint

dat

a an

d lo

gs

IP a

ddre

sses

/Nod

es

Adve

rsar

y/A

ttac

ker a

ttrib

utio

n

Une

xecu

ted

or u

ndet

onat

ed

mal

icio

us fi

les

Oth

er

Heu

ristic

s/Si

gnat

ures

fr

om p

revi

ous

even

ts

Com

mun

icat

ions

bet

wee

n sy

stem

s an

d m

alic

ious

IP a

ddre

sses

Dom

ain

data

Tor N

ode

IP a

ddre

sses

Repu

tatio

n da

ta

Hos

t and

net

wor

k in

dica

tors

of

com

prom

ise

(IOCs

)

Figure 6. Type of Intelligence Used

100%

80%

60%

40%

20%

0%

Provided by Third Party Internal Discovery Both

The full functionality

of security

implementations

cannot be achieved

without automation

in analysis,

correlation and

reporting.



What’s Not Working: Impediments to Response

Despite improvements in technology, IR processes and their analytics capabilities,

organizations still face obstacles that impede effective IR. Leading the list is staffing

and skills shortages, flagged by 66% of survey respondents as one of the top obstacles

to effective IR. The third top problem reported is lack of visibility, indicating that for as

much automation and integration respondents are attempting, they still do not have the

full-picture view across systems they need for fast, accurate response. See Figure 7.

This lack of visibility is making it difficult for 37% of respondents to distinguish between

real malicious events and nonevents. Lack of budget for tools and technology, cited

by 54% of respondents is only contributing to this lack of visibility, and staffing issues

account for lack of procedural reviews and practice (41%).

Top 10 Impediments to Effective IR

Staffing and skills shortage

Budgetary shortages for tools and technology

Inability to distinguish malicious events versus nonevents

Lack of comprehensive automated tools available to investigate new technologies,

such as BYOD, IoT, and cloud-based IT

Not enough visibility into events happening across different systems or domains

Organizational silos between IR and other groups or between data sources or tasks

Integration issues with our other security and monitoring tools

Lack of procedural reviews and practice

Too much time to detect and remediate

Difficulties in detecting sophisticated attackers and removing their traces

Figure 7. Impediments to Investigations

0% 20% 60%40%



12 www.esg-global.com/blogs/new-research-data-indicates-that-cybersecurity-skills-shortage-to-be-a-big-problem-in-201513 www.bls.gov/oes/current/oes151122.htm

Recruiting and Retention

Let’s first tackle the people issue: In many instances, the cause of understaffing is not due to a lack of funding, but to a lack of available skilled professionals to fill open positions. Based on a 2014 survey conducted by Enterprise Strategy Group (ESG), 28% of organizations say they have a “problematic shortage” of IT security skills.12 One recommendation to aid in recruitment is to consider filling positions with remote workers.

According to the SANS 2015 IR surveys, 73% of organizations use a dedicated team, 70% are drawing team members from their internal staff assigned to other functions, while 32% are drawing from third-party services. For surge team augmentation, 61% used a dedicated internal surge team in both 2014 and 2015, 63% draw additional surge staff from internal resources and 28% (27% in 2014) use outsourced services. See Figure 8.

Location is also important to staffing. According to the U.S. Bureau of Labor Statistics, the highest concentration of information security professionals is in the Washington, D.C. metropolitan area. In comparing the highest and lowest concentrations of InfoSec professionals by metropolitan areas, 9,070 workers were identified in the DC area, whereas only 430 were located in Albuquerque, New Mexico, the area with the lowest concentration of InfoSec.13 Options for companies looking to hire skilled technical professionals outside of major metropolitan areas include enticing a potential employee to relocate or building a remote IR team. This second option is becoming more feasible as infrastructure to support telecommuting is now commonplace in most organizations. One of the obstacles that may exist to employing remote workers, admittedly, is the difference in the cost of living and salary requirements associated with different areas.

Core Versus Surge Staffing

Other

Outsourced services (e.g., MSSP-managed services security provider) with dedicated IR

services (alerts, response)

Drawn from other internal staff (security group, operational/administrative IT resources)

Dedicated internal IR team

Figure 8. Resources Used in Incidents

0% 20% 60% 80%40%

Core team Surge



Diversity of Investigations

More platforms are involved in today’s investigations, driving the need for more

specialized skills. More virtualized and cloud-based systems are being supported by

in-house IR capabilities since last year, for example. Last year, data center servers hosted

in the public cloud (e.g., Azure or Amazon EC2) were investigated in-house by only

37% of our respondents, compared with 61% in 2015. Other notable changes include

employee-owned systems. Last year, only 58% of respondents investigated employee-

owned equipment, whereas 69% do this year. This supports the growing prevalence of

employees bringing their own devices, whether laptops, tablets or smartphone devices,

and connecting them to the organization’s network resources. See Figure 9.

We included a new category of “employee social media accounts” as an area of possible

investigation for IR teams because this medium is being used effectively by sophisticated

attackers for targeted reconnaissance. Just 59% of respondents cite including this

element in their in-house investigations.

What business processes and systems are involved in your investigations? Check only those that apply. Please indicate whether your capabilities for these investigations

exist in-house, are outsourced or both.

Embe

dded

, or n

on-P

C de

vice

s, su

ch a

s m

edia

and

ent

erta

inm

ent b

oxes

, prin

ters

, sm

art c

ars,

conn

ecte

d co

ntro

l sys

tem

s, et

c.

Empl

oyee

soc

ial m

edia

acc

ount

s

Busi

ness

app

licat

ions

and

ser

vice

s (e

.g.,

emai

l, fil

e sh

arin

g) in

the

clou

d

Corp

orat

e-ow

ned

lapt

ops,

smar

tpho

nes,

tabl

ets

and

othe

r mob

ile d

evic

es

Corp

orat

e-ow

ned

soci

al m

edia

acc

ount

s

Third

-par

ty s

ocia

l med

ia a

ccou

nts

or p

latf

orm

s

Inte

rnal

net

wor

k (o

n-pr

emis

es) d

evic

es

and

syst

ems

Dat

a ce

nter

ser

vers

hos

ted

loca

lly

Empl

oyee

-ow

ned

com

pute

rs, l

apto

ps,

tabl

ets

and

smar

tpho

nes

(BYO

D)

Oth

er

Web

app

licat

ions

Dat

a ce

nter

ser

vers

hos

ted

in th

e pu

blic

cl

oud

(e.g

., A

zure

or A

maz

on E

C2)

Figure 9. Investigated Media, Platforms and Apps

100%

80%

60%

40%

20%

0%

In-house Outsourced Both



Visibility

How do you achieve visibility across these systems for a full picture view of actual events

in progress versus nonevents? This is not the only SANS survey to indicate a lack of

visibility as being among the top three inhibitors of effective detection and response. As

we showed earlier, respondents are integrating across some platforms and using SIEM to

analyze the data.

More than 64% of respondents identified the need for better security analytics and

correlation across affected systems, making it the top target area for improvement. This

is an important milestone because respondents can acknowledge weaknesses and point

to reasons why detection is failing. See Figure 10.

Figure 10. Planned Improvements in Next 12 Months

[Begin figure content]

TAKEAWAY:

Focus on key areas to achieve

integration of security

information into automated

policy where possible and

reduce reliance on specialized

workers to “catch things” and

seek out infected systems

manually.

What improvements in IR is your organization planning to make in the next 12 months? Select all that apply.

Bett

er s

ecur

ity a

naly

tics

and

corr

elat

ion

acro

ss e

vent

type

s an

d im

pact

ed s

yste

ms

Full

auto

mat

ion

of d

etec

tion,

re

med

iatio

n an

d fo

llow

-up

wor

kflow

s

Mor

e au

tom

ated

repo

rtin

g an

d an

alys

is

thro

ugh

secu

rity

info

rmat

ion

and

even

t m

anag

emen

t (SI

EM) i

nteg

ratio

n

Addi

tiona

l tra

inin

g/ce

rtifi

catio

n of

sta

ff

Oth

er

Bett

er re

spon

se ti

me

Impr

oved

vis

ibili

ty in

to th

reat

s an

d as

soci

ated

vul

nera

bilit

ies

as th

ey a

pply

to

the

envi

ronm

ent

Mor

e in

tegr

ated

thre

at in

telli

genc

e fe

eds

to a

id in

ear

ly d

etec

tion

70%

60%

50%

40%

30%

20%

10%

0%

Figure 10. Planned Improvements in Next 12 Months



Additional training and certification will be big next year, with 57% of respondents

adding training and certification for their IR staff. This is a reoccurring theme in this

year’s survey results, with staffing and skills shortages ranked as one of the top five

impediments to effective IR by 66% of respondents.

The other top targeted areas for improvement include improved visibility into threats

and vulnerabilities, as well as more automated reporting and analysis via SIEM

integration. Many of these areas of improvement have a symbiotic nature—one

depending on an improvement in another to truly benefit an organization. Clearly, an

improvement in visibility (gaining more insight into endpoint and network traffic) will

result in more collected data, which will require automated analysis.

TAKEAWAY:

Organizations that take the

initiative to “grow their own”

in-house skills will increase the

efficiency of their IR process

and improve effectiveness of

security implementations and

technology.


Conclusion

Although automation was the most commonly cited area for future IR improvement in

last year’s survey, only a little progress has been made in increasing visibility through

automation of endpoint and network data collection and analytics, or remediation.

This continues to be a key factor in improving IR process efficiency. As the amount of

data collected from endpoints and network traffic grows, teams must move toward

automation to conduct analysis and data correlation with the goal of shortening the

time needed to detect and remediate incidents.

Our survey results also suggest the need for more specialized IR skills. By reducing false

positive alerts and baselining endpoint and network traffic to better detect anomalies,

understaffed teams will have more actionable alerts. The shortage of skilled technical

staff may not have an immediate solution, but organizations can maximize the actions

of existing IR team members by moving to automated detection and remediation

processes.

Reports of data destruction and denial of service attacks have been covered in the

media recently, and the responses from our survey participants substantiate the growing

frequency of such adversary tactics. IR teams, frequently overworked and charged

with constantly putting out fires, rarely have time to craft a new playbook for attacks

requiring different IR processes and containment procedures. Current trends, as seen in

the Sony and Las Vegas Sands Casino attacks, foreshadow what today’s IR teams will be

faced with in future attacks. Anticipate, plan, test and validate response procedures for

the worst attacks—because, inevitably, they are coming.

Alissa Torres is a SANS analyst and certified SANS instructor specializing in advanced computer

forensics and incident response (IR). She has extensive experience in information security in the

government, academic and corporate environments. Alissa has served as an incident handler and

as a digital forensic investigator on an internal security team. She has taught at the Defense Cyber

Investigations Training Academy (DCITA), delivering IR and network basics to security professionals

entering the forensics community. A GIAC Certified Forensic Analyst (GCFA), Alissa holds the GCFE,

GPEN, CISSP, EnCE, CFCE, MCT and CTT+ certifications.

Jake Williams is a SANS analyst, certified SANS instructor, course author and designer of several

NetWars challenges for use in SANS’ popular, “gamified” information security training suite. Jake

spent more than a decade in information security roles at several government agencies, developing

specialties in offensive forensics, malware development and digital counterespionage. Jake is the

founder of Rendition InfoSec, which provides penetration testing, digital forensics and incident

response, expertise in cloud data exfiltration, and the tools and guidance to secure client data against

sophisticated, persistent attack on-premises and in the cloud.


About the Authoring Team

Sponsors

SANS would like to thank this survey’s sponsors:

Documents

Maturing and Specializing: Incident Response Capabilities ...malicious events from nonevents, and 45% cited lack of visibility into events across a variety of systems and domains as