SANS Technology Institute - Candidate for Master of Science Degree

Establishing a Security Metrics Program

Tiger Team Final Report

Chris Cain & Erik CoutureOctober 2011

SANS Technology Institute - Candidate for Master of Science Degree

Introduction

• Team Members• Mandate• Overall project aim• Methodology

SANS Technology Institute - Candidate for Master of Science Degree

Security Metrics Overview

• “How secure are we?”• “Are our security investments making a

difference?”• “Where can we have the most impact on

our security posture?"

SANS Technology Institute - Candidate for Master of Science Degree

Why Metrics?

• Metrics vs Measurement• The importance of context and knowledge,

not just data• The challenge of what to measure

SANS Technology Institute - Candidate for Master of Science Degree

Goal/Scope

• Paint a clear picture of our security posture• Identify areas of greatest risk• Help educate resource allocation towards

areas of greatest security gain• Educate senior management on possible

business impacts of our security posture • Provide a method to monitor the

effectiveness of our policy and technological changes over time

SANS Technology Institute - Candidate for Master of Science Degree

Example 1 Secure Firewalls, Routers, and Switches

Aim• Visibility of the

‘ground truth’• Ensure minimal

ports/services exposedInput Data•Network Device Threat Level•Average days to fix configuration issues•Total insecure configurations foundVisualization• Horizontal bar charts – give a good sense of progress over

several reporting periods and between each device type

SANS Technology Institute - Candidate for Master of Science Degree

Example 2 Boundary Defense

Aim• Reduce by 80% the number of internet entry points• Achieve 100% of hosts pointed at secure DNS servers • Achieve 100% physical network verification.

Input Data•Total quantity of defenses scored Score from 1 to 5•Boundary Defense Threat Level (subjectively assigned)Visualization•Line graph comparing boundary device types against their scores

SANS Technology Institute - Candidate for Master of Science Degree

Example 3Incident Response Capability

Aim• Assess ability to detect and respond• Fuse/visualize end-to-end IH timelines

Input Data• Mean time to incident recovery• Number of Lessons Learned

as a result of the incident.• Mean time to incident eradication• Mean time to incident detection/identification

Visualization• Stacked Bar Chart – allows reader to quickly compare the

relative time involved in each phase of incident handling

SANS Technology Institute - Candidate for Master of Science Degree

Visualization / Dashboard (1)

SANS Technology Institute - Candidate for Master of Science Degree

Visualization / Dashboard (2)

SANS Technology Institute - Candidate for Master of Science Degree

Recommendations

• The establishment of an enterprise-wide security metrics program.

• The adoption of the SANS Top 20 Security Controls framework as a basis for the ongoing gathering and reporting of security metrics.

• The institution of a security metrics board which will regularly assess the effectiveness and adjust the security metrics program.

SANS Technology Institute - Candidate for Master of Science Degree

References

• Twenty Critical Security Controls for Cyber Defense: SANS/CAG• NIST Special Publication 800-61• Beautiful Security Metrics by Elizabeth Nichols• Twenty Most Important Controls and Metrics for Effective Cyber Defense

and Continuous FISMA Compliance by John Gilligan• Seven Myths about Information Security Metrics by Dr. Gary Hinson• Security Metrics, Replacing Fear, Uncertainty and Doubt, Gary McGraw• FISMA FY2011 - CIO Reporting Metrics by US DHS• IT Security Metrics, A Practical Framework for Measuring Security &

Protecting Data, Lance Hayden, Ph.D.• A Guide to Security Metrics (SANS Reading Room), Shirley C. Payne• CSO Security and Risk by Scott Berinato