26
Information Security Metrics: Practical Steps to Measurement Jack Nichelson & James Tarala

Jack Nichelson - Information Security Metrics - Practical Security Metrics

Embed Size (px)

Citation preview

Page 1: Jack Nichelson - Information Security Metrics - Practical Security Metrics

Information Security Metrics:Practical Steps to Measurement

Jack Nichelson & James Tarala

Page 2: Jack Nichelson - Information Security Metrics - Practical Security Metrics

I defend my companies competitive advantage by helping solve business problems through technology to work faster and safer.

Who is Jack Nichelson? Director of Infrastructure & Security for Chart Industries.

Recognized as one of the “People Who Made a Difference in Security” by the SANS Institute and Received the CSO50 award.

Adviser for Baldwin Wallace’s, State winner Collegiate Cyber Defense Competition (CCDC) team. “Solving Problems, is my Passion”

Introduction

Page 3: Jack Nichelson - Information Security Metrics - Practical Security Metrics

• How do you measure & report progress?

• Is your team focused on the right problems?

• How do you promote accountability & transparency?

• How do you find waste, time and money?

• Are your projects improving the daily jobs of your end users?

“Secure more with less, show continuous improvement and value”

Problem Statement

In an era of security breaches we tend to have only one metric – Have my systems been compromised?

Page 4: Jack Nichelson - Information Security Metrics - Practical Security Metrics

Why are Metrics Needed?• Businesses use metrics to facilitate decision making• Better data leads to better decisions• Metrics allow organizations to set appropriate priorities

• Measurement allows comparison:– Between our organization and industry benchmarks– Between our organization and other organizations risk

levels– Between levels of accepted risk over time– Between business units within an organization

Page 5: Jack Nichelson - Information Security Metrics - Practical Security Metrics

Metrics from the Business World

• The business world uses metrics all the time• Consider the following examples:– Price to Earnings Ratio– Profit & Loss Statements– Product Sales Quotas– Number of Safety Incidents– Unit Production– Web Advertisement Click Counts– Number of Facebook “Likes” per Post

Page 6: Jack Nichelson - Information Security Metrics - Practical Security Metrics

Metrics in Technology• Organizations even commonly use metrics to help measure

the performance of technology systems as well• Consider the following examples:– System uptime– CPU Utilization Percentage– Memory Use Percentage– Average Email Mailbox Size– Support Technician to Computer Node Ratio– Help Desk Ticket Time to First Touch– Help Desk Ticket Time to Resolution

Page 7: Jack Nichelson - Information Security Metrics - Practical Security Metrics

IS Metrics: Too Broad?• The first question we need to ask is, “What do we mean by

the term Information Security metrics?”• IS Metrics is too broad of a term

• “Begin with the end in mind.” – Stephen Covey

• Measurement for measurement’s sake helps no one• Organizations must be specific on what they are measuring

and the benefits they hope to achieve from it

Page 8: Jack Nichelson - Information Security Metrics - Practical Security Metrics

Suggested SolutionCreate an effective, sustainable security aware culture that is results driven. Foundation Leading Change Gemba Board

• Security• Quality• Delivery• Cost• People

Case Study Examples & Results

Page 9: Jack Nichelson - Information Security Metrics - Practical Security Metrics

Begin With The End In Mind

Example of how some simple goals that are tracked as a team will move security forward.

Page 10: Jack Nichelson - Information Security Metrics - Practical Security Metrics

Primary Recommendation1. Start small, excel at gathering a small number of metrics2. Integrate these metrics into your business process3. Grow the number of metrics you collect

• United States Department of State iPost began with only three data sensors:– Tenable Nessus– Microsoft Active Directory– Microsoft System Management Server (System Center)

Page 11: Jack Nichelson - Information Security Metrics - Practical Security Metrics

iPost

Page 12: Jack Nichelson - Information Security Metrics - Practical Security Metrics

Foundation• Obtain a security charter from senior management• Create an organization wide IS Steering Committee• Document your organization’s overall security goals• Create an asset inventory & Assign data owners to all of your systems• Deploy a vulnerability scanner & scan your hosts on a regular basis• Start with 4 data sources:

• Microsoft Active Directory

• Help Desk Ticketing System

• Microsoft System Center (SCCM)

• Tenable Nessus or Qualys

Recommended elements for getting started:

Page 13: Jack Nichelson - Information Security Metrics - Practical Security Metrics

Leading Change

Step 1: Create Urgency - For change to happen, you need to make the case why and be brutally honest.

Step 2: Form a Powerful Coalition – Get visible support from key people and link metrics to performance.

Step 3: Create a Vision for Change - Develop what you "see" as the future that people can grasp easily and remember.

Step 4: Communicate the Vision - Talk about it every chance you get. Use the vision daily to make decisions and solve problems.

Culture Eats Strategy - Make metrics part of your culture

Page 14: Jack Nichelson - Information Security Metrics - Practical Security Metrics

Leading Change

Step 5: Remove Obstacles - Empower the people you need to execute your vision, and help the change move forward.

Step 6: Create Short-Term Wins - Nothing motivates like success

Step 7: Build on the Change - change projects fail because victory is declared too early.

Step 8: Anchor the Changes in Corporate Culture - Your culture determines what gets done, so the values behind your vision must show in day-to-day work.

You have to work hard to change a culture successfully. If you're too impatient, and if you expect too many results too soon, your plans for change are more likely to fail.

Page 15: Jack Nichelson - Information Security Metrics - Practical Security Metrics

Gemba BoardGemba (現場 ) is a Japanese term referring to the place where value is created. The idea of Gemba is that the problems are visible, and the best improvement ideas will come from going to the Gemba.

Page 16: Jack Nichelson - Information Security Metrics - Practical Security Metrics

Gemba Board: SecurityExample Metrics: • # of systems not monitored & tracked in inventory by Location or LoB• # Top Vulnerabilities by Location or LoB• # of Legacy Systems by Location or LoB• # of Users with Local Admin & Accounts with Domain Admin• # of Total Security Incidences by Location or LoB• # of Past Due Security Awareness Training by Location or LoB

Security - The current security posture at a glance

Page 17: Jack Nichelson - Information Security Metrics - Practical Security Metrics

Gemba Board: QualityExample Metrics: • # of Servers & Workstation missing OS & App patches (30 day SLA)• # of infections/Re-Images tickets (3 day SLA)• # of Security Event tickets (5 day SLA)• # of Security Request tickets (15 days SAL)• Cause Mapping Analysis to find root cause of problems

Quality – Results for SLA goals of events & requests

Page 18: Jack Nichelson - Information Security Metrics - Practical Security Metrics

Gemba Board: DeliveryDelivery – Active Projects & Audits at a glanceExample Metrics: • Active Projects Status• Active Audit Status• Remediation Progress by Location or LoB• On-Site Awareness Training by Location

Page 19: Jack Nichelson - Information Security Metrics - Practical Security Metrics

Gemba Board: CostCost – P&L at a glanceExample Metrics: • Operating budget spending plan (OPEX & CAPEX)• ROIC Qualitatively Rating of Perceived Value• Support Agreements Costs & Renew dates• Consultant Support Agreements Costs & Renew dates• Running total of cost savings

Page 20: Jack Nichelson - Information Security Metrics - Practical Security Metrics

Gemba Board: PeoplePeople – Skills matrix at a glanceExample Metrics: • Skills Matrix of everyone in Security• Training and development plans• On-Call & Vacation Schedules• Awards

Page 21: Jack Nichelson - Information Security Metrics - Practical Security Metrics

Practical Steps: Base• To create an effective, sustainable program to implement

metrics, don’t start by creating metrics• Recommendation would be:

1. Obtain a security management charter from senior management

2. Create an organization wide IS Steering Committee3. Document your organization’s overall security goals4. Create & approve appropriate security policies,

procedures, & standards5. Educate your organization on those documents

Page 22: Jack Nichelson - Information Security Metrics - Practical Security Metrics

Practical Steps: Phase IOnce a base or foundation for information assurance is laid, then you can begin with metrics• The next phase would be to:

1. Identify what information security sensors you have already successfully deployed

2. Determine what meaningful metrics can be gleaned from these sensors

3. Deploy a tool that can centrally aggregate, normalize, and report on the data collected by the sensors

4. Create basic reports based on the metrics from strep #25. Work with business owners to remediate risk

Page 23: Jack Nichelson - Information Security Metrics - Practical Security Metrics

Practical Steps: Phase IINow you are ready for continuous process improvement • The last steps are to refine your effort, gather more data, and

remediate more risk:1. Deploy additional sensors & aggregate the results2. Determine meaningful metrics that new sensors can

bring3. Collaborate with business owners to make metrics more

meaningful4. Remediate new risks as they are discovered5. Automate the response to as many metrics as possible

Page 24: Jack Nichelson - Information Security Metrics - Practical Security Metrics

Software Tools to Help• Open Source Projects:– Practical Threat Analysis (PTA) Professional– OSSIM Open Source SIEM

• Commercial Tools:– Archer Technologies SmartSuite– OpenPages Enterprise GRC– Bwise GRC– MetricStream– Methodware ERA– Protiviti Governance Portal– CCH TeamMate, Sword, & Axentis

Page 25: Jack Nichelson - Information Security Metrics - Practical Security Metrics

Bare Minimum Response1. Create an asset inventory2. Assign data owners to all of your systems3. Deploy a vulnerability scanner & scan all of your hosts on a

regular basis4. Create overall CVSS risk scores, by business unit, and publish

those scores to key business owners5. Remediate the risk you discover

• Focus on the basics, then improve your efforts• Run a 5K first, then try a marathon

Page 26: Jack Nichelson - Information Security Metrics - Practical Security Metrics

Further Questions• Jack Nichelson– E-mail: [email protected] – Twitter: @Jack0Lope– Website: http://www.linkedin.com/in/nichelson

• Resource for further study:– Security Metrics: Replacing Fear, Uncertainty, and Doubt by Andrew Jaquith