Auditing: Measuring something against a standard

How do you know you…?

ObjectivesThe student shall be able to: Define audit, vulnerability, threat, policy, procedure, baseline, auditor,

audit exception, and audit exception root cause. Describe the purpose of a baseline, and the contents of a Network Traffic

Baseline and System Baseline. Define the terms detective, corrective, and preventive controls, and

correctly classify a control into one of these categories. Define detection time, response time, and exposure time, given an example

time-based security situation. Describe the purpose of the audit plan’s scope, purpose, checklist, policy

resource guideline, audit strategy. Write an audit plan. Describe the purpose of each stage of an audit. Describe important points of staying out of jail while doing an audit. Conduct a complete audit, procedurally. Develop a mini-audit plan and audit report based on logs and security

configuration (Lab).There is no reading this week. Work on your audit plan/report

Parts of AuditsSecurity Audit: Measures how well our security

policies/procedures are relative to best-in-classAssessment or Verification: Analysis of security

improvements. Are our procedures effective?Conformance Audit: Measures how well a system

or process conforms to policies/proceduresValidation: How well are we following our

guidelines?

Firewall example:Verification: Is our plan effective?Validation: Is it really protecting us?

VocabularyVulnerability: An unlocked door in

infrastructure or organizationVulnerability Assessment: An evaluation of

potential vulnerabilities related to the described scope

Threat: An action that exposes a vulnerabilityExamples: File deletion, information exposure,

improper use of assets, malware attackIntentional versus Accidental Threat: Both

have same effectExposure = Vulnerability + Threat

Time ofEvent

Detective Controls:Detecting problem when it occursIncludes:Intrusion Detection SystemError messagesCheck against baselinePast-due account reportsReview of activity logs

After Event Before Problematic Event

Preventive Controls*:Preventing problemsIncludes:FirewallIntrusion Prevention SystemProgrammed edit checksEncryption softwareWell-designed procedures, policiesPhysical controlsEmploy qualified personnel

CorrectiveControls:Fix problemsand preventfuture problemsIncludes:Rebuilding PCBackup proceduresReruns

Time Based SecurityCan we react to an attack quickly enough to

control it?Defense in Depth requires multiple layers

Exposure = Detection + ResponseProtection > Detection + ResponseEstimate Best and Worst Detection and

Response Time to get Exposure

Time-Based ExamplesExample 1: Defending a Castle Example 2: Home AlarmOn a hill or mountainHas a moatHas an outer wallTrees cut down around the

wallProtection: How long will it

take to get through the multiple layers of defense?

Detection: How long will it take for us to recognize an attack?

Response: How long will it take to react to an attack?

An apt. alarm beeps for 15 seconds waiting for a passcode to be entered

The alarm takes 15 seconds to dial the security company

The security company takes 30 seconds to inform the police

It takes the police 2-5 minutes to arrive at the site

Protection: It takes one minute to empty a

jewel box in the bedroom and walk out

It takes n minutes to steal all expensive appliances in a home with one person

More ExamplesExample 3: USS Cole

Example 5: Network Traffic Baseline

USS Cole Attack Response: Move all US military vessels out of foreign ports and onto the open sea

Example 4: Edge router, IDS, Firewall

Shadow IDS measures traffic and reports hourly of traffic against a baseline. What is best and worst Detection times?

Example 6: Sluggish Web service

What is best and worst Detection times?

Implementation: Measure D + R using stopwatch

Security DocumentationPolicy: Requirements Rule:

Describes ‘what’ needs to be accomplished

“Only students currently enrolled in computer science courses shall have access to the computer science lab”

Policy Objective: Describes why the policy is required

Policy Control: Technique to meet objectives

May include a procedure

Example 1:Policy Objective: Reduce

highway deathsPolicy Control: Set speed

limit to 55 Example 2:Policy Objective:

Differentiate between different users on a system

Policy Control: Logon restrictions, smart card, biometric authentication

Discussion: Are these effective controls by themselves?

ProcedureProcedure: Outlines ‘how’ the Policy will be

accomplished1.“The CS System Administrator shall provide a

list of student IDs to the lab entrance system by running the XXX program using the YYY file one week before classes begin.”

2.“Students must slide their student ID card through the card reader and enter the last four digits of their SSN to gain entry at the CS lab door”

BaselineBaseline: Snapshot of a system in a Known

Good StateIs a static measure of a systemEnables recognition of changes in system via

activity profilesEnables description of how a system has

changedMost useful when generated automatically

Example BaselinesExample Baselines:Network Traffic Baseline: Shows traffic

volume per hour of day (Wireshark, Shadow/NFR IDS, etc.)

System Baseline: Shows OS version, available disk space, description of system files, size of different major directories…

Start-> Run-> winver: Prints the version of OSStart-> Run-> psservice > Export: Saves

system baseline info

Preparing a BaselineBest: Take a copy of a new system or

To achieve Known Good State:Update virus signaturesEnsure system fully patchedDo comprehensive virus scanCheck all files (not just system files)Turn on heuristic virus scanning, which

recognizes suspicious patterns in addition to signatures

Save baselines to CD for offline storage

Auditor ResponsibilitiesResponsibilities

include:Measure and report

on riskRaise awareness of

security issues in order to reduce risk

Often provide input to policies and procedures

Raising Awareness:It’s not ‘if’ we’ll be

hacked but ‘when’.You can never be too

secure…

Audit Plan OutlineObjective:What do we hope to accomplish or measure through the

audit?

Scope: What part of the organization are we auditing? Can audit a process, a technology, a department/divisionExample: “Enumerate vulnerabilities for X web server”

Can include:Validation: Are rules implemented correctly?Baseline Comparison:

Measure conformance to policyMeasure if system has been compromised

Audit Plan Outline Cont’dPolicy Resource Guidelines: Documentation for existing and recommended security guidelines

Audit Strategy: A definition of how the audit will occur. What tools and

techniques will best meet the objectives?

Checklist: Each policy has a number of checklist line items Each checklist line item describes a procedure of what and how to

measure a policy

Signatures: On cover page request signature of the audit team, the instructor,

and the team from the audited company. Make sure that both you and company have signed copy of Audit

Plan

Policy Resource GuidelinesCompany policies: Statement in full or summary

Best Practice references:Center for Internet Security: www.cisecurity.org Provides documents that can easily be used as part of an audit checklist, including

procedures, standards, tools, benchmarks

ISO/IEC 27001:2013 and 27002:2013 Information technology -- Security techniques -- Code of practice for information security management.

International Standard

COBIT: Control OBjectives for Information and related Technology: www.isaca.org IT-oriented framework for control and mgmt of corporations Adherence to Sarbannes Oxley (SOX)

NIST: National Institute of Standards and Technology www.nist.gov Set of Standards (FIPS) and Guidelines (Special Publications) Adherence to FISMA: Federal Info Security Mgmt Act.

Audit Process OutlineAudit Process includes:Audit Planning: Create Audit PlanEntrance Conference: Inform people of

processFieldwork: Measurement of the systemReport Preparation: Complete reportExit Conference: Discuss report with

affected personnelReport to Management: Provide revised

report to management

Step 1: Audit PlanningAuditor works with contracting individual to

determine scope/purpose of auditResearch corporate policies, industry best

standardsPrepare audit strategy, checklist, and audit

procedures

Step 2: Entrance ConferenceAuditor meets with all people involved in the auditMgmt schedules the meeting, including mgmt, security,

system administrators, users being audited (e.g., if random workstations are being audited, those users shall attend)

Manager introduces you and explains purpose of audit and discusses his/her support for audit

Auditor then takes control over meeting to discuss: Audit Scope/Objectives Auditor’s role Role of others Audit Process Timeframes: Make appointments with all parties you need

to during the meeting.Take team approach: Do not offend anyone or play power

games. People should be excited, not intimidated by you.

Step 3: FieldworkAuditor performs audit (often with worker)Report facts as you find them – as a detective

wouldEven if a security vulnerability is fixed when

found, still report the vulnerability and the fix

Results and RecommendationsAudit Exceptions: Items that fail to meet the

audit criteriaMitigation: Recommendation to reduce

loss/harmRemediation: How to fix an Audit Exception,

by policy, procedure, best practiceRoot Cause: Why is there an audit exception?

Treat the illness, not the symptom

Step 4: Report PreparationInclude Objective/Scope of auditDevelop technical write-up of report first

What organization does wellWhat organization needs to do better If system administrator patched a hole, mention that

Organize findings in a logical way.Write Executive Summary last

Put Executive Summary as first section in ReportExecutive Summary should be understandable to non-

technical executive managerDescribe good and bad points in bullets (Make people look

competent)Your report must be written professionally, if it is to be

credible.Have another writer/auditor proofread

Step 5: Exit ConferenceAuditor communicates findings to entire team

Exit Conference Team = Entrance Conference Team

Go over Executive Summary firstThen give a copy of Audit Report to the teamTeam may defend themselves in meeting.

Discussion (not argumentation) is healthyAmend report after meeting if new information

arisesBe careful in wording: “Best Practices include …”

NOT “Most administrators know better than …”Stay out of arguments if you can

Step 6: Report to ManagementPrepare PowerPoint

Presentation (Plan for 60 minutes)

Power Point should include:Audit purpose, scope,

goalsExecutive Summary:

Positive and Negative points

Schedule 2-hour meeting

Meeting PointersHave highest executive schedule the meetingHighest executive kicks off the meeting. Auditor then

takes overGive out copies of power point slides – executives love

themPresent for ½ hourGive full report out and take 15 minute break. This break

gives mgmt a chance to talk to technical staff and ask questions

After 15 minutes, start promptly again (or try to)Complete reportPut a list of names of people who did exceptionally well –

and should be encouraged and retainedAnswer additional questions when report is complete

Additional Recommendations Clear up scope/purpose in one meeting (You will look unprofessional if

you keep returning for clarification) Do not test/venture beyond what is agreed will be done. Extraneous

information is not always welcome Do not go beyond scope – do not demonstrate vulnerabilities for legal

reasons Always maintain a professional demeanor – not too chummy or

informal Always have company representative present who is most

knowledgeable about the matter being validated Company retains control: No surprises in tests, results Work together: Two heads are better than one Work with in-house expert. Involve them. Be humble Teach each other: Teach someone to fish is better than giving them a

fish Discuss your findings with the in-house experts as you find them.

There should be no surprises in the exit conference

Oh yeah – dress well!

Audit Report OutlineAudit ObjectiveScopeExecutive SummaryResultsReferences

Audit Report ExampleAudit Purpose:Determine amount of traffic not related to

business goals. Identify potential risks and additional controls.

Scope:Determine the internet traffic at headquarters,

including which applications are run, by whom, and when. Determine which web pages are accessed both internally and externally. The time frame for measurement is one week.

Audit Report Example (2)Executive Summary:“At least M% of bandwidth is used for chat, external

email, SSL, streaming media. N% of web references are for non-business use. External email is prone to viruses not protected by company email screeners. Most illegal web use comes from Building 205, 206, and in particular, the Sales department.”

Recommendation:Block chat IP/port addresses in firewall. Train management on handling inappropriate use of

time.

Audit Report Example (3)Results –Verification: Best-in-class standards (i.e., COBIT) define that

policies should be written and communicated to employees relating to what they can and cannot do [1]…

Results - Validation: This section shows line charts demonstrating

usage for each protocol type per hour of working day (on average). It also shows pie charts showing usage of different categories of web page accesses. Actual results are provided in Appendix A.

Changes for University EnvironmentSANS recommends providing a technical

summary of the results of the checklist tests. However, the professor needs to see more detail Each checklist item must describe:

1) the procedure of how to measure the policy2) the outcome of the test3) any recommendations arising from the audit step.

This technique allows the instructor and the organization to learn how the auditor arrived at his or her conclusions, and determine the validity of the report.

Audit Report Example (4)References:IT Control Objectives for Sarbanes-Oxley, 2nd

Ed., Exposure Draft, IT Governance Institute, April 30, 2006.

How to Stay Out of Jail!Audits often require scanning a network to

determine open ports, open applications.Results can include:

Aborted production systemsVERY upset administrators and managers

The difference between a hacker and a security analyst is PERMISSION!!

Your written permission is your GET OUT OF JAIL card.

To stay out of jail and keep your job Get permission in Writing! Plan to scan one subnet at a time! Pick an off-peak time in case something

does go wrong. Publicize the scan! The managers and system administrators must know

the exact date and time of the scan. Eventually something will go seriously wrong, so always take

precautionary steps. System administrators who go into panic mode for hours over your audit

will not appreciate you! Be present! Be available for the entire duration of the scan, in case

something does go wrong or you do get questions. Also, expect to answer questions up to a few days later.

Be persistent! Be careful to check all devices within the scope. False positives and false negatives occur, so be extra careful.

Provide Feedback! When the audit is complete, report to the system administrator or network manager and help them fix vulnerabilities. Complete the cycle within schedule, then begin scanning the next subnet.

Note: If a host reboots due to an audit scan, it would have happened with a hacker – just a matter of time.

Example Written Notice Subject: Security Audit Tuesday Oct 10  Next Tuesday, Oct 10, from 4-6 PM we will be conducting an

audit of the firewall. We plan to validate the services that the firewall allows to pass through, both inbound and outbound. As part of this audit’s scanning process, a significant number of TCP and UDP packets will be generated, and some ICMP packets. Specifically, we will be scanning ports 1-NNNN with a UDP scan, a SYN half-open scan and a full tcp-connect scan. In order to try to minimize any significant impact to the firewall operations, we will generate packets slowly, at the rate of 1 packet every X seconds.

  During the scan period, I will be available in room XXXX. I will

also be reachable via phone: 255-5466; via pager 262-445-9933; or email: youraddress@cs.uwp.edu. I will be happy to reply to any questions or concerns, and provide more detail about our audit if necessary.

SummaryStay out of Jail:Get signature on audit planBroadcast what you plan to do whenOnly do what is in the audit plan

For this class:Be very specific about what tests you did and what results

you gotBe sure you have a member of the organization with when

you do audit – allow them to see all problems at time of audit

Double-check with me before submitting proposal or report to your customer – submit most professional document

Summary ReviewSecurity Cycle Review

Verification: Is our plan effective?

Validation: Is it really protecting us?

Controls (Preventive, Detective, Corrective)Policies & ProceduresBaseline

Security PlanSecurity Report