Mission Assurance Risk Management System

Antiterrorism / Force Protection Assessment Tool Training

Trainer: Caleb JonesContact: EPRMhelp@alionscience.com

Supporting Joint Staff J33 via US Army Armament, Research, Development and Engineering Center

1

Agenda• Module 1 – Foundational Points: (30 min) Slides

– Background on MARMS, Policy drivers, Terms, Role of Automation; Intro to EPRM

• Module 2 – Legacy Vulnerability Data: (30 min) Slides and live demo– Accessing legacy data. Managing corrective actions

• Module 3 – AT/FP Risk Assessments: (45 min) Slides and live demo– Conducting AT/FP risk assessments, analyzing and managing risk

2

Course Overview• Scope

– Primary: Focus on entering and managing Antiterrorism/Force Protection (AT/FP) assessment data

– Secondary: Future implications to Mission Assurance (MA) assessments

• Delivery method:– Lecture and demonstration

3

Terminal Learning Objectives (TLO)1. Understand the operational and policy drivers for MARMS and risk assessments (Why

and who)2. Understand the timeline for transition to EPRM MARMS modules (When)3. Describe a “risk scenario” and its components (What)4. Describe the benefits of risk-based assessments (Why)5. Understand how to access and update legacy vulnerability in EPRM (How)6. Understand the process of entering an AT/FP risk assessment in EPRM (How)7. Understand how to obtain EPRM account, training and help (How)

4

Module 1 – Foundational Points (30 min)

5

Why not vulnerability assessments?• Risk management has long been AT Standard #3 in DoDI 2000.16, however the process

& tool really focused on vulnerability• Previous CVAMP assessments, while good for an installation, made it very difficult to

aggregate or roll-up enterprise or regional views to expose trends: – Had little quantification of threats– Had little standardization in asset categories– Had no standardized relationships between benchmarks and threats– Had minimal functionality to facilitate the Risk Management process, so results

were difficult for leadership to assess where the greatest risks, and make investment decisions.

6

Why ‘new’ risk assessments?• The new method, better supports AT Standard #3 through:

– Benchmark focus: Walks assessors thorough benchmarks to provide leadership more complete picture of security posture…not just identified observations

– Standardization in threats & assets: Facilitates roll-ups and cross-unit reporting– Standardized risk framework: Has common relationships that help users prioritize

activities for their mitigation strategies– Aggregates risk results: Inherently supports trend and risk analysis at the

installation, regional, and enterprise level. • This will provide leadership with the data they need to make smart decisions on

where best to reduce risk on limited dollars.

7

Why use the new tool?• New tool has efficiencies to assist users in executing a quality risk analysis

– Pushes baseline threat levels by region or allows HHQ to develop localized threat baselines to push to ATOs

– Allows ‘copy from’ to leverage previous assessments. HHQ can create ‘Templates’ for common sites

– Users can export benchmark questionnaires exporting to an Excel spreadsheet for the other installation MA partners to complete their section, and import it back into tools

– Tools performs the approved math and presents results graphically and textually in Word, Excel and PowerPoint

8

Background on MARMS• The Mission Assurance Risk Management System (MARMS) is a Joint Staff initiative,

funded by DoD CIO and managed by the US Army Armament, Research, Development and Engineering Center (ARDEC)

• MARMS is a multi-year program that encompasses a family of systems that will be integrated as a part of MARMS Requirement Definition Package 1

• The second of MARMS’ capability drops (CD2) provides assessment tools that:1. Provide ability to hold and update observations from vulnerability assessments

currently in CVAMP2. Provide replacement risk-based capability to conduct AT/FP risk assessments 3. Provide follow-on capability to do risk-based capability to do MA assessments

9

Policy Drivers (TLO #1)• 2012 Mission Assurance Strategy and 2016 Mission

Assurance Assessments Concept of Operations:• Defines risk as a process integrating threat, vulnerability,

consequence (criticality)• Specifically includes installation-level AT/FP assessment as a

required component of the MA construct

• 2016 DoDD 3020.40 Mission Assurance: • Requires Components to “develop and implement a

comprehensive and integrated MA risk-management construct” and “align associated security, protection, and risk management efforts under an MA construct.

• 2018 J33 Mission Assurance System of Record Designation:

• Established MARMS as the replacement of the Core Vulnerability Assessment Management Program (CVAMP)

10

Timeline for Transition (TLO #2)• Phase 1 – Replace CVAMP & Provide AT/FP Risk Assessment Tool (Feb-Jun 2018)

• Cut-off of CVAMP data entry was 15 APR 2018, ‘released’ observations to migrate• Account requests by 15 MAY 2018 (for accounts on turn-on date)• Initial version of EPRM must be operational in place by 1 JUN 2018

• Provide management of migrated ‘observations’ from CVAMP• Provide installation personnel a mechanism to facilitate risk-based AT/FP assessments

• Phase 2 – Mission Assurance Assessment Enhancements (Jun-Dec 2018)• Frame Mission Assurance Assessments approach into assessment tool using guidance/input from DTRA JMAA teams• Develop and incorporate full MA assessment capabilities for fielding targeting 31 DEC 2018

• Phase 3 – MARMS Enhancements (Jan-Sep 2019)• Integration planning and execution with the MARMS Registry• Push ‘asset criticality’ from authoritative sources to MA & AT/FP assessors• Improved mission-risk analytics and dashboard capabilities• Improved Geospatial Risk Visualization• All development work on assessment tool complete by October 2019

CD2-Phase 1

CD2-Phase 2

CD2-Phase 3

11

EPRM Functionality• Walks users though the life-cycle of risk assessments

12

& Hazards

Assets (TLO #3)• Asset. A distinguishable entity that provides a service or capability.

– Assets are people, physical entities, or information located either within or outside the United States and employed, owned, or operated by domestic, foreign, public, or private sector organizations.

• Must have quantified (or qualified) value to the unit’s / organization’s missions

13

Asset criticality (TLO #3)Task Critical Assets (TCA) and Defense Critical Assets (DCA) are defined in DoDD3020.40 and have established criticality

Other assets are characterized by their criticality in 4 criteria (UFC 04-20-01 DoD Security Engineering Facilities Planning Manual)

• Criticality to Mission• Criticality to National Defense• Replacement (time, LOE) • Relative Value (monetary, classification, etc.)

14

Threats (TLO #3)Threat is any circumstance or event with the potential to cause the loss of or damage to an asset• Threats are considered in terms of a threat source (sentient actor or natural hazard), a

threat tactic (threat method) and a severity or likelihood.

15

Threat severity (TLO #3)Threats are characterized by their severity (UFC 04-20-01 DoD Security Engineering Facilities Planning Manual)

• Local Activity• Intentions and history • Local Operational Capability• Local Operating Environment

16

Vulnerabilities (TLO #3)A situation or circumstance which, if left unchanged, may result in the loss of life or damage to mission-essential resources from a terrorist attack. (DoDI O-2000.16-V1)Vulnerabilities can result from characteristics of

– building characteristics– equipment properties– personal behavior– locations of people, equipment and buildings– operational procedures and personnel practices

• List of potential AT/FP vulnerabilities are drawn from the 2018 DoD Mission Assurance Assessment Benchmarks https://intelshare.intelink.gov/sites/jmaap/SitePages/JMAA%20Home.aspx

• Each benchmark can reduce vulnerability one or more threat tactics

17

https://intelshare.intelink.gov/sites/jmaap/SitePages/JMAA%20Home.aspx

Risk Scenarios (TLO #3) • Risk is calculation that is based on ‘risk scenarios’ • A risk scenario has:

– Asset with a criticality (C) on a 0-1 scalelinked to a:

– Threat adversary-tactic combination (T) on a 0-1 scale of severity/likelihoodwith a:

– Vulnerability to the tactic (V) calculated on a 0-1 scale

𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅 = 3 𝑇𝑇 ∗ 𝑉𝑉 ∗ 𝐶𝐶

Risk =

18

Analysis of Risk Scenarios (TLO #3)• Risk is understood by evaluation “risk scenarios” in accordance with approved metrics

19

Benefits - risk-based assessments (TLO #4)• Provides standardized/common analytical framework

• Converges multiple protection disciplines into a common sight picture

• Allows roll-up of multiple units into a single analysis

• Supports commanders in making better informed decisions on where to best allocate security resources

20

CJCSM 3105.01, Figure 7

Module 2 – Accessing Legacy Vulnerability Data and Updating ‘Corrective Actions’ on ‘Observation’

(30 min)

21

MARMS Module AccessOnly designated users will see icon Legacy Assessment Data

22

CVAMP Starts with Quad Summary

Installation users currently land on this CVAMP page.

Will have them land on different page, but will provide access to these statistics

23

Mapping ‘Legacy Assessment Data’ moduleHierarchy Node (unit) Attributes

Not importing,

Focus Area

Not needed; no new assessments

Is a query tool; will handle with advanced analysis grid

24

Linked to observations & Hierarchy Node Attribute

CVAMP ‘Manage Observations’ Screen

Will show details of observations in tabs below grid

Will call up window for data entry

Note column headers

Will use new sorting and

filtering fields

Replace tabs to a ‘status’ Duplicative

25

CVAMP ‘Observation Detail’ screen

Button is on Management Grid

Use existing feature

In tab below ‘observation’

Fields to be in tabs below observation management grid

Not editing ‘released’ observations

26

MARMS “Manage Observations” screen

Headers match CVAMP ‘Observation Management’ screen. (Some additions.) Mouse-

overs for full text.

Upon selecting an observation in the grid above, data renders below. Tabs match sections in “Observation Details”. Data fields match sections.

27

Replacement for ‘search’ & 2 tabs

Search window replaced by text filters and sorting. User can sort or filter on the various grid fields to view observations falling into

specific criteriaUse of ‘Status’ column eliminates need for

‘No action required’ and ‘Risk Accepted Tabs’

28

Attachments & Statistics

‘Corrective actions’ for selected observation

‘References & attachments’

Will pull up statistics page that is the CVAMP

landing page

29

Excel Report

User can export the grid data to Excel, just like CVAMP. Moving functionality to larger button at top

30

CVAMP Corrective action input screen

“Corrective Action” button will put up editable window, like the CVAMP window,.

Will show previous corrective actions in tab below “Observation Management” screen.

New features to allow users to revert back from ‘closed’ and ‘risk accepted’

31

Demo of ‘Legacy Assessment Data’ Module

32

Module 3 – Entering AT/FP Risk Assessments (TLO #6) (30 min)

33

• “Start” assessment brings assessors to the workflow (below) to collect data. • Opportunity to ‘copy from’

• Each icon takes users to the appropriate screen

Starting a risk assessment

34

• Guides personnel through standards-based assessment; fillable forms for each step

CD 2 Phase 1 – AT/FP Risk Assessments

35

Profile the Organization

36

Mouse-over info bubbles provide guidance

• Profile and Scope screens contain information that: – Filters subsequent screens– Provides ‘hooks’ on which queries can be

conducted– Collects data that can be inserted to the MS

Word Assessment Report

• Select and score assets. Add comments / justifications

Name

Export to Excel for off-line data entry

Asset identification

37

Local name of assetAsset Subcategories

Pull-down list / filter of all Asset

Groups

• ‘Yes’ selection triggers questions from UFC 04-020-01 (DoD Security Engineering Facilities Planning Manual)

• Responses to questions calculate criticality on 0-1 scale

• TCAs use pre-scored criticality from authoritative source

Asset characterization

38

• Threat/hazard assessment is filterable, sortable, printable

• Preloaded with regional baseline

Name

Duplicate Selected Threat

Threat selection

39

Relevant Adversary-Tactic Pairs

Default Adversary Threat Level preloaded by region

Local name of Adversary

Threat characterization

• ‘Yes’ selection triggers questions from UFC 04-020-01

• Responses drive 0-1 score• Current ‘baseline’ preloads are available

based on region

40

• Filterable list of benchmark ‘questions’ with assessor guidance

Drill-down questions, where appropriate

Description / assessor guidance window

Assessing to benchmark standards

41

Export list in Excel for off-line entry & upload

Observation made. View / edit with icon

• The contribution of individual benchmarks is used to model vulnerability levels to individual threat tactics/hazards.

Communicating aggregate ‘vulnerability’

42

Calculate risk by individual scenario

43

• Risk scenarios viewable on Risk Assessment Tool dashboard

Threat adversary / tactic with 0-1 scale

for severity

Vulnerability to tactic calculated on 0-1

scale

Asset and criticality on 0-1 scale

Calculated Risk Score

Analyze risk contribution of benchmarks

44

• Mitigation dashboard prioritizes benchmarks based on contribution to risk mitigation

Assessor proposes mitigations and can assign to an individual and provide due

date

Amount that implementation will

reduce overall risk profile

Cost benefit analysis

45

• Cost Benefit Analysis (CBA) provides commanders a framework for risk-based allocation of resources– Can be used for Integrated Priority List, POM & budget exercises

• Mitigation dashboard ranks benchmarks based on the amount of risk they reduce– If cost estimates are entered for proposed mitigations, system compares the risk reduced per

dollar spent– The comparison is a relative calculation that can be done for security measures in a single

assessment or across a collection of assessments

Cost benefit analysis

46

• Total costs and risk-reduction-per dollar calculated

• Drop-downs for status of funding for selected remediation

Reports

47

Current and revised by asset • Generate editable report

contains a combination of:– Boilerplate with system

generated insertions (e.g. dates, installation name)

– Tables with system generate insertions (e.g. team member, asset lists, etc.)

– Outputs from risk analysis– Comments, observations and

other assessor-entered text

Output of prioritized mitigations and status of implementation plan

Reports

48

• Reports of risk by unit/installation & benchmark implementation

Relative risk of units / assessments or

installations

Benchmarks along left & units/installations

along top

Finalizing a risk decision

49

• Installation personnel can review all proposed mitigations on mitigation dashboard to:– Accept or reject proposed mitigations– Develop proposed implementation schedule– Assign responsibility for a mitigation to installation personnel (email automatically generate to

them and task added to their dashboard)

• Submit completed package for Commander’s approval

Documenting recommendations

50

• Document risk acceptance or reduction– Yes = Accept Risk– No = Reduce Risk

• Identify target dates for implementation

• Comments

• Document recommendation for Commander to either Accept or Reduce overall risks to installation

Obtaining Commander’s approval

51

• Commander approves assessment results and releases risk decision packageReview history of

assessmentReview risk and

mitigations Approve and release

Managing implementation of decisions

52

• Finalized assessment results are locked and released

Continue to manage implementation of

mitigations

Risk scores update to show progress towards risk goal

Attachments

53

• Signed reports (and other artifacts) uploaded to assessment

User Support (TLO #7)• Requesting Access - Email the following information to caleb.l.jones.ctr@mail.mil and

raleigh.a.onks.ctr@mail.mil or (SIPRNET) raleigh.a.onks.ctr@mail.smil.mil– Name– Title/Rank– Phone Number (NOT DSN)– Service or Component– Major Command (i.e. MAJCOM or ACOM)– Installation (i.e. base, post)– Unit– NIPR E-Mail– SIPR E-Mail– Type of account required: MARMS, OPSEC, IP, DODInt

• Accessing system: (SIPRNET) https://eprm.csd.disa.smil.mil• Help: For assistance and for any questions, please email EPRMhelp@alionscience.com or call 1-800-754-

4204. 0700-1700 Eastern time• Resources: User guides, videos & other materials are available on the EPRMHelp page and on EPRM in the

resources section (MARMS users guides are currently being created and will be added soon). http://eprmhelp.countermeasures.com/marms.html

54

mailto:caleb.l.jones.ctr@mail.milmailto:raleigh.a.onks.ctr@mail.milmailto:raleigh.a.onks.ctr@mail.smil.milhttps://eprm.csd.disa.smil.mil/mailto:EPRMhelp@alionscience.comhttp://eprmhelp.countermeasures.com/marms.html

Mission Assurance Risk Management SystemAgendaCourse OverviewTerminal Learning Objectives (TLO)Slide Number 5Why not vulnerability assessments?Why ‘new’ risk assessments?Why use the new tool?Background on MARMSPolicy Drivers (TLO #1)Timeline for Transition (TLO #2)EPRM FunctionalityAssets (TLO #3)Asset criticality (TLO #3)Threats (TLO #3)Threat severity (TLO #3)Vulnerabilities (TLO #3)Risk Scenarios (TLO #3) Analysis of Risk Scenarios (TLO #3)Benefits - risk-based assessments (TLO #4)Slide Number 21MARMS Module AccessCVAMP Starts with Quad SummaryMapping ‘Legacy Assessment Data’ moduleCVAMP ‘Manage Observations’ ScreenCVAMP ‘Observation Detail’ screenMARMS “Manage Observations” screenReplacement for ‘search’ & 2 tabsAttachments & StatisticsExcel ReportCVAMP Corrective action input screenSlide Number 32Slide Number 33Starting a risk assessmentCD 2 Phase 1 – AT/FP Risk AssessmentsProfile the OrganizationAsset identificationAsset characterizationThreat selectionThreat characterizationAssessing to benchmark standardsCommunicating aggregate ‘vulnerability’Calculate risk by individual scenarioAnalyze risk contribution of benchmarksCost benefit analysisCost benefit analysisReportsReportsFinalizing a risk decisionDocumenting recommendationsObtaining Commander’s approvalManaging implementation of decisionsAttachmentsUser Support (TLO #7)