NA-40 Training OverviewJanuary 31, 2014

EOTA Qualification Standards

Qualification Process

Analysis

Evaluation TasksCurriculaExercise

Objectives

3

ADDIE Model• Systematic instructional design model used

globally:

• Analysis

• Design

• Development

• Implementation

• Evaluation

Analysis• Performs Feasibility Study

• Identifies learning problem, the goals, and objectives

• The audience’s needs

• Existing knowledge

• Learning Environment

• Any constraints

• Delivery options

• Timeline

Design

• A systematic process of specifying learning objectives based off of tasks

• Design Document – outlining details of course or exercise, including types of evaluation

Exercise Design Document

Development• The actual creation (production) of the content and learning

materials based on the Design Phase/Document

Exercise Design Document

Implementation• Training is executed utilizing ISO registered processes for:

Web-Based/Computer-Based

Instructor-Led

Self-Study

Blended

Webinars

Mobile Training Teams

SharePoint and Learning Management System (Defensibility)

Repository

Version Control

Easy access from remote locations

Evaluation

• Formative

• Is present in each stage of the ADDIE process

• Monitors student learning to provide ongoing feedback throughout

curriculum

• Summative

• Evaluates student learning at the end of the course

• Tests

• Feedback from students

Kirkpatrick’s LevelsFour Levels of Evaluation

1. Reaction - immediate satisfaction2. Learning - results in increase of knowledge/skills/attitudes3. Behavior - transfer from classroom to job (3-6 months post training)

4. Results - can be measured in reduced cost, improved quality, increased production, efficiency, etc.

Learning Management System

• Handles course distribution and course management

• Facilitates course exams and allows item analysis

• Course registration/repository

• Creates custom reports

• Generates certificates

NA-40 Training

EOTA

DOE

MANAGEMENT RESPONSIBILITY- Provide Direction/Set Standards- Quarterly Reviews- Reports/Metrics- Provide SMEs- Monitor Students/Customers Satisfaction Ratings

RESOURCES- Experienced Staff- Objective Assessments- Standardization- Formal Review of Courseware- Partnerships- Training Technology- Tracking/Reporting Capabilities- Document Repositories- Collaboration Tool (SharePoint)

QUALITY- Maintain ISO Registration- Corrective/Preventive Action- Audits/Reviews (internal/external)

INPUT- Management Support- Feedback- NA-40 Program Knowledge- Subject Matter Expertise- Lessons Learned from Training, Exercises and Real World Events

OUTPUT- Defensibility- Clearinghouse- Repository for all NA-40 Courses- Credibility- Customer Trust- Consistent Products- Standardized Methods- Reliability

NA-40.2 Technical

Training Liaison

Exercise Planner Support

• EOTA offers exercise capabilities and a performance-based exercise program for exercise design, development, conduct, evaluation, and improvement planning• Standardized policies • Methodologies• Terminologies

• “Reachback” to Subject Matter Expertise• On staff controllers, evaluators, and facilitators

Exercise Types

• Discussions-based • Seminar• Workshop• Tabletop

• Operations-based• Drill• Functional Exercises• Full-Scale Exercises

Implementation

• Development of resource documents

• Situation Manual

• Exercise Plan

• Controller/Evaluator Handbook

• MSEL

• Player Handout

• Exercise Evaluation Guides (EEGs)

• After Action Report

After Action Reports

• After Action Report (2 sections)

• Capture observations/recommendations (process driven)• What was accomplished• What worked well• What requires improvement (tactics/techniques)

• Participant Reaction Form Summary• Qualitative• Quantitative

ObservationsRecommendations

After Action Report

ParticipantReaction Form

Summary

Corrective Action Plan

• ISO registration includes:• Corrective Actions• Preventative Actions• Improvement Opportunities

• Assistance with CAP for Watch Office• Discussion• Path forward• Outline responsibilities• Tracking

Corrective ActionPlan

EOTA Exercise Progress

• Conducted a Feasibility Assessment• Analysis Completed

• Needs Analysis (helped identify if training was truly the answer)

• Training Requirements Analysis (focused on what the regulatory requirements

are that pertain to training and or areas of responsibility)

• Resulted in Standard Curriculum

Note: Review sample DIF surveys

Exercise Curriculum• Web-Based Training

• EXR101DW: Intro to the DOE/NNSA Exercise Program• EXR102DW: Exercise Design• EXR103DW: Exercise Scenarios• EXR104DW: Completing the Exercise Plan• EXR105DW: Post Exercise Activities• EXR106DW: Optimizing Your Site Exercise Program• EXR107DW: Controlling Security Exercises• EXR108DW: Designing Exercise Evaluations• Basic Exercise Controller and Evaluator Tutorial (ORISE)

• Instructor-Led Training• EXR 207: Controlling Security Exercises Practicum• EXR 231: Exercise Design Laboratory (Pilot course January 14-16, 2014)

Where we are going…

• Clearinghouse– Validate content and format for consistent results

across all vested customers• Repository for All NA-40 courses– One area to look for all NA-40 available training as

well as direction to locate other training offerings available (FEMA, DHS)

• Defensible Training