11

TENA and JMETC for Distributed and Cyber Test and Training

Gene Hudgins

TENA and JMETC User Support Team Lead

Gene.Hudgins@TENA-SDA.org

2

Cyber Test & Training Relationship

2015 NDAA specifies collaboration between DoD Components and DoD EAs for Cyber Training and Test Ranges to:

Ensure interoperability with other DoD training & test infrastructure (kinetic and non-kinetic)

Develop cyber test and training infrastructure interface standards and other technical and operational standards

Establish a standard language (data exchange protocol) for representing and communicating cyber event and threat data during a cyber-range event

TRMC has a 16+ year history of using TENA to meet these requirements for non-cyber test and training

TRMC intends to use TENA for cyber test range integration

TRMC is proposing leveraging TENA to meet above DepSecDef and NDAA requirements

3

TRMC Priorities

• Produce High Fidelity Investment Roadmaps

o Develop infrastructure enhancements (implement as much capability in software as possible) to support drivers resulting from the Third Offset Strategy

o Work with OSD(R&E) and the Service S&T communities to better forecast technologies that will transition into weapons systems

• Gather Validated Infrastructure Enhancement Requirements Through the Reliance Process (Closer Cooperation with Services)

• Develop Better Standards to Promote Interoperability and Technology Insertion Throughout the T&E Infrastructure

o Data correlation between modeling & simulation, installed test facilities, and open air testing

• Defend the MRTFB as Critical Infrastructure

o Defend against encroachment and provide for sustainment

• Develop and Retain Highly Skilled T&E Workforce

• Ensure Nuclear Survivability

4

Urgent Capability Acquisition

• 2017 National Defense Authorization Acto Section 901(c) – Reorganized USD(AT&L) to separate Development for Acquisition

o House Report – Directed a review of the T&E Enterprise’s posture to support urgent capability acquisition

o Senate Report – Directed a review of the balance of resources to support DT and OT

• The T&E community must be positioned to support urgent capability acquisition (UCA)o Strategic rebalancing

o Increasing threat capability

o Shortened threat system update cycle

o EW and cyber capability costs

• The Intel Community/Test Enterprise must have a robust relationship to support UCAo In-place, advanced T&E capabilities

o Current, flexible and scalable threat capability-emulations

o Swift integration of emerging threat capabilities

o Extensive/tailorable threat model libraries

5

Risk mitigation needsTechnology shortfalls

The TRMC “Blueprint”

Capabilities

Service Modernization

and Improvement

Programs

Acquisition Programs and

Advanced Concept

Technology Demonstrations

T&E Multi-

Service/Agency

Capabilities

DoD Corporate

Distributed Test

Capability

TRMC

Joint Investment Programs

Transition

Requirements

Annual T&E Budget

Certification

MRTFB

Oversight

Defense Strategic Guidance

Service T&E Needs and Solutions Process

Acquisition Process

Capabilities

Requirements

T&E Range Oversight

Strategic Plan for

DoD T&E Resources

Risk mitigation solutions Advanced development

S&T Investments

(COI)

6

Vision: Agile Test Infrastructure for Today’s Acquisition

7

JMETC Distributed Test Architecture

7

Systems

Under

Test

Joint Operational Scenarios

Integrated

Test

Resources

JMETC

Infrastructure

on DREN

* TENA: Test and Training Enabling Architecture

VirtualPrototype

Hardwarein theLoop

InstalledSystems

TestFacility

Range EnvironmentGenerator

ThreatSystems

TENACommon

Middleware

TENACommon

Middleware

TENACommon

Middleware

TENACommon

Middleware

TENACommon

Middleware

TENACommon

Middleware

Reuse Repository

TENAStandard Interface

Definitions

TENAStandard Interface

Definitions

TENAStandard Interface

Definitions

TENAStandard Interface

Definitions

TENAStandard Interface

Definitions

TENAStandard Interface

Definitions

Distributed Test Support Tools

JMETC depends on TENA to support distributed testing

8

Joint Mission Environment Test Capability

(JMETC) Program Investment Areas

• Distributed Testing (Events, Tools, Expertise, etc.)

• JMETC Secret Network (JSN)

• Test & Training Enabling Architecture (TENA)– TENA Object Models

– TENA Web Services

– TENA Software Repository

– TENA Tools

• Big Data / Knowledge Management Initiative

• National Cyber Range Complex (NCRC)– National Cyber Range (NCR)

– Regional Service Delivery Points (RSDPs)

– NCR Expansion (Service Sites)

• JMETC MILS Network (JMN)

• Executive Agent (EA) for Cyber Test Ranges

9

Joint Mission EnvironmentTest Capability (JMETC) Benefits

Acquisition Programs, Testers, & Evaluators

Enables early verification that systems work in Joint and Cyber contested environments

Test whether systems work well together Test whether systems are resilient to cyber threats Identify issues early when they are less costly to fix

Provides access to high-demand, low availability systems

Supplements number of live Systems Under Test (SUTs), threats, or “supporting cast” to create a realistic environment

Feasible alternative to Live testing in early DT and risk reduction for OT

Provides access to cyber ranges

Ability to conduct unconstrained but nondestructive cyber activities in representative environments

Provides a collaborative engineering environment

Gives SMEs an opportunity for collaboration without leaving home station

Supports all aspects of testing across the acquisition lifecycle

Interoperability, cybersecurity, rapid fielding, DT, OT, etc.

Reduce Acquisition Cost, Schedule, and Risk

10

National Cyber Range Complex Vision

JIOR -

JMN

SCADA

SITES

• Common Cyber Testing Ontology, Taxonomy,

and Lexicon

– How we communicate

• Common Concept of Operations

– How the cyber ranges operate

• Common Cyber Test Data Model

– How to describe a cyber test

• Common Cyber Descriptions (Object Definition

and Transforms)

– How components of the cyber test are

defined

• Common Cyber Universe Description

– How the environment that surrounds the

components of the cyber test are defined

• Common Instrumentation and Control

– How data is collected and assessed

– How cyber tests are controlled

TECHNICAL CHARACTERISTICS

• Realistic: Large-scale, high-fidelity virtualized cyber environments operating actual software integrated with hardware-in-the-loop capabilities

• Repeatable: Archived, reusable environments, procedures, parameters, and event restoration checkpoints to facilitate test-fix-test verification

• Rapid: Standard tools and processes to automatically create, re-create, and modify mission-specific environments

• Isolation: Cryptographic segregation of multiple, concurrent cyber environments at varying security classifications

• Sanitization: Restore all assets to a known, clean state – not just range infrastructure, but also mission system equipment

E n a b l e d b y a C y b e r - S a v v y W o r k f o r c e

Manage and operate the National Cyber Range (IAW RMD 407A1, Issue #1, Title: Cyber, Jan 12, 2015)

to provide test capability and capacity for the T&E Community

FY17 FY18 FY19 FY20 FY21 FY22 FY23

20,813$K

19

11

NCRC Key Capabilities

Rapid emulation of complex, operationally representative network environments

Red/Blue/Gray networked environments

Operational systems (e.g., weapon, C2, business, etc.)

Realistic traffic types, flows, and scale

Customized instrumentation

Automation provides significant efficiencies that enables high OPTEMPO

Time to deploy environment baseline on the order of hours, not weeks or months

Minimizes potential for human error and ensures capability to replicate scenarios and phenomena

Sanitization to restore all exposed systems to a known, clean state

Allows assets to be reused even when they are exposed to the most sophisticated or uncharacterized malicious code (i.e., “non-destructive” cyber testing)

Support multiple concurrent events at varying classification levels

Events, users and data are isolated

Secure connectivity

Integration of distributed capabilities (e.g., HWIL, SILs, etc.)

Remote user access

12

T&E Infrastructure Interoperability

12

Driver Challenge

Goal/EndstateSolution• A highly flexible infrastructure for operationally realistic T&E.

Existing Components:

• Integration Architecture: Test and Training Enabling

Architecture (TENA)• Mature, continuously improved software architecture (15+ years)

• Architecture enables integration of Red/Blue capabilities

• Interface Standardization: TENA Object Models• Interface definitions for integrating both Red and Blue systems

• Common Tools: JMETC Tools and TENA Utilities• Proven tools suite to rapidly integrate and operate LVC-DE

• Corporate MILS Network: Joint Mission Environment Test

Capability (JMETC)• Mature, continuously improved network infrastructure (9+ years)

• Used to support 250+ distributed test events since 2007

• DoD test range infrastructure was not originally designed to

be interoperable. Impacts:

• Data loss between systems degrades test quality

• Government maintains redundant system interfaces

• Changes break functionality between related systems

• Producing complex, immersive environments for T&E is

expensive and time consuming. Impacts:

• Systems do not get stressed against realistic

conditions

• T&E infrastructure won’t be able to meet demands of

agile acquisition

• Persistent network connectivity between government and

industry test locations

• Interoperability between inter- and intra-range Test and

Training assets

• Testing that can span kinetic and cyber test infrastructure

• Elimination of proprietary interfaces on test infrastructure

• Pre-Integrated common tools that reduce resources

needed for test planning, execution, and analysis

• Pre-Integrated scenarios that can be pieced together into

complex and challenging test environments via automation

• Sharing and reuse of common capabilities across existing

and new investments

Goal: Make complex, realistic T&E environments efficient and routine

13

TENA at a Glance

What does TENA enable? Interoperability between inter- and intra-range assets

Elimination of proprietary interfaces to range instrumentation

Integration of multiple vendors/providers

Efficient incremental upgrades to test and training capabilities

Integration of Live, Virtual, and Constructive assets (locally or distributed)

Sharing and reuse of common capabilities across existing and new investments

What is included in the TENA architecture? Customizable “data contracts” that standardize repeatable information exchange

Interoperability-enabling, auto-code generated software libraries

A core set of tools that address common test and training requirements

Collaboration mechanisms that facilitate sharing and reuse

TENA has a plan for continued evolution and funding to execute this plan

TENA is DoD’s GOTS range integration architecture

14

How TENA is currently Used InTest and Training Facilities

Common specifications for test and training data

Data Dissemination across variable applications, platforms,

programming languages, networks, and classification levels

Data Collection and Playback

Local and Remote Command and Control

Health & Status Monitoring

Real-Time simulations

Stimulation of live sensors and instrumentation

Connecting non-interoperable inter- and intra-range systems

Eliminating proprietary interfaces to range instrumentation

Sharing and reuse of common range tools and capabilities

Online Collaboration and File Sharing

Data

Management

Event

Management

Sharing &

Reuse

LVC

Integration

These activities are all relevant to cyber experiments

15

Where TENA Fits in Cyber

TENA

Communication &

Tools

“Real World” / “Tactical”

Traffic

TENA

Communication &

Tools

From CRIS WG Event Interoperability Fundamentals for Cyber--‐Range Tools and Processes

Solution:

TENA is an implemented integration architecture;NOT a cyber tool suite

16

Benefits of TENA for Cyber Training

TENA saves money through composability & reuse All TENA software and support is free to users

TENA has standard LVC object models enhancing interoperability

The TENA web site has extensive documentation, training, and collaboration capabilities

The DoD has invested significant resources in already proven TENA-enabled tools & utilities

TENA Auto-Code Generation makes creating a TENA application as simple as possible

Auto-generated starting points mean you never start with a blank page

Rapid development of real-time, distributed, LVC applications

Auto-generated test programs greatly reduce system integration time & effort

TENA’s technical approach emphasizes rapid integration and reliability TENA is the most capable and sophisticated interoperability solution

TENA software is thoroughly tested

The TENA software is hard to use wrong

TENA catches many user errors at compile time rather than run time

TENA Tools provide unprecedented understanding of an event

Requiring vendors to use TENA interfaces promotes a modular open cyber test and training architecture

17

How TENA Supports Events:Notional Walkthrough

Test

Execution

Event Construction,

Setup and Rehearsal

Test Planning &

Requirements Definition

Test Design

Pre-Test Test

Analysis & Reporting

Post-

Test1

2

3

4

5

TENA

Repository

TENA Object

Models

TENA Tools

& Utilities

Test

Execution

Examples

TENA Data

Collection

System

TENA enables efficiencies through

inherent interoperability and reuse

18

Summary

TENA is a proven modular and open architecture that meets 2015 NDAA guidance

Establish a standard language (data exchange protocol) for representing and communicating cyber event and threat data during a cyber-range event

TENA provides the capabilities necessary for use as a foundational architecture for Cyber Test and Training

A suite of software and best practices matured over 15+ years

Free tools for common Event Planning, Execution, and Analysis functions

Auto-code generated for 68 operating system / compiler combinations and C++ / Java / .NET programming languages

Leverages, but does not require, use of data standards

Includes support for other transport protocols in its architecture

Institutionally Resourced and Sustained

TENA is THE DoD standard for integration of range systems for efficient T&E and training

An open architecture that promotes modularity reduces integration time and cost

100% Government off the Shelf (GOTS)

A rich history of integrating disparate, often proprietary, systems over 15+ years

19

Summary

Vision: Make distributed T&E routine

JMETC Mission: Robust distributed T&E infrastructure and subject matter expertise

The JMETC Team is here to help Event Planning / Design / Development / Integration / Monitoring /

Execution for cyber and non-cyber T&E

TENA “Free Upgrade” support offer

Local and Distributed Network Engineering support

Corporate knowledge of assets available for distributed use

Information Assurance / Cybersecurity assistance

• JMETC investments are driven by user requirements• Deployment of JSN / JMN nodes are based on user need

• JMETC provided tools and services are based on user input

20

Important Contact Information

Project Website: https://www.tena-sda.org/

Download TENA Middleware:

https://www.tena-sda.org/repository/

Submit Helpdesk Case:

https://www.tena-sda.org/helpdesk/

Use for technical questions regarding TENA

TENA Feedback: feedback@tena-sda.org

Provide technical feedback on TENA Architecture or Middleware

Ask non-technical questions regarding TENA

Provide responses to AMT action items

Request TENA training

21

Reference Slides

22

TENA “Cyber Crawl” Demonstration Overview

Goal: show how to use TENA within a Notional Cyber Training Use Case: Red Cell: Initiates attacks to systems on a network

Blue Cell: Defends attacks identified on systems on a network

White Cell: Monitor and Controls Cyber training Events

Demonstration is NOT meant to be a Cyber training solution Notional system characteristics, cyber attacks, and defenses are used

3 Generic System characteristics are represented in this demonstration

Red Cell attacks affect these system characteristics in a systematic way designed to be easily detected

Blue Cell monitors, detects and responds accordingly

For this example, TENA is used for: Distributing system diagnostics

Distributing Red Cell attacks and attack information

Distributing Blue Cell defenses and metric information

Providing Exercise Control for the Training Event

Providing Event Monitoring for the Training Event

Providing data recording for the Training Event

23

TENA “Cyber Crawl”Notional Cyber Training Environment – Design

(Applications and Systems)

Training Network:

Multiple Systems

networked together

White Cell:

Monitoring

Training

Event

(Starts/Stops

exercise)

Blue Cell:

Identify and

Defend

Cyber

Attacks on

Systems

Red Cell:

Initiates

Cyber

Attacks on

Systems

TENA Data Collection

System (TDCS)• Records all TENA

traffic on the network

• Generates SQL file for

analysis/playback/etc.

Development Metrics :- 6 days of Total Development

- Use Case Development- Application Design- Interface Development- GUI Development- Application Development- Integration Test

- 3.5 Hours of TENA (<10% Time)- Developing TDL- Submitting TDL for Auto-code

Generation- Downloading TENA OM package- Integration

24

TENA Cyber Demonstration Notional Cyber Training Environment –System and Cyber Attack Parameters

hood88.jm

etc

.mil

192.1

68.3

.88

5%

12%

98%

A System is a device on a network represented by a name and IP address

A system has three Characteristicsmeasured in percentages:

CPU Utilization

(Normal Operating Range : 0-20%)

Memory Utilization

(Normal Operating Range : 0-20%)

Network Throughput Availability

(Normal Operating Range : 80-100%)

Attacks happen against systems which will adjust the system characteristics

Corrective Actions are responses to observed changes to system characteristics.

58%

100%

25

TENA Cyber Demonstration Training Run – White Cell

White Cell monitors and Controls Exercise

1. Starts/Stops Exercise

2. Monitors Environment

4. Grades Actions

3. Monitors Actions

5. Records Exercise

TENA Data Collection System (TDCS) is recording all TENA traffic on the network and generates a SQL file for further review/analysis. A Log file of all info displayed in the “Alerts” window is also generated.

26

TENA Cyber Demonstration Training Run – Red Cell

Red Cell initiates Cyber Attack on Systems

1. Selects Systems

to attack

2. Selects Attack(s) 3. Initiates Attack(s)

27

TENA Cyber Demonstration Training Run – Blue Cell

Blue Cell monitors Systems and takes Defensive Actions

1. Selects Systems

to Defend

2. Selects Actions(s) 3. Initiates Action(s)

28

TENA Cyber Demonstration Notional Cyber Attacks and Defenses

Attack Type Observable System EffectsExpected Corrective

Action

CPU UtilizationCPU characteristic

increases abnormallyCPU Utilization

before 100%

Memory UtilizationMemory characteristic

increases abnormallyMemory Utilization

before 100%

ThroughputThroughput characteristic

decreases abnormallyThroughput before 0%

Memory Utilization +

Throughput

Memory characteristic increases +

Throughput characteristic decreases

abnormally

Memory before 100% +

Throughput before 0%

CPU Utilization +

Memory Utilization

CPU + Memory characteristics

increase abnormallyCPU + Memory

Before 100%

CPU Utilization +

Throughput

CPU characteristic increases +

Throughput characteristic decreases

abnormally

CPU before 100% +

Throughput before 0%

29

What Did You See?

1) Cyber Distributed Event Support Network Setup/Monitoring - TENA/JMETC Network Tools Event Execution - TENA Integration and Monitoring Tools (TENA Console, Tena

Protocol Dissector (TPD)) Event Monitoring / Situational Awareness (SA) – TENA Standard OMs Physical/Virtualized Stimulation – TENA OMs and Capabilities for Threat Emulation

2) Instrumentation Control and Monitoring (Alerts and Summaries) Cyber Test Analysis and Simulation Environment (TASE) like use case

Controlling data recorders Controlling/monitoring Red Cell capabilities/actions Controlling/monitoring Blue Cell defense capabilities/actions

Control and Monitoring Threat Emulators Integration into LVC environments/applications

3) Analysis Support (TENA Data Collection System (TDCS)) Data Collection/analysis of Cyber activities to support forensics LVC and System analysis for Cause & Effect of Cyber activities Playback for detailed investigation and training

Cyber interoperability & event management requirements mirror non-cyber training / test environments

30

TENA Mission

Historically, range systems tend to be developed in isolation, focused on specific requirements, and constrained by aging techniques/technologies

Range infrastructures have grown organically with minimal coordination or sharing, resulting in duplicated effort and many “stove-pipe” systems

Working with the Range Community to Build the Foundation for Future Test and Training

Range Infrastructure

The purpose of TENA is to provide the necessary enterprise-wide

architecture and the common software infrastructure to:

➢ Enable interoperability among range, C4ISR, and simulation systems

used across ranges, HWIL facilities, and development laboratories

➢ Leverage range infrastructure investments across the DoD to keep

pace with test and training range requirements

➢ Foster reuse of range assets and reduce cost of future developments

31

What TENA Is…

An open architecture that promotes modularity reduces integration time and cost

100% Government off the Shelf (GOTS)

A rich history of integrating disparate, often proprietary, systems over 15+ years

A suite of software and best practices matured over 15+ years

Free tools for common Event Planning, Execution, and Analysis functions

Auto-code generated for 34 operating system / compiler combinations and C++ / Java / .NET programming languages

Leverages, but does not require, use of data standards

Includes support for other transport protocols in its architecture

Institutionally Resourced and Sustained

Constantly improved to meet new user requirements

Subject Matter Experts for distributed exercise and system integration

TENA IS DoD’s DoD’s range integration architecture

32

Worldwide Use of TENA

TENA is used in

13 countries

outside the US

33

Demonstration