AUTOMATE TESTING USING MITRE ATT&CK® WITH VERODIN SECURITYVALIDATION

Leverage continuous validation to proactively understand exposure and optimize defenses

2© Verodin 2020

INTRODUCTION1

MITRE ATT&CKis a cybersecurity framework helping to bring communities together to develop more effective approaches to cyber defenses. ATT&CK is a living knowledge base of the tactics, techniques, and procedures (TTPs) used by adversaries in the real-world. Organizations are adopting MITRE ATT&CK as a foundational element of their security programs, but they are challenged by the resources and skills required to continually determine the effectiveness of controls relative to the framework. In this paper, we will outline how security validation technology enables organizations to automate their ATT&CK- based effectiveness programs to optimize their defenses.

About MITRE ATT&CK Framework

The MITRE Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK) Framework represents a key resource for security teams that manage defenses against threat actors. MITRE ATT&CK categorizes known adversary behaviors into Tactics and Techniques to help organizations focus

on tuning their defenses to identify and detect discrete malicious behaviors of threat actors in their environment. Adversaries have learned to adapt and easily morph their attacks to evade traditional signature-based detections, leaving indicators of compromise (IOCs) valid for only a finite period of time. What has remained the same is the set of behaviors used by adversaries, such as how they interact within a system, to fulfill their objectives.

The Tactics and Techniques found in the ATT&CK framework are based on real-world activity including observed incidents. This gives organizations insight into the variety of ways adversaries have compromised real-world systems. This makes the ATT&CK framework invaluable in planning defenses for real environments.

The beginnings of ATT&CK were used in a MITRE project called FMX. The project team used the ATT&CK behaviors to build adversary emulation scenarios. These scenarios were created to inject real-

world malicious activity into the production network. The team was able to validate that network sensors were working to detect the adversarial behaviors. The approach was tremendously successful and resulted in the acceleration of improvements to the detection capabilities, and most importantly, the testing was done in a repeatable, measurable way.

ATT&CK became the go-to map for defenders to plan their improvement programs and to verify their progress. The framework proved to be such a useful tool in the MITRE research program that they decided to release it to the public in 2015 to solve cybersecurity problems for a safer world. Since its introduction,ATT&CK has grown significantly, such as expnding from a Microsoft Windows focus to including techniques used against macOS, Linux, mobile, cloud, and Industrial Control Systems (ICS).

3© Verodin 2020

ATT&CK TACTICAL IMPLEMENTATION: THE CHALLENGES

The implementation of an ATT&CK-based testing program can often require a substantial investment in highly skilled resources and the time needed to plan, create, execute, and analyze the results. According to MITRE best practices, the implementation of a program starts with defining the specific threats the organization is looking to defend against.

From this starting point, the team must map these threats to the relevant ATT&CK Techniques from the framework. Once mapped, the team will analyze and organize by the operational flow of the organization to create a plan for evaluating their readiness to defend against these Techniques.

The organization must then develop the tools that would safely execute attacks for the relevant behaviors in their production environment to test their actual security controls. Once the tools are ready and these test attacks executed according to the plan, the team would collect, normalize and analyze the ability of their security controls to detect, block, and alert on these attack behaviors.

The goal being to conduct these evaluations on a regular basis to successfully create a program that would ensure that changes to the controls or related areas like the IT infrastructure wouldn’t negatively impact the organization’s security posture.

One of the biggest challenges for organizations is the availability of resources to effectively run a comprehensive program on a consistent basis. It may be possible to validate security defenses against targeted subsets of Techniques but broadly covering the entire ATT&CK spectrum on a daily or weekly basis is out the question for most organizations.

For organizations adopting the MITRE ATT&CK framework as the basis for the continued improvement of their cybersecurity program, automation becomes a necessary requirement for success.

2

4© Verodin 2020

During the MITRE FMX project, the team used emulations of threat actors’ behaviors on the production network to get a real-world assessment of security controls’ effectiveness. Their efforts focused on emulating the exact behaviors in the exact environment the adversaries would use to compromise the organization.

On the other hand, others have proposed using simulations as a safer approach, rather than attempting to model the target environment or modeling the attackers’ exact behaviors and techniques. Going as far as running defanged simulated attacks in a sandbox environment or a quasi-replica of the production environment. The challenge with using simulations or lab environments is that the results are frequently not representative of production environments and can increase the risk to the organization by providing a false sense of security.

Emulation guarantees that the results are exactly the same as those of real attacks.

When using simulations to measure attacks, the results may not be representative of your production environment due to inevitable discrepancies between the sand-box and production environments. Additionally, in many cases, security controls recognize the simulation for being exactly that, a benign activity, and deprioritize or even ignore it, leading to organizations thinking their controls are operating effectively when in fact they weren’t even effectively tested.

EMULATION vs SIMULATION

3

5© Verodin 2020

Verodin Security Validation platform is designed to safely emulate the behaviors of attackers in production environments. By using Verodin, an organization can to determine precisely how their security controls will react when different adversary behaviors are executed.

The Verodin Security Validation platform tests the controls within the production environment to provide evidence of effectiveness, identify gaps that may exist, and yield proactive and actionable insights to defenders so they can improve their security posture. Since Verodin is safely executing the same attack behaviors as adversaries, there is a high degree of confidence in the results. This ultimately assures that the business has thoroughly and accurately validated its defensive coverage.

In addition to the behavioral testing executed by Verodin, a comprehensive set of dashboards, graphs, gauges and reports provides information on the effectiveness of your security program. This provides organizations with a way to measure effectiveness at not just a point in time, but to also obtain continuous validation of that effectiveness.

Leveraging the reports and visualizations within the platform, it’s easy to communicate tactical detection gaps and configuration issues to the security analyst working in your security operations center (SOC), but also to report measurable effectiveness score on your whole security program to business executives including all the way to the board of directors.

BEHAVIORAL TESTING WITH SECURITY VALIDATION TECHNOLOGY

4

6© Verodin 2020

Verodin Security Validation platform provides organizations with a clear understanding of security effectiveness and is the only platform that can deliver the technology and process needed for an ongoing, automated, and effective ATT&CK emulation program.

Verodin has committed substantial resources to align to the MITRE ATT&CK framework within the Verodin platform. To start, one of the visualizations included in the platform is the MITRE ATT&CK Dashboard. The ATT&CK Dashboard offers a single pane of glass to view exactly how well various security controls react to the different Techniques for every Tactic. It works like this – every Verodin “Action” (behavioral test) in the content library includes

tags corresponding to related ATT&CK Tactics and Techniques. When a Verodin attack “Action” executes, the results are included in the MITRE ATT&CK dashboard and aligned to the framework.

Instead of building a program from the ground up as described in the MITRE best practices, Verodin provides a tremendous shortcut to quickly launching an effective program that can begin generating results for Techniques across all 12 ATT&CK Tactics within hours of initial implementation.

The Verodin security content library, the defenders can filter “Actions” by ATT&CK Tactic and Technique.

VERODIN ALIGNED TO MITRE ATT&CK FRAMEWORK

Verodin content library tags “Actions” (behavioral tests) to align to ATT&CK Tactics and Techniques. Defenders can easily sort on the available “Actions” to emulate attackers for the different Techniques.

5

7© Verodin 2020

This comprehensive content library of attack behaviors or “Actions” is available to all customers within the Verodin platform. Updates to content are made available by the Verodin Behavioral Research Team (BRT).Content updates are broken down into two categories: Headline and Baseline releases.

Headline releases aim to provide customers with new “Actions” based on the most recent behaviors seen in the wild. This can range from a recent 0-day vulnerability, to a widespread malware or ransomware campaign affecting multiple industry verticals, even behaviors being executed by a specific nation-state advanced persistent threat (APT) group.

Baseline releases provide customers with updates to existing content to provide both new and improved approaches to testing different behaviors. The Verodin platform is also highly customizable, providing all customers the flexibility to extend the content library by creating their own custom attacks. This allows organizations to create tests that are more specific to their environments or test exfiltration of critical data.

As explained in the Challenges section, the MITRE best practices for augmentation of an attack emulation program without the assistance of a security validation platform include steps to:

• Define the program based on the threats to defend against

• Map the threats to the relevant Techniques in the framework

• Analyze Techniques and create a plan based on the operational flow of the organization

• Develop tools to emulate the adversary behaviors

• Execute the tests and collect the results from the SIEM, security controls, and network traffic

• Normalize results and analyze to determine effectiveness in blocking the behaviors

• Research remediation information and approaches and make improvements to defenses

• Re-run tests and re-collect and re-analyze data

• Schedule time to repeat the testing process

8© Verodin 2020

With Verodin Security Validation Platform, these steps become much more simplified:

• Define the program based on the threats and Tactics most important to the organization

• Map the plan to the related ATT&CK Tactics to validate against

• Choose one or more “Actions” filtered by ATT&CK Tactic from the Verodin content library

• Execute the “Action”

• Review the results of the “Action” to determine if the security stack detected and blocked the behavior

• Make targeted improvements to defenses based on the detailed insights provided by Verodin platform

• Retest the environment with a single click

• Iterate over the last two steps until controls defend at acceptable levels and use these results as your new known-good baseline

• Add relevant “Actions” to a recurring test to continuously and automatically validate controls

• Gain assurance that security teams, processes and technologies can detect and/or block and alert on the attack behaviors moving forward

9© Verodin 2020

The MITRE ATT&CK Dashboard allows the defender to identify how well their enterprise environment is protected according to the MITRE ATT&CK framework. The initial view of the Dashboard shows the Tactics, ordered by the lifecycle of adversary behavior. Each Tactic displays the following information:

• Tests Completed – How many Actions were run that tested the ATT&CK Tactic

• Green box – How many of those Actions were blocked

• Red box – How many of those Actions were not blocked

• Grey box – No Actions ran for the Tactic

When a Tactic is expanded, the security technologies observed during the test are listed.

VERODIN MITRE ATT&CK DASHBOARD

6

Verodin Security Instrumentation Platform includes a MITRE ATT&CK dashboard showing how cybersecurity defenses either blocked or missed the associated emulated behavioral attacks.

10© Verodin 2020

Below the list of security technology icons, the Tactic’s Techniques are displayed. If any Actions executed for a Technique, the percentage of Actions that were blocked are displayed.

The Techniques listed under each Tactic are the ones that are included as tags on one or more Verodin Actions. All Actions map back to the dashboard by the Technique ID (e.g. ‘T1048’). You can use this identifier to search the security content library for Actions that will test that particular Technique.

Techniques that have related executed jobs can also be expanded. When a Technique is expanded, the security technologies that were seen during tests are listed, and a segmented bar displays the Actions that have been run.

Expanding the Tactic in the ATT&CK dashboard shows the security technologies that were observed as being involved, and the status of the behavioral tests aligned to the individual ATT&CK Techniques.

11© Verodin 2020

Clicking on a percentage will bring up all completed Jobs that are related to that Technique. For each job, expand to review the individual Actions and Events. This allows for troubleshooting and determination for why the Action was not blocked, or why a security technology couldn’t be identified. It also brings true understanding as to how the system performed in the past for each Action for easy comparison.

.

The Job Action summary in the Verodin Platform shows exactly how the security stack performed against the behaviors.

12© Verodin 2020

Using security validation technology as a guide, organizations can determine precisely where they should invest time and resources to improve cybersecurity. For example, if email controls are operating at a low detection rate for adversary behaviors, the information provided may demonstrate the need to invest in new controls to specifically defend against spearphishing links and attachments. Using Verodin Security Validation with the MITRE ATT&CK dashboard and the security effectiveness gauges demonstrates the ability of existing security controls to defend the organization.

Security validation frequently identifies deficiencies in configurations or capabilities of security controls as they relate to defending against relevant adversary behaviors. This further assists in identifying gaps in detection coverage and provides insight into specific areas of improvement for existing security controls.

For example, a Verodin customer in the government space found a next-generation firewall was reporting back to the SIEM that it has been detecting and blocking malicious traffic. Once the use of the Verodin platform challenged the security controls with behavioral attack tests, it became clear that the firewall was not actually inline blocking the traffic. For months, the organization was under the impression that they were blocking the attacks when in fact their firewall was simply being bypassed and doing nothing to block malicious network traffic.

IDENTIFY GAPS AND MAKE IMPROVEMENTS

7

13© Verodin 2020

To continuously improve cybersecurity posture, it’s important to not only identify and fix problems but also to quickly validate that the fix worked. Verodin greatly simplifies the process of re-running a test down to just a few clicks. And since the testing methodology is consistent, the results can easily be compared, and defenders can validate that the fix worked with a high degree of confidence.

Validating the security effectiveness of the cybersecurity program cannot occur as a single event at a single point in time. IT environments change quickly, especially in the cloud and simple updates to configurations may have unintended consequences to the security posture – what we refer to as environmental drift.

Verodin Security Validation platform provides the option for an Advanced Environmental Drift Analysis (AEDA) module. This capability enables the defenders to select an Action,

Evaluation or Sequence to automatically run on a pre-determined basis. AEDA Actions run silently in the background and immediately trigger syslog or email alerts to your security team only if the test results change from the known good baseline.

By automating the recurring testing against ATT&CK Tactics, the organization gains a new level of assurance that cybersecurity is properly defending the business.

CONFIRM IMPROVEMENTS BY RETESTING VALIDATION

AUTOMATE MONITORING OF CYBERSECURITY THROUGH BEHAVIORAL TESTING

8

9

14© Verodin 2020

In addition to emulating the behaviors behind the Techniques in the MITRE ATT&CK framework, organizations can benefit from proactively testing defenses against the specific adversaries targeting them by emulating the different behaviors these adversaries use to compromise systems. To bridge the gap between ATT&CK and today’s adversaries, Verodin’s Security Validation Platform leverages threat intelligence source through the Threat Actor Assurance Module (TAAM).

Verodin has established integrations with several threat intelligence providers such as FireEye Threat Intelligence, Intel471, ThreatConnect, CrowdStrike, and Anomali to develop the Threat Actor Assurance Program (TAAP) to aggregate the latest information on adversaries and provide insight through existing threat intelligence subscriptions. By ingesting feeds from one or more partners, Verodin Security Validation creates a comprehensive profile

linking the threat actors and adversaries to the behaviors they employ.

The tactics and techniques used by key threat actors of interest are also automatically turned into a set of test plans in the Verodin platform. These plans, called “Evaluations”, emulate the approaches adversaries use to compromise an environment. Defenders can execute each

“Evaluation” to gain insight into their readiness to defend against specific threat actors.

As threat intelligence providers update the techniques used by adversaries, Verodin

“Evaluations” are automatically updated, ensuring validation against the latest reported adversarial attack behaviors. The Verodin Threat Actor Assurance Module also includes an adversary based view of readiness allowing an organization to quickly determine their ability to defend against that adversary. Since the “Evaluation” is made up of Verodin “Actions” consisting of real attack behaviors, the results also feed into the MITRE ATT&CK dashboard.

EMULATE ADVERSARIES WITH THREAT INTELLIGENCE

10

Verodin Threat Actor Assurance Module provides evidence of an organization’s readiness against specific threat actors and their attack behaviors.

15© Verodin 2020

The use of the Verodin Security Instrumentation Platform allows organizations to accelerate improvements to their detection and prevention capabilities, and most importantly, to do it in a safe, repeatable, and measurable way.

By automating the behavioral testing aligned to ATT&CK, collecting and analyzing the results, and providing those results in easy to consume dashboard, Verodin enables organizations to gain assurance that they are prepared to defend against the latest and most sophisticated adversaries and their attacks behaviors.

For more information on the Verodin Security Instrumentation Platform, Advanced Environmental Drift Analysis and the Threat Actor Assurance Module, please contact:

verodin.com 571.418.8684

Verodin, part of FireEye, is a platform that has made it possible for organizations to validate the effectiveness of cyber security controls, thereby protecting their reputation and economic value. The Verodin Security Instrumentation Platform

(SIP) proactively identifies gaps in security effectiveness attributable to equipment misconfiguration, changes in the IT environment, evolving attacker tactics, and more.

By measuring and testing security environments against both known and newly discovered threats, Verodin SIP identifies risks in security controls before a breach occurs and permits companies to rapidly adapt their defenses to the evolving threat landscape. Verodin SIP does this by instrumenting an IT environment to test the effectiveness of network, endpoint, email, and cloud controls and provides quantifiable evidence that investments made in controls are truly delivering the expected business outcomes.

VERODIN ENABLES RAPID ADOPTION OF AN ATT&CK BASED EFFECTIVENESS PROGRAM

11

About Verodin

16© Verodin 2020