Security Whitepaper - Threat Intelligence Visibility - The Way Forward

Threat intelligence visibility – the way forwardMike Adler, Senior Product Manager Assure Threat Intelligence

The modern challenge

Today, organisations worldwide need to protect themselves against a growing range of devious and diverse threats. They also find themselves having to tackle the conflicting challenges of meeting compliance regulations.

The picture gets even more complicated when you consider the vast variety of security systems that customers deploy, and the enormous quantities of data they produce.

Organisations also use an enormous array of device types, ranging from firewalls and Unified Threat Management to tablets, smart phones and specialised devices such as SCADA.

So what organisations really need is a security ecosystem that is adaptable and has a flexible ability to connect, gather information, and convert it into actionable intelligence. It also needs to provide users with both broad and detailed visibility of their entire network.

In this white paper, we outline ways to enhance security analysts’ capabilities and situational awareness, creating a more effective and optimised security environment for any organisation.

A complex and global issue

These days, all organisations are affected by global connectivity and threats, and need to be able to fine-tune their protective capabilities constantly to reflect this.

Organisations connect to systems outside their core network that they don’t control – and unless a network is physically isolated it can no longer guarantee who is blocked from access. So an organisation’s protection system needs to be able to monitor, gather and analyse information regarding every aspect of activity within the environment, and respond.

As today’s networks are often large and cumbersome, approved devices may also have unexpected connections to devices that aren’t approved. Without the right tools, it’s hard to determine exactly what is connecting to the device.

Even some specialised security tools can make the situation worse, because they are only designed to tell you about a specific event or type of traffic, and cannot give information about the threat context if it relates to other events the system does not cover.

All of this complexity forces analysts to face a multitude of outputs that aren’t easily reconciled. This makes trying to identify broader potential threats nearly impossible. Using so many divergent layers of systems is awkward and creates conflicting information, which means it can be hard to identify what information to act upon.

| 2Threat intelligence visibility – the way forward

This requires not only being able to view activity, but also having a clear set of intelligent analytics that shows what the activity might mean as part of the total network threat picture.

Today’s complex security environment, with its silos of isolated products, needs deep analytic capabilities that can act as a comprehensive threat intelligence environment, revealing not only the general state of threats but also detailed activity.

For these capabilities to work, an organisation needs advanced overall network threat intelligence that filters out noise and spots activities that should be causing concern.

On top of removing clutter, a system should give analysts easy and visual access to information about the overall computing environment, before using analyst and system intelligence to provide deeper analysis and response.

The analyst’s capacity can be enhanced with a system that uncovers and highlights events accurately, reflects the current threat state, and provides easy access to this information.

Reducing unnecessary data, allows the analyst to focus on critical information that can improve discovery and response times. An intelligence system should provide a set of flexible, user configurable views to show information of interest, using graphics to make it easy to read.

An example of this overall view is shown in figure 1.

This image shows a selection of pull-down options to the left, with a set of individual views (Portlets, of which there are nine here). Portlets provide a snapshot of monitored network elements. Examples by number in the portal image: 1) General ‘Dashboard’ includes overall view of events being monitored as represented in an upside down 2) Service metrics pyramid with a 3) Top events pie chart view of the same information below the pyramid. The view below the pie chart at bottom is 4) Blacklisted hosts.

At the top middle is 5) Problem ticket and change requests for client tickets. The portlet in the centre is 6) Device status.

On the far right is 7) Country geolocation of events, and then middle right are displayed rules, 8) Rule utilisation that govern filtering and responses to identified events.

The last to the right is 9) IP destination for a service, and in the bottom centre are 10) Top unique pairs for firewalls and IDS devices according to source/destination addresses.

These areas are discussed in more detail on the following page – in each of these windows an analyst can enlarge and drill down for more detail of that information category.

Figure 1

Visibility and intelligence requires a single pane of glass

Organisations can’t get an overview of their overall security by switching between a myriad different systems and consoles. To get effective visibility, an organisation must have some means of identifying and reviewing potential or active threats all together.


Further analysis can then be based on previously-agreed operational standards for an analyst to follow in dealing with individual events or groups.

The service metrics section in the dashboard provides an at-a-glance event insight by using a set of clear views of everything happening within the network. The event views are populated with information captured by pre-selected security and other devices, which have gone through initial processing and categorisation by pre-set filters and rules.

The service metrics screens represented by figure 2, provide a view of logged events both before and after they are processed through the filters. The filters both reduce event noise (low or non-impactful events) and isolate and identify events of interest according to filter rules.

These rules can be added to, or changed, to continually tune the system and the filters for greater efficiency and to address network or other environmental changes. Identified events are further analysed by one or more embedded systems engines that can correlate information in different ways for selected events.

This allows the analyst to see a separate list of items that may have had action taken, resulting in a ‘ticketed’ event that might need further review and potential action. The graphics below make it easier for the analyst to understand current event status.

The view highlights the unfiltered, filtered and correlated event traffic down to a ticketed event. With additional screens as noted in figure 1, a reviewer can drill in, and get more detailed views of the traffic displayed in different ways better understand event status and potential responses.

Figure 2

Anyone managing security first needs to get an overview of what’s taking place, and then can drill into areas needing further investigation – a system can be designed to highlight specific events or categories for this.


Reliability – a key metric for active security

If a system can be health-checked to make sure it is operating correctly, security teams can trust that it is active and providing event data.

Enabling dynamic changes to changing conditions

The areas that security professionals need to monitor and react to in their computing environment are changing constantly. This constant churn comes from system updates, patches, adding new users and removing others, actions by internal people or external organisations, and changes to the type and volume traffic circulating in the computing environment.

Figure 3

Figure 4

Both traffic type and volume are variable, and analysts who manage and protect networks and content need tools to adjust their view of this information and respond to the ongoing changes.

They also need to be able to adjust their protection stance to reflect these changing states by managing, creating and altering protection rules that have an impact on the computing environment.

For instance, figure 4 shows a view of the Rules Engine in the BT Assure Threat Monitoring system. The Rules Engine allows viewing of existing rules so their structure can be understood and any impact can be analysed to show how well they are providing intended filtering protection.

There is also a ‘Rule Builder’ capability which allows an authorised user to change or add new rules to address new threat models, or to tune a system further with changes or an extension to an existing rule.

The system provides analysts with a seamless and rapid view into multi-tiered, multi-source rules and allows them to review how any rule or set of rules could impact event correlation.

The baseline of the system includes more than 50 built-in threat detection functions, or rules, that can be further tuned. These can also be used as guides to create new rules by an analyst. This means a virtually unlimited combination of dynamic threat detection models, so the analyst can respond flexibly to events by tuning or creating custom rules on demand and in real time.

Without verifiable system reliability you can’t be sure all events are represented, or that it gives a correct picture of the network environment – and that means you can’t analyse activities without the data variations that unknown system failures can create. Verifiable system reliability also means information from past and present systems can be fairly compared.

System health metrics can show the average status of a device in simple operation, and they can also summarise the current or average health status of single devices or groups, as shown in figure 3.

To achieve reliable network event status, you need to gather information on all devices connected to the network, as well as their uptime. The information can then be used to review the reliability of each device.

This view provides at-a-glance information analysts can use initially to determine if a device is operating correctly, and then for more detailed drill-down views of individual or groups of devices historic and current status.

As is the case for a number of the views (portlets) in the overall UI, users can filter for particular elements. An analyst can gather more information via filter regarding a specific individual device type or a group of devices, its current event activity and health history.


Visible event logging

Being able to classify and display data in a clear, understandable form means security professionals can do their jobs more easily. This requires a tool that can collect and display the detail of current or past events so it can be compared and can support in-depth analysis, helping analysts expose unusual characteristics of events.

Selective, dynamic visualisation

Being able to view, or ‘slice’ information in diverse ways can have a significant impact on recognising whether an unusual event took place, or isolating a normal event.

Separating events into different types of ‘what if’ ad-hoc comparisons requires powerful Log Management Search and Retrieval functionality.

The search capability means analysts can search with flexible parameters for types or classifications of information, helping to separate key events from general network traffic. The view of this search and retrieval capability is shown above in figure 5. Analysts can choose a standard or customised view.

Figure 5

The search and retrieval system allows analysts to view a device’s raw log data, and export the logs if they want to carry out further analysis.

This log information can be derived using specific search parameters, with the results run through an additional analysis tool. It’s a powerful and flexible tool that clarifies and exposes issues, enhancing analysts’ capacity to address a wider variety of threat event scenarios, and decide how they might want to deal with them.

The view in figure 5 shows how an analyst can search on events via areas such as device name, source IP, destination IP, individual port and other areas critical to understanding what events may be current or emerging within the network.

It can help identify when an event might have taken place, over a specific time frame and within a particular device or group of devices. For example, if events that might be considered normal are occurring outside normal hours, something untoward could be taking place.

Viewing the aspects of an unusual event graphically, as shown on figure 6, can help highlight or flag points for further analysis and aid analyst exploration. Viewing the relationships between events – both security and otherwise – can enhance an analyst’s decision-making and response options.

Figure 6


Making sense of events

With so much data flowing around a network, attempting to identify and correlate it into a usable form is obviously difficult.

Figure 8

Figure 7

Intelligence for action

A system that is constantly scanning for events needs to go through a process of elimination that can be cumbersome and time-consuming if not done well.

Using NetFlow to capture, process, correlate and de-duplicate NetFlow information with other log data feeds such as IDS/IPS and firewalls is an important element in this area.

If analysts can compare and correlate events identified by different systems, they can better isolate items that need further investigation. It can also help them compare events that on their own seem innocent, but require further investigation when seen in context with events from other systems.

There is no such thing as to much information, but you can certainly have too much unusable information. Huge volumes passing through a network make it easy to miss information that may be an indicator of troublesome activity.

Figure 7 shows information important to an analyst, such as traffic flow trends, top source of events and top destination. It also has graphics that show top traffic areas, and geolocation of top country for both source and destination of traffic and the source IP for blacklisted sites. These all provide an analyst with added intelligence about the traffic transiting their network.

If traffic noise (non-critical or normal events) is not reduced, an analyst ends up with far too many events to be able to pull out important ones for added investigation. The volume of noise in normal network event traffic takes valuable time and resources and can easily misdirect an analyst from areas needing advanced investigation.

A system can be designed to extract notable events based on an organisation’s standard rules and policies. These can then be highlighted by a ‘ticket’ that denotes further action is required. An integrated ticketing system needs to allow complete event management, with built-in communications and troubleshooting by both customers and analysts.

In the process of pulling out a ticketed event, noise is easily excluded, and remaining events filtered from the noise are then exposed. The reduced set of events that actually generate a ticket, means analysts can get quick snapshots of their status and conditions and summarise any attention required to a customer.

The results of this process can be combined with links to ticket drill-down so an analyst can gain quick access to ticket updates and troubleshooting information as well as pulling up more information regarding relevant device activity.

Figure 8 shows the Problem Ticket portlet, with information regarding the status of a ticket (new, closed, re-opened, etc) and the date created as well as the name of the ticketed event. It also categorises the severity of the event, and can have filters set to search through the event logs for particular aspects of an event or group event tickets.


Bringing it all together

There is no magic bullet for better security. Several standards organisations recommend various approaches, but threat monitoring is the common thread that can provide the ability to detect and identify a variety of activities and act accordingly.

The fact is, no organisation alone can expect to have all the internal resources needed to address all the requirements for securing their systems and content.

One problem is keeping staff up to date. The number of tools, staff, and required specialisation to cover the huge array of available security solutions has become too large for most organisations to manage. There are tens of thousands of organisations that need security, but aren’t security companies – still have to act like one, creating a budget to cover for expensive expertise.

Being able to do this effectively is becoming more difficult as threats and the solutions to deal with them become more complex, and organisations need to look at how they can extend their security reach within their limited budgets.

The goal should be to enhance their security footprint and to increase overall threat intelligence without the complexity or expense of trying to add costly but limited internal personnel or systems.

One way to address this problem is for organisations to reach beyond their own internal security resources and add global intelligence and visibility by associating with other professional groups or service organisations.

That way, their partners’ extended capabilities will improve their proactive capabilities to protect themselves – the aim is to protect against threats that are known globally but haven’t yet appeared in their local environment.

With a greater global visibility of current or suspicious activity an organisation can pinpoint areas of concern – and make the necessary response in good time to protect both itself and its customers.

Organisations need superior intelligence and visibility capabilities to allow them to get on with their core business. Obviously, security must play a part within every organisation, but it must do so at the highest level in order to contend with the reality of today’s cyber threats.

These demands require a security ecosystem that extends the organisation’s capabilities both within and outside their principal environment. And it has to account for the global connectivity that every organisation and individual contends with today.

Organisations have to be able to adjust to activity volumes and continuous change in the threat landscape by applying intelligence and visibility capabilities, and using dedicated expert security resources. This should provide them with access to a global knowledge base of security issues, so they can increase their protection footprint more effectively than by simply hiring more staff and layering on yet more security systems.


Technology

Security Whitepaper - Threat Intelligence Visibility - The Way Forward