IBM zEnterprise System - Network Security

IBM zEnterprise System - Network Security July 2010

IBM zEnterprise System - Network Security


2

Table of Contents Abstract.............................................................................................................................3 Network Firewall Introduction............................................................................................3 IBM zEnterprise System Network Security Overview........................................................ 6

899

121314171819

zEnterprise Security Framework.................................................................................... Physical Infrastructure ................................................................................................... Logical Security ............................................................................................................. IEDN Workloads and Security Zones .......................................................................... External Network Access.............................................................................................

Exploiting External Firewalls ........................................................................................... Summary ......................................................................................................................... Acknowledgments and Contributions............................................................................. About the Authors: ..........................................................................................................


Abstract This paper is provided for both IBM and IBM customers who have an interest in the IBM zEnterprise™ System “Network Security” topic. It is assumed that readers already have a basic background in both the zEnterprise System and fundamental Network Security concepts. The primary purpose of this paper is to describe why, for many customers, traditional network firewalls will not be required for their network traffic associated with multi-tier application workloads within a zEnterprise Ensemble. This subject is organized into the following three topics:

1. Network Firewall Introduction General introduction and overview of the concepts of network security zones and how firewalls can be used to control security zone crossings for network traffic related to multi-tier workloads

2. IBM zEnterprise System - Network Security Overview Overview of the zEnterprise System internal (intraensemble) data network and how the IBM zEnterprise Unified Resource Manager (zManager) provides innovative network management features such as network virtualization, access control and network isolation to protect the heterogeneous platforms within the Ensemble

3. Exploiting External Firewalls How (when required for compliance, mandated standards, and / or other business imperatives) customers can continue to exploit their existing external firewalls to provide the same level of protection for resources within the Ensemble (i.e. when network traffic crosses intraensemble security zones).

Network Firewall Introduction One of the core security technologies common in most, if not all, network-attached computing environments, large or small, is the firewall. Firewalls take many shapes and forms, from host-based solutions targeting the personal computer as an integrated security suite to large dedicated purpose built appliance hardware protecting high volume traffic at the network’s edge. There are hundreds of variations on firewall solutions and their uses, each with their own value add or benefit in a particular situation, but there is one clear requirement that firewalls bring to the table no matter what size or how many bells and whistles are present. Firewalls must have the ability to block access or connectivity that is deemed as unauthorized, while still letting authorized traffic reach the intended target system or application.

In its simplest form the firewall acts as a basic packet filter, looking at each packet and checking a set of rules or policy to determine which packets are granted access, passing through the firewall, and which packets are denied. This basic packet filtering capability can be found in both network firewalls (either hardware or software based) and host firewalls. Host solutions, like that found in IBM’s Proventia® Server for Linux® on IBM System z® or z/OS® Communications IP Filters,

3


run within the server image and are used to protect network traffic flowing into and out of the server. These types of host solutions are targeted at self-protection. Another firewall solution that might be found on the host is an application firewall, designed to protect a particular application or server, such as a Web server, FTP, database, Telnet, etc. from unauthorized or malicious attack.

Security Zone 1 Security Zone 2 Security Zone 3

Figure 1: Basic DMZ

It is the job of a network firewall to ensure that only the intended traffic passes though the firewall from one security zone to another security zone. One of the classic deployments of a network firewall is the Demilitarized Zone, or DMZ, which can be defined as a perimeter network, located between an external network and a private or protected network, that provides isolation for a publicly available service in the perimeter network, with the ultimate goal of protecting the private network, data and services in the enterprise. An example of a DMZ can be seen in figure 1. The yellow perimeter network, identified in security zone 2 might include a web server, which is available to the Internet or external network shown in the red security zone 1. There is no path directly from the untrusted red external network to the trusted green private network found in security zone 3. In this case the DMZ, with its two firewalls that bound the perimeter network, isolates the enterprise resources from the threats of the Internet.

As virtualization continues to drive consolidation on all platforms, the network in these environments will need to be evaluated for compliance with the corporate security policy as well as with possible regulatory requirements. Different network architectures might need to be explored including the use of proxy servers, virtual LANs, Network Address Translation (NAT), and other technologies in conjunction with a basic packet filter or stateful firewalls. In a distributed environment it might be simpler to segregate security zones, deploying each zone on isolated, inexpensive

External Network

Private Network

Firewall

Firewall

Perimeter Network

DMZ

Publicly Available Application

4


hardware. As you explore the benefits of deploying diverse workloads on System z and the advantages the platform has to offer, it becomes necessary to reexamine the goals, intentions and requirements of network security in the light of this environment. It is always important to question and revise security decisions as environments and threats evolve. Reevaluating the placement and need for firewalls is no different. They need to be placed where they make sense and were they provide value.

System z is now an integration of new multiple architectural technologies with the introduction of the IBM zEnterprise System. It is comprised of the IBM zEnterprise 196 (z196), the IBM zEnterprise BladeCenter® Extension (zBX) Model 002, the Unified Resource Manager, and optimizers and IBM blades. The heterogeneous resources of the zEnterprise System are managed and virtualized through the Unified Resource Manager as a single pool of resources, providing integrated system and workload management across this multisystem, multitier, multiarchitecture environment.

One function of the Unified Resource Manager is network virtualization management, including the provisioning of a secure private data network called the Intraensemble Data Network (IEDN). The IEDN is designed to ensure the safety, security and isolation of network traffic into and out of applications running within this environment. It is important to ensure the right technologies are used in network flows to minimize latency while providing the required level of security and isolation of intellectual property and mission critical applications and data. Understanding the level of security required and the isolation provided by the network virtualization management function of the Unified Resource Manager in collaboration with other firmware elements of the IBM zEnterprise System will help clients determine what, if any, additional security devices, like firewalls, are required in their enterprise solutions.

As the end-to-end solutions that clients build or consolidate on a System z Enterprise System are explored, the security requirements of that solution must be understood. Careful consideration should be given to the security requirements. These requirements might be born out of a security policy that identifies various security zones and transitions, or it might be based on regulatory requirements such as PCI DSS (Payment Card Industry Data Security Standard). Either way, the requirements are real and must be addressed in a way that satisfies the client and in many cases the auditor. The determination of the requirement for, and placement of, network firewalls needs to be reevaluated with a new understanding of the security and isolation inherently provided by the IBM zEnterprise System as an integrated and optimized multiplatform environment.

5


IBM zEnterprise System Network Security Overview This section provides an overview of the zEnterprise physical infrastructure associated with network communications. Key concepts such as the node, how a cluster of nodes can be formed into an “Ensemble”, and finally how network communication is provided for within the Ensemble are also introduced in this section. The resources within the ensemble are managed across heterogeneous platforms by an innovative zEnterprise function called “Unified Resource Manager”. Unified Resource Manager will orchestrate various forms of platform management and virtualization by interacting with various elements of platform firmware and hardware.

Figure 2: System zEnterprise

Figure 2 illustrates a z196 with an attached zBX. A z196 can support up to four locally attached racks in a zBX. Together the CPC and the optional zBX are considered a single logical node. Individual nodes (up to eight) can be grouped into an Ensemble which is defined at the HMC. The Unified Resource Manager (HMC) can then manage the resources of the entire Ensemble by communicating with the Support Element (SE) of each node.

The zEnterprise provides a dedicated system data network. This data network spans all nodes within the Ensemble reaching all servers within each node across the entire ensemble. The security attributes and considerations associated with zEnterprise network communications is the primary focus of this document.

6


Figure 3: System zEnterprise Node

Figure 3 illustrates a zEnterprise node and the various networks associated with the node.

Within the zEnterprise node there are two new “internal networks” which are both private and dedicated to specific communication purposes. The customer managed “external” networks (data and management) are not changed. zCPC access to the customer’s external data network is through an OSA configured as OSD. The access to the new internal networks is controlled by the Unified Resource Manager (zManager).

The zEnterprise provides the following two new internal networks:

1. Intranode management network (INMN) – a 1GbE network used by zEnterprise Unified Resource Management firmware to communicate with the various virtual servers and hypervisors within the node. From a zCPC z this network is accessed by an OSA configured as an OSM CHPID. It is restricted to zManager related functions and therefore can not be used or accessed by customer management applications.

2. Intraensemble data network (IEDN) – a 10 GbE flat layer 2 network that spans across all physical and virtual resources of all of the nodes within the ensemble and is used by customer application workloads fully contained within the ensemble to provide normal network communications for these workloads. From a zCPC this network is accessed by an OSA configured as an OSX CHPID. Layer 3 IP routing is not required to communicate with resources within the ensemble.

7


This security and access control attributes related to the intra-ensemble data network is the underlying topic of this paper.

zEnterprise Security Framework The industry leading system security related features of the zEnterprise System are achieved by providing a security framework that spans multiple tiers of the zEnterprise platform. This multiple layer security model is very similar to previous System z platforms, but it is enhanced with Unified Resource Manager network related functions associated with the IEDN. The following figure provides an overview of the multi-tier security model.

ProfessionalServices

ManagedServices

Hardware& Software

Common Policy, Event Handling and Reporting

The IBM Security FrameworkSecurity Governance, Risk Management

and ComplianceSecurity Governance, Risk Managementand Compliance

People and Identity

Data and Information

Application and Process

Network, Server, and End-point

Physical Infrastructure

Common Policy, Event Handling and Reporting

The IBM Security FrameworkSecurity Governance, Risk Management

and ComplianceSecurity Governance, Risk Managementand Compliance

People and Identity

Data and Information

Application and Process

Network, Server, and End-point

Physical Infrastructure

Figure 4: IBM Security Framework

Figure 4 illustrates the existing IBM security framework and how this framework is preserved and enhanced for zEnterprise. All security functions provided at the “Application and Process” layer and above are not affected by zEnterprise and will still be leveraged and used by customers without change. However, the lower two layers “Physical Infrastructure” and “Network, Server and End-point” are affected and enhanced by the zEnterprise environment. Both layers require closer examination.

8


Physical Infrastructure The physical security provided by the customer’s secure system environment will continue to apply to zEnterprise. This typically consists of aspects such as:

Lock and key areas (i.e. badge access to the physical systems lab and restricted areas)

Locked systems and physical infrastructure (locked covers and access panels)

Employee access control (IDs and passwords) to system administrative functions at the system consoles, HMC and SE

This existing physical security is enhanced by the following new networking hardware features offered in the zEnterprise System:

Dedicated and private networking hardware equipment which reside within the frames of the zEnterprise System and zBX under locked covers that reduces the number of typical physical network hops reducing the scope of security vulnerability

All administration and management interfaces for the new networking hardware equipment are provided exclusively via the Unified Resource Manager (HMC)

New Unified Resource Manager (HMC) administrative roles and passwords for secure access to the HMC for network virtualization configuration settings

New OSA-Express3 OSM and OSX CHPID types for the Intranode Management Network and Intraensemble Data Network which contain new system Unified Resource Manager firmware that provide secure access control to the internal networks (which cannot be defined on the same physical CHPIDs/ports used as OSD connections to access the customer managed external data networks)

An OSX CHPIDs identifies and verifies the physical switch to which it is connected. If the switch is not the expected/supported zBX TOR switch, then an alert is raised, and a Call Home event is generated.

Logical Security The next layer in the framework describes security features related to the network, server and End-point. System z supports many security features related to this layer including:

Network security – Identification, authentication, and encryption using TLS/SSL SSH and IPSec, network isolation and access control using IP Filters, Firewalls, VLANs, and other technologies.

Sserver and end-point security – Operating System and middleware/application identification, authentication and access controls, security managers such as RACF, administrative roles and passwords, Operating System specific security features, zVM security features, I/O configuration (e.g. using IOCDS NOTPART in device candidate list or HCD) security features, logging, and other capabilities.

9


The existing features (listed above) continue to be available and can be deployed within a zEnterprise system. However, zEnterprise also introduces several key advanced network security features for the IEDN in support of the new heterogeneous multi-tier, multiplatform, virtualized workloads that can be deployed in a zEnterprise ensemble.

These new capabilities are provided by the Network Virtualization Management (NVM) component of the Unified Resource Manager and include the following network virtualization and isolation features:

The ability to define multiple distinct virtual networks in the IEDN with platform enforced access controls for all network access points in the IEDN. These virtual networks can be viewed as distinct security zones and exploit VLAN technology for strict isolation of these networks across a common shared physical network fabric (IEDN). The OSA-Express3 OSX CHPID also includes the OSA ISOLATE function (assures virtual server isolation for shared OSA).

The ability to associate virtual servers and optimizers within an ensemble with one or more virtual networks. All virtual servers and optimizers must be explicitly associated with a virtual network in order to access the IEDN.

When combined, these two features provide administrators the ability to deploy workloads with distinct security and isolation requirements into zEnterprise nodes within the ensemble while retaining complete network separation of the servers comprising each workload in the IEDN.

The following figure provides a summary of the previous topics illustrating the network access control system architecture provided by the zEnterprise Unified Resource Manager.

10


11

Figure 5: zEnterprise Network Access Control

As virtual networks are defined and provisioned, the Unified Resource Manager will push all relevant network configuration information to the virtual switch (hypervisor) and physical switches within each node of the ensemble. These virtual and physical switches within the Ensemble will then serve as the access control points for the IEDN. All ensemble network traffic must pass through one or more of these applicable network access control points.

Operating Systems that are to be loaded into virtual servers must coordinate their network VLAN configurations with the Unified Resource Manager (VLAN configuration). If the Operating System attempts to use to a virtual network to which it does not have access to (authorization) it will fail to connect.

The NVM component of the Unified Resource Manager also supports the following additional network security features for the IEDN:

Access controls are provided for the enablement of each external port on the zBX TOR switch. Using the Unified Resource Manager an administrator must enable physical TOR switch ports by defining the specific VLANs (and MAC addresses for access to zBX system resources by servers that are not part of the ensemble) that are to be granted access to the TOR switch (ports).


The Unified Resource Manager controls all dynamic MAC address generation by assigning a MAC address prefix to all hypervisors and virtual switches (Note: OSX is also considered a virtual switch of the PR/SM™ hypervisor). This central configuration approach eliminates MAC address conflicts and unauthorized virtual MAC generation.

IEDN Workloads and Security Zones Groups of related virtual servers can be isolated into smaller networks by defining multiple virtual networks and then restricting virtual servers to specific virtual networks. This isolation can be based on workload, line of business, or other related security criteria that customers define. Virtual servers can be System z Logical Partitions (LPARs), guests virtualized by the various hypervisors within the Ensemble or optimizers. Virtual networks have no physical boundaries within the IEDN; any virtual server within the IEDN can be connected to an IEDN virtual network.

The following figure illustrates how virtual servers can be isolated by deploying multiple virtual networks.

Figure 6: Multiple Virtual Networks

12


Within the IEDN security zones can be viewed as virtual networks. When a new zone is required, the user can define a new virtual network and then grant access to the new network to the appropriate virtual servers. zManager will then orchestrate the virtualization among the server, network, hypervisor and the underlying VLAN technology (within the physical switches) in a manner to provide a complete and secure network solution.

This approach provides significant additional flexibility and isolation controls when compared with traditional deployments of multi-tier multiplatform workloads across physical servers connected via one or more external networks. In this latter scenario, unrelated workloads may be hosted on the same external physical networks and firewalls may need to be deployed to ensure isolation across the various workloads and network boundaries. With the zEnterprise virtual network isolation features, multi-tier, multiplatform workloads hosted within an ensemble can now be associated with a single virtual network to maintain strict isolation from other workloads hosted within the ensemble. When this type of deployment is possible the requirement for traversing a firewall between all server tiers may be reduced or eliminated. Each OS can continue to implement their existing IP filters as necessary. Coupling the IP filters with the virtual network isolation provides a very strong form of access control for secure communications within servers that are part of the same virtual network in the IEDN. Traditional firewall requirements will however continue to exist for certain scenarios, such as traffic entering/leaving the zEnterprise System or scenarios where workloads belonging to different virtual networks need to be able to communicate with one another.

The focus of this paper has been on the network isolation and firewall considerations for the zEnterprise IEDN and a discussion of the new physical and logical network security features being introduced. In addition to these topics, the use of network security protocols like SSL/TLS, IPsec and SSH should also be carefully considered within the intraensemble data network. Some workloads may only require the endpoint or packet-based authentication offered by these protocols while others might still require full encryption of network traffic for privacy or regulatory requirements. Careful analysis of the specific security requirements within the confines of the intraensemble data network could show that more selective use of network encryption is warranted in some cases.

External Network Access The following figure illustrates the two options customers can use to connect their external network traffic to the zEnterprise (i.e. connecting blades to the outside

13


world). Both options involve leveraging the customer’s traditional external Firewall and access controls used to protect their enterprise systems.

Figure 7: External Network Access

In most cases customers will use both forms of external access based on traffic destination, load balancing, or other QoS criteria. The access control from the customer’s external network remains within the customer’s scope of control. Once network traffic is within the Ensemble, then the IEDN virtual networking access control provisions take control.

Exploiting External Firewalls It is recognized that in some environments customers will still be required to force traffic back out of the IEDN to route some network connections through a specific firewall. This can be achieved by using standard network routing within the OS of the virtual servers.

If you want to ensure that all packets that cross VLAN boundaries in the IEDN go through a firewall router, create a static default route whose next hop is the address of the firewall router. If you require all traffic to go through this firewall, this is

14


sufficient. If traffic that is not crossing VLAN boundaries does not have to go through the firewall, you can create a static subnet route to direct that traffic to go directly where it needs to go on the VLAN.

For example if you are attached to a VLAN with subnet, and traffic to destinations within that subnet do not require firewall use but all packets leaving that VLAN do, you create the following static routes:

Outgoing interface Next Hop comment Route destination

0.0.0.0 Your interface attached to the VLAN

The firewall router's IP address on the VLAN you are attached to

Default route. Causes all packets not routed by any other routes to be routed to the firewall

The VLAN’s subnet address

Your interface attached to the VLAN

none Direct route to destinations in the VLAN subnet, will ARP for the destination address and go directly there, bypassing the firewall.

Table 1: Routing Table Overview – Accessing External Firewall

Referencing the Figure 7 (external network access), a server within the zBX could use the option 2 external network path to access the customer’s external firewall and then re-enter the intraensemble data network for connecting to the next tier server. The following figure illustrates this approach.

15


Figure 8: Exploiting an External Firewall

Figure 8 provides an example illustrating the network path of a blade server accessing a System z server via an external firewall. IP addresses are now shown in Table 2 (below) that correspond to the example in figure 8. Some key points to note are:

1. Server 72B (IBM Blade virtual server) has two virtual network interfaces defined as follows:

a. Eth1 – IP Subnet 192.12.144. 0/24 VLAN B - used to access the external network and Firewall via zBX TOR switch – note the red network path

b. Eth2 – IP Subnet 10.24.104.0/24 VLAN C - used to connect directly to servers via the IEDN within the Ensemble (e.g. Linux 55 is also defined to access VLAN C and same IP subnet) – note the direct blue network path

2. Server 22A (z/OS LP) has a virtual network interface subnet 10.67.124.0/24 VLAN A (used to access the external network and Firewall via zBX TOR switch) that is defined to support connections from the blades which must use the external Firewall. This z/OS server would most likely have other virtual NICs (VLANs) defined for direct IEDN access to other virtual servers (both system z and IBM blades) within the Ensemble.

16


Outgoing interface Next Hop comment Route destination0.0.0.0 Your interface attached

to the VLAN B 192.12.144.100 (Eth1)

The firewall router's IP address on the VLAN you are attached to 192.12.144.1

Default route. Causes all packets not routed by any other routes to be routed to the firewall

10.24.104.0/24 Your interface attached to the VLAN C 10.24.104.108 (Eth2)

none Direct route to destinations in the VLAN subnet, will ARP for the destination address and go directly there, bypassing the firewall.

Table 2: Sample Routing Table in Virtual Server 72B (Figure 8)

Summary This paper has provided a basic review of the notion of a network firewall and considerations regarding the requirements for deploying one in a zEnterprise environment. It has also described the internal networking support introduced with the IBM zEnterprise and how, due to its enhanced physical and logical security, in many cases it may eliminate the need for a network firewall to protect network traffic within a zEnterprise environment. Finally, it has described how you can use an external firewall if it is deemed necessary, e.g. for regulatory reasons or due to general mandated corporate policy, to utilize a specific network firewall solution to protect traffic between virtual servers in a zEnterprise environment.

17


Acknowledgments and Contributions This paper was a collaborative effort. Thanks to the following individuals for their contributions to this paper.

Kim Bailey Bill Carey Anna Coffey John Dayka Gwen Dente - IBM S&D Advanced Technical Skills (ATS) Patty Driever Mike Fox Gus Kassimis Gary McAfee Christopher Meyer Linwood Overby

18


About the Authors:

Jerry Stevens is a Senior Technical Staff Member with IBM SWG and works in AIM Enterprise Networking Solutions Architecture Strategy and Design with a focus on communications hardware architecture. He has 25+ years experience with z/OS network communications. Jerry can be reached at [email protected].

Peter Spera is a Senior Software Engineer with IBM Corp. He is focused on security for Linux on the System z platform, but he is also involved with other areas, such as system integrity and vulnerability reporting for System z. Peter can be reached at [email protected].

19

mailto:[email protected]

mailto:[email protected]

IBM zEnterprise System - Network Security

1

Copyright IBM Corporation 2010 IBM Systems and Technology Group Route 100 Somers, New York 10589 U.S.A. Produced in the United States of America, 05/2010 All Rights Reserved IBM, IBM logo, BladeCenter, Proventia, PR/SM, System z, zEnterprise, z/OS and z/VM are trademarks or registered trademarks of the International Business Machines Corporation. Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries. Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom. InfiniBand and InfiniBand Trade Association are registered trademarks of the InfiniBand Trade Association. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office. IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency, which is now part of the Office of Government Commerce. All statements regarding IBM’s future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here.

ZSW03167-USEN-00