A Strategic Approach to SCADA Cyber Security – Water and

Standards

Certification

Education & Training

Publishing

Conferences & Exhibits

A Strategic Approach to SCADA Cyber Security –Water and Wastewater Network Architecture and Segmentation

2013 ISA Water / Wastewater and Automatic Controls SymposiumAugust 6-8, 2013 – Orlando, Florida, USA

Speakers: Bill Phillips and Norman Anderson

2013 ISA WWAC Symposium Aug 6-8, 2013 – Orlando, Florida, USA 2

Presenter

• Norman Anderson, PE : Norman has over 6 years experience in the design and commissioning of Process Control Systems for the Water Sector. Norman has provided secure and reliable PLC, SCADA, and Network hardware and software architecture designs and provided control system automation solutions for a range of facilities. Norman has an M.S. in EE from Iowa State University and an M.S. in Physics from the University of Florida.


Presenter

• Bill Phillips, PE : Bill specializes in delivery of secure and reliable process control and SCADA network and communications systems, cyber security vulnerability assessment, and facility automation and information system planning and implementation. Bill has over 30 years of process control and SCADA system experience and has focused on control system network and communications cyber security for the last decade. Bill has a BSEE from Clemson University.


Presentation Outline

• Securing Networks• The Layered Network Architecture• Network Organization and Segmentation• Configuration• Summary


Importance of Security

Why Security is Important at a Water or Wastewater Facility:

• Critical Infrastructure and Public Safetyo Critical resourceso Downtime can affect life safety

• Operational Reliability and Availabilityo Attacks can lead to significant downtime

•Financial Impactso Loss of revenue for utility and its customerso Mitigation and legal costs

•Media Attentiono Loss of public confidenceo Staff intimidation

2013 ISA WWAC Symposium Aug 6-8, 2013 – Orlando, Florida, USA

Securing Networks

6

• Securing networks requires proper planning to ensure successful implementation. There are four basic stages of planning and implementation for network security:

1. Assessment• Determine Risks and Mitigation techniques• Risk impact versus cost of mitigation

2. Design• Develop appropriate network architecture and segmentation

(NOTE : Tailor to selected HMI suite TCP/UDP port requirements)• Choose necessary hardware and software

3. Implementation• Qualified and certified installers and designers

4. Operation and Maintenance• Develop operational procedures for staff• Maintain network, hardware, and software


Defense-In -Depth

7

Security Risk

Assessment / Design

Implementation

Operational

Security Policies, Procedures, and Maintenance

Training and Experience

ICS Vulnerabilities

Network Configuration

Vulnerability Awareness

Secure Programming

Firewall Rules


Differences Between Corporate IT and Water Sector PCS Networks

8

Paraphrase From NIST SP 800-82 “Guide to Industrial Control Systems (ICS) Security” Table 3.1., Summary of IT System and ICS Differences

Process Control System IT SystemsReal Time Non-Real Time

Mainly used for equipment and processes to function

Mainly used by personnel to create and store data

Response time is critical Consistent response time desired

Generally low bandwidth High bandwidth requirements

Rebooting must be scheduled or avoided Frequent rebooting is acceptable

Human safety and process uptime are paramount

Data confidentiality and integrity is highest importance

System uptime is most critical System and data protection is most critical


A Layered Approach

• A Layered Network is part of the Defense-in-Depth Strategy.

• Divide the network into zones to provide a hierarchy of control for information flow.

• Generally most trusted zone is nested inside the other zones with the least trusted on the exterior.

• Creates a “Peel-the-Onion” environment for attacks.

9


Example Layered Architecture

10


Available Guidance

11

• ANSI/ISA-99.02.01-2009 Security for Industrial Automation and Control Systems: Establishing an Industrial Automation and Control Systems Security Program

o Builds upon global standards ISO/IEC 17799 and ISO/IEC 27001 and addresses the difference needed for industrial security

o Defines procedures for implementing and assessing secure industrial control systems

• Cisco/Rockwell Automation – Converged PlantwideEthernet (CPwE) Design and Implementation Guide

oDesign and implementation guidelines for industrial control systems

oGuide provides real network architecture examples and security and implementation methods


Similarities to other Guides and Standards

12

Cisco/Rockwell Automation – CPwEDesign and Implementation Guide ANSI/ISA-99.00.01-2007


Least Trusted Layer

13

• Business networks and large networks such as the Internet or Metropolitan Area Networks (MANs).

• Use to route between trusted networks using encrypted VPNs.

• Allowed access to read-only applications for SCADA view-only and reporting applications.

• Used for maintenance access by package system vendors but not direct to PLCs.

• No direct access to the process control network from this layer.

• Used for access to other services such as software updates and NTP for time synchronization.

• Not a required layered. Only used when necessary to help operations and provide better service. If possible, external access should be avoided.


DMZ Layer

14

• Location for equipment that accesses the Process Control Network and Outside networks.

• Domain controllers in this layer should be read-only (slaves) from the Process SCADA network.

• Equipment located in this layer can access the outside network for alarming, reporting, and updating services but cannot write to the internal network without manual initiation from the Process SCADA network.


Process SCADA Network Layer

15

• SCADA system location with no direct access to the outside untrusted networks.

• Maintenance access can be provided by hopping through the DMZ.

• SCADA servers can directly access the Process control PLC network.

• Terminal services used for SCADA clients, or similar, can access SCADA servers and Operator workstations but not the PLC network.

• Control should only be allowed from this layer and the PLC network.


PCS PLC Network Layer

16

• Innermost layer requiring the most “hoops” to jump through for access from the outside.

• This layer is still segmented on separate networks to minimize broadcast domains and separate dissimilar traffic to allow for implementation of QoS rules.

• Devices on the same network can communicate in the absence of the firewall or a router to allow the control system to continue operation if the network “head end” devices were to fail.

• The PCS Firewall is shown but not required and is mainly used for routing between the SCADA and PCS PLC networks but may be needed for other functions.


Network Organization

• Start by defining networks using the logical Class.• Generally . . .

– Class A is used for the internal networks having the largest number of devices, e.g., the PLC network

– Class B is used for communications between private networks, e.g., between treatment plants on a Metro-Ethernet network.

– Class C for public networks, e.g., Webserver or Firewall connected to the Internet and City-Wide network.

• Networks should be selected where they make sense, but should follow industry standards such as RFC 1918.

17

RFC 1918 Name Address Range Network Class

24-bit block 10.0.0.0 – 10.255.255.255 Class A (10/8 prefix)

20-bit block 172.16.0.0 – 172.31.255.255 Class B (172.16/12 prefix)

16-bit block 192.168.0.0 – 192.168.255.255 Class C (192.168/16 prefix)


VLAN Approach

18

• VLANs accompany subnetworks selected and a 1:1:1 relationship should be maintained between VLANs, Subnets, and Broadcast Domains.

• Virtual LANs (VLANs) - Useful for SCADA systems because VLANs define broadcast domains that can be widely dispersed (i.e. not on the same network segment)

• Can reduce costs, by allowing host on different networks to share layer 2 switches.

• Use 802.1q VLAN encapsulation protocol• Layer 3 device required to route between VLANs.• Layer 2 devices support VLANs and VLAN Trunking.• VLAN Approach:

o Use VLANs in the range of 2-1001, various restrictions apply to other VLANso Don’t Use VLAN 1 (Native or Default VLAN)o Verify VLAN capabilities of network switches & routerso Use logical approacho Incorporate VLAN designations into IP Addresses


IP Addressing Example

19

• Y=0 is the network and Y=255 is the broadcast address

• Subnet mask can be 9-30 bits.

1st /29 subnet: network address 10.10.52.0, host range 1-6.

2nd /29 subnet: network address 10.10.52.8, host range 9-14


VLAN Example

20

•VLANs should be selected in a logical order, recommend using Trust Level. In the example below, VLANs are numbered inversely to Trust Level numbers.

•Aids in network organization and identification of networks, locations, and components.

•Reduces broadcast domains to reduce network traffic and unnecessary requests to components.

•Increases network security.


Providing a coordinated system

21

• Approach:• Incorporate facility & VLAN numbers into IP addresses• Limit broadcast domains to a single facility and to a 254 host max.

• Primary VLAN Example:• 10.VLAN.Facility.Host/X or 10.Facility.VLAN.Host/X• X = Subnet Mask bit count• X (Generally between 24 &30) based on anticipated host count

• WAN Example:• 192.168.1.Y/X• X = Subnet Mask bit count• (Generally between 24 & 30) based on number of nodes• Y = Host Number and depends on Subnet Mask


Example Network Configuration

22

SF

P

Gi1

/0/2

1

Gi1

/0/2

2

Gi1

/0/2

3

E0

/0

E0

/1

E0

/2

Gi2

/0/2

2

E0

/2

E0

/1

E0

/0

Gi2

/0/2

3

Gi2

/0/2

1

Gi2

/0/2

4

40-E

NS

-1

40-E

NS

-2

SF

P

Note: Use separate physical media or routers to separate VLANs that have public access to prevent VLAN attacks such as ARP poisoning.


Example Remote Connections

23

Cellular

3G

Cellular Wireless

Network

3G Wireless Digital

Cellular Modem

15Mbps/3M

bps 15Mbp

s/3M

bps

Site-to-Site VPN

Site-to-Site VPN

User VPN

User VPN (Disaster Recovery)

Internet

Remote Internet

Connected Workstations

User VPNUser VPN

(Disaster Recovery)

40-FWL-1,2

Firewalls

50-FWL-1,2

Firewalls

NOTES:

1. USER VPN CONNECTIONS ARE VPN

CONNECTIONS INITIATED BY

REMOTE DEVICES.

2. SITE-TO-SITE VPN CONNECTIONS

ARE VPN CONNECTIONS INITIATED

BY THE HOST (POLLING PLC).

3. DISASTER RECOVERY

CONNECTIONS ARE USED WHEN

PRIMARY CONNECTIONS FAIL OR

ARE LOST.

Disaster Recovery Pathways

LINE LEGEND:

Primary Connection Pathways

Service Provider Connection Pump Station

PLC

Remote Control RoomCentral Control Room

Mobile Cellular

Wireless

Workstations


VLAN Assignments and Rules

• Local network in example is broken into multiple VLANs.• VLANs incorporated into IP Addresses along with facility and Host

numbers.• Provides an organized network allowing for internal staff to easily identify

devices and networks.• Multiple VLANs can reside within Layer 2.• Routing accomplished by the Firewall using extended ACLs.

24


IP Addressing Table

25

CENTRAL CONTROL ROOM

SCADA (VLAN10) DEVICE LINKSTATE (VLAN11) DEVICE MUNICIPAL WAN (VLAN800) DEVICE WEBSERVER(VLAN30) DEVICE

10.40.10.0/24 10.40.11.0/29 192.168.1.0/29 10.40.30.0/24

1 ENS int Vlan 10 (gateway) 1 FW (virtual, gateway) 1 City gateway 1 FW (virtual, gateway)

2 FW (virtual) 40-FWL-1,2 2 FWa 40-FWL-1 2 FW (virtual, gateway) 2 FWa 40-FWL-1

3 FWa 40-FWL-1 3 FWb 40-FWL-2 3 FWa 40-FWL-1 3 FWb 40-FWL-2

4 FWb 40-FWL-2 4 Primary SCADA (LinkState) 40-SVR-3-2 4 FWb 40-FWL-2 11 Primary RODC 40-SVR-1-1

5 40-UPS-1 5 5 12 Secondary RODC 40-SVR-1-2

6 40-UPS-2 6 6 13 WebServer 40-SVR-1-3

7 7 BROADCAST 7 BROADCAST *

8 MetroEthernet (Vlan801) Device *

9 172.16.0.0/29 *

10 1 Gateway 255 BROADCAST

11 Primary DC/DNS Server 40-SVR-3-1 PLC (VLAN20) DEVICE 2 FW (virtual, gateway) BUSINESS (VLAN40) DEVICE

12 Primary SCADA Server 40-SVR-3-2 10.40.20.0/24 3 FWa 40-FWL-1 10.40.40.0/24

13 Historian Server 40-SVR-3-3 1 FW (virtual, gateway) 4 FWb 40-FWL-2 1 FW (virtual, gateway)

14 SCADA Terminal Server 40-SVR-3-4 2 FWa 40-FWL-1 5 2 FWa 40-FWL-1

15 SCADA NAS 40-SVR-3-5 3 FWb 40-FWL-2 6 3 FWb 40-FWL-2

16 Alarm Server 40-SVR-3-6 4 7 BROADCAST 11 Primary DC 40-SVR-2-1

17 Monitoring Server 40-SVR-3-7 5 PUBLIC - (V900) DEVICE 12 Primary MS Exchange Server 40-SVR-2-2

18 6 111.111.111.0/29 13 Business Terminal Server 40-SVR-2-3

19 7 1 ISP (Gateway) 14 Business NAS 40-SVR-2-4

20 8 2 FW (virtual, Port Address Translation) 17 Printer #1 40-PRT-1

21 SCADA Full Client 40-WKS-3-1 9 3 FWa 40-FWL-1 18 Printer #2 40-PRT-2

22 SCADA T.S. Client 40-WKS-3-2 10 4 FWb 40-FWL-2 21 Business Client 40-WKS-2-1

23 SCADA T.S. Client 40-WKS-3-3 11 Master PLC#1 (Internet) 40-PLC-1 5 Spare for Future Use 22 Business Client 40-WKS-2-2

24 Reserved for future workstation 12 Master PLC#1 (Server) 40-PLC-1 6 Primary MS Exchange Server 40-SVR-2-2 23 Business Client 40-WKS-2-3

* 13 Master PLC#2 (Internet) 40-PLC-2 7 BROADCAST *

* 14 Master PLC#2 (Server) 40-PLC-2 *

* * Spare for Future PLC Equipment *

255 BROADCAST 255 BROADCAST 255 BROADCAST

Spare for Future Use

Spare for Future Use

Spare for Future Equipment

Spare for Future Equipment

Do not use

Reserved for Future Network Equipment

Spare for Future SCADA Equipment

Reserved for Future ServersReserved for Future Network

Equipment


Configuration and Management

• Configuration and management are simpler.• Network expansion is simpler.

– Subnets are already set with IP Addresses reserved or easy to determine.

– The appropriate routes between devices are already configured via subnet and VLANs.

• Router and Firewall rules are simplified using subnets and VLANs instead of individual addresses.

• Management is simpler since addresses are easily identified with equipment, facility, and VLAN assignments. Identifying an intruder is also more obvious.

26


Firewall Trust Level Assignments

27

• Security Levels - Implicit Deny Lower-to-Higher level:

• Each Interface & Sub-interface• Inside – 100 (Most trusted)• Outside – 0 (Least trusted)• DMZ – 50

• Interfaces• Typically 3-4 separate physical ports on Firewall for small to medium size firewalls.

Allows separation of business and control networks. Sub-interfaces allow a single firewall port to be shared by a number of VLAN subnets.

• Network organization allows for logical assignment of Trust Level with VLANs and Subnets.

• Use Firewalls with Stateful Inspection• Can drop otherwise legitimate packets that are not part of an active connection• Holds in memory variables defining the state of each connection• State variables include things like source and destination addresses, port numbers,

packet sequence numbers


Firewall Rules – Access Control Lists (ACLs)

28

• Access Control Lists• Used to apply access control rules at interfaces• Permit DMZ –to-Inside SCADA specific traffic such as web server, terminal

server and historian traffic.• Permit VPN LAN-to-DMZ authenticated remote user traffic such as web server,

terminal server and historian traffic.• Remote PLC Connections:

• Consider a Remote PLC DMZ to avoid direct connections between Internet connected PLCs and the SCADA network

• Consider dual Ethernet DMZ PLC interfaces (i.e. separate VLANs) to increase separation.


Example Firewall Configuration

29

• Define addresses for system components:• set address "Trust" "172.28.0.0/24" 172.28.0.0 255.255.255.0• set address "DMZ" "Historian_Svr" 172.28.1.12 255.255.255.255 “HMI-SCADAHIS in DMZ“

Addresses for the SCADA network 172.28.0.1 through 172.28.0.254 and the Historian server 172.28.1.12 have been set and assigned to the Trust and DMZ trust levels.

• Set Rule for allowed communication:• set policy id 16 from "DMZ" to "Trust" "Historian_Svr" "172.28.0.0/24" "_RDP_TCP" permit log count

Policy allows service “_RDP_TCP” from the Historian in the DMZ to the SCADA network in the Trust level.

• Define the policy:•set service "_RDP_TCP" protocol tcp src-port 0-65535 dst-port 3389-3389Policy defines the allowed ports for communication. All other ports are denied.

Using an organized and logical network organization allows for simpler and logical configuration.


Summary

• Network security is an important aspect of any Water Sector Process Control System.

• Multi-layered network organization provides a foundation for building a secure Process Control Network.

• Using logical subnet and VLAN selections provides a usable segmentation framework that allows for easily identifiable components, eases expansion, and makes network configuration and management simpler.

• A layered network provides additional protection from attacks and allows more time to identify an intruder.

• VLANs minimize broadcast domains, reduce bandwidth requirements and increase network response and security

30

Documents

A Strategic Approach to SCADA Cyber Security – Water and