SAER-6114

Previous Issue: New Page 1 of 63 Primary contact Almadi, Soloman Musa on 966-3- 8739939

Copyright©Saudi Aramco 2007. All rights reserved.

Engineering Report

SAER-6114 22 May 2007 Process Automation (I-Field/SCADA/Remote Sites) Communication Network Architecture Document Responsibility: Process & Control Systems Department

Document Responsibility: Process & Control Systems Dept. SAER-6114 Process Automation (I-Field/SCADA/Remote Sites) Issue Date: 22 May 2007 Communication Network Architecture

Page 2 of 63

Table of Contents

1 Executive Summary.................................................................................................... 4 Part I: Wireline Network Architecture 1 Introduction................................................................................................................. 7 2 Background................................................................................................................. 7 3 Existing SCADA Network Design............................................................................. 10

3.1 SCADA Over Dedicated Transmission.......................................................... 10 3.2 SCADA Over Dedicated SDH Network…….................................................. 12 3.3 SCADA Over IT WAN Packet Network.......................................................... 15 3.4 SCADA Over Radio Network……………………………………...…............... 17

4 Industry Trends......................................................................................................... 17 4.1 Purdue Enterprise Reference Architecture (PERA)....................................... 17 4.2 Instrumentation, Systems and Automation Society....................................... 18 4.3 Industry Trends Summary............................................................................. 19

5 Future I-Field/SCADA/Remotes Communication Network Architecture................... 21 5.1 Existing Performance & Reliability Issues.................................................... 21 5.2 Analysis & Results......................................................................................... 22 5.3 Remote Process Automation Application Network Architecture.................... 24

5.3.1 Future Process Automation (SCADA, I-Field, Remote sites) Network Design Architecture……….................................................. 25

5.3.2 Next Steps......................................................................................... 32

Part II: Process Automation Wireless Access

1 Background............................................................................................................... 33 2 Introduction................................................................................................................ 33 3 Wireless for Process Networks................................................................................. 33 4 Industrial Wireless vs. Non-Industrial Wireless......................................................... 34 5 Industrial Wireless Classifications............................................................................. 35

5.1 Wireless Personal Area Networks (WPAN)................................................... 36 5.2 Wireless Local Area Networks (WLAN)......................................................... 38 5.3 Broadband Wireless Connectivity.................................................................. 42 5.4 Wireless Wide Area Networks (WWAN)........................................................ 43


Page 3 of 63

Table of Contents (Cont'd)

6 Wireless for Process Network Conclusion................................................................ 45 7 Next Steps................................................................................................................ 45

Part III: Network Timing & Synchronization Architecture

1 Synchronization Architecture..................................................................................... 46 2 Next Steps................................................................................................................ 48

APPENDICES.................................................................................................................... 49

Appendix 1: Design Performance Criteria Analysis................................................. 50 Appendix 2: Bandwidth Calculation for Wireline Network........................................ 53 Appendix 3: Wireless Technology Comparison........................................................ 56 Appendix 4: Wireless Technology Classifications.................................................... 57 Appendix 5: ISM Frequency Band............................................................................ 58 Appendix 6: IEEE 802.11 WLAN Standards Model.................................................. 59 Appendix 7: Support Structure………………………………………………………..… 60

GLOSSARY....................................................................................................................... 61


Page 4 of 63

1 Executive Summary

This report summarizes the findings of a detailed assessment of existing Saudi Aramco implementations, industry trends, and recommends network design architecture for data acquisition, control, and monitoring for remote Process Automation applications; which includes I-Field and SCADA. This report also covers other non-real time process control applications such as Cathodic monitoring, Vibration monitoring, etc. The network infrastructure covered in this document, includes Wireline Networking, Wireless Networking, and Network Synchronization.

Recommendations outlined in this document will be mapped to the different Process Automation standards and procedures (e.g., SAES-Z-010, SAES-Z-001, 23-SAMS-020, draft SAES-Z-004, etc...). In the interim, this report will serve as a guide for any future SCADA, I-Field, and remote site network connectivity until all related standards are updated with these recommendations. These recommendations are based on benchmarking of international organization standards, industry trends, existing Saudi Aramco different implementations, and lessons learned. The outcome of this effort shall provide higher level of end-to-end network reliability, flexibility, and optimal design configurations.

In summary, the recommendations call for the following:

I. Wireline Network Architecture: The remote Process Automation applications network architecture shall be based on either one of the following configurations:

1. Industrial Ethernet using dedicated fiber in a self healing ring topology for grass root projects such as Khurias, Manifa, etc.

2. For existing implementations, utilize Synchronous Digital Hierarchy (SDH) dedicated bandwidth based on self healing ring architecture.

These two network toloplogies implemenation shall be based on the following general guidelines:

• A standalone fiber optics cable shall be utilized to connect remote sites (i.e., well sites) to a centralized Process Automation Network (PAN). Alternatively, Radio links, or Broadband wireless network solution could be used to connect dispersed remote sites where fiber optics cable installation proves economically not feasible.

• Self healing ring topology is a ring network topology based on a uni/bidirectional links between a set of stations providing optimum protection against fiber cable cut and equipment failure.


Page 5 of 63

• Utilize point-to-point ring system for scattered sites and/or where a true closed loop physical ring topology is economically not justifiable.

• A PAN comprising of multiple scattered (PANs), shall interface with the Corporate Network via a centralized firewall(s). These scattered PANs shall be connected together to form a unified PAN utilizing the corporate SDH transmission infrastructure with a dedicated bandwidth of 10 Mbps. Alternatively, dark fiber can be utilized if available.

• SCADA over IP/ATM is a unique implementation for East/West Pipelines. It was recommended to PMT to move SCADA traffic from ATM and have it directly transported over a dedicated SDH transport network to fulfill the physical separation requirements as per SAES-Z-010. This would be part of the ongoing BI (BI-10-00179) that will replace the existing Microwave system to SDH.

Any future implementation similar to the East/West Pipelines such as Water Injection Plants, scattered GOSPs will be based on either one of the two design options stated above. This is mainly because remote GOSPs or WIPs operation is tightly coupled with centralized process control systems.

Detailed network analysis and recommended guidelines for the Wireline Network Architecture are included in Part I, section 5.3 of this report.

II. Wireless Network Architecture: Wireless network solution provides cost effective and efficient connectivity for different Process Automation systems. The following are the recommended wireless connectivity solution mapped to their associated applications.

1. Zigbee (in conjunction with IEEE 802.15.4): wireless solution considered as a potential emerging short distance wireless technology for instrumentations networking, subsystems networking of network latency tolerant applications.

2. Wireless LAN (based on standards IEEE 802.11a, b, and g technologies): considered a potential mature and proven wireless technologies for mid range (100’s meters up to few kilometers) applications.

3. Wireless Wide Areas Network (e.g., GSM phone based service such as Short Messaging Service, General Packet Radio Systems): can be utilized to provide efficient and cost effective process monitoring solution for semi-real time non-critical applications.

4. Broadband Fixed Wireless connectivity (WiMAX or Propriety solution)

• Provides coverage up to 50km+ radius and data rates up to 75+ Mbps


Page 6 of 63

• Provides a cost effective and flexible point-to-point connection

• WiMax requires frequency approval from local authority Communications and Information Technology Commission (CITC) and should be requested through a 3rd party local service provider

Hence, the strategy is to use propriety Broadband Fixed Wireless Solution to avoid licensing cost and dependence on 3P

rdP party service providers.

Detailed network analysis, benchmarking, and recommended next steps for Wireless Network Architecture for Process Automation are included in Part II of this report.

III. Network Synchronizations: Network synchronizations will be based on Global Positioning System (GPS) coupled with Network Timing Protocol (NTP) server within a plant to provide precise time stamping for systems and network infrastructure. To minimize cost, existing plant’s GPS/NTP servers will be extended to its remote sites.

Detailed analysis and recommended next steps for the Network Synchronizations are included in Part III of this report.

P&CSD will utilize and benchmark ongoing technology trials in both Wireless and Wireline networks to develop Saudi Aramco engineering deployment guidelines such as material specifications, standards, and best practices.

As part of the overall recommendation, future Capital Projects (i.e., Budget Items) for Process Automation networks will mainly be referenced under the Z standards index (Process Control). Telecommunication T-index will be sub referenced for overlap areas.


Page 7 of 63

Part I: Wireline Network Architecture

1 Introduction

This part of the report provides recommended network architecture for SCADA systems, I-Field, and other remote site Process Automation applications. The recommended architecture is based on a comprehensive network evaluation for existing Process Automation applications that include SCADA systems, ongoing I-Field initiative, and other remote sites Process Automation requirements.

This evaluation includes network topologies, protocols, reliability and recommended network configuration options. Selected Saudi Aramco SCADA and I-Field implementations have been used as a baseline in this assessment. Further, industry trends and international organization bodies (e.g., ISA, API, etc.) have been used to provide an optimal recommendation that would meet existing and future Process Automation applications requirements.

2 Background

In April 2006, an assessment to existing Saudi Aramco Standards and Material specifications revealed the need for comprehensive detailed requirements for Process Automation Network (PAN) to address SCADA and other remote site Process Automation applications’ communication network. In addition, different network implementations types in Saudi Aramco have emerged over time based on different site topologies and the available technologies at that time. These can be described as follows:

1. Dedicated Transmission network was used for some of these implementations; while others were based on transmitting Process Automation traffic utilizing shared transmission infrastructure.

2. These different network implementation either using self healing rings topology, or point to point redundant links, or single threaded links.

3. Propriety protocols were used as a defacto of the applications. However, there are some implementations that are based on open standard communication protocols.

These different configurations have caused a non streamlined network design implementations and created a gap in Process Automation Standards. Here is a summary of what the existing standards, as of April 2006, are calling for a long with high level analysis of these standard requirements.


Page 8 of 63

● SAES-Z-010:

○ Scope: This standard establishes the requirements for design, installation, configuration and commissioning of Process Automation Networks (PANs).

“Geographically spread Remote Terminal Units (RTU's) are not in the scope of this standard”

○ This standard calls for:

5.2 “The network design shall provide physical and logical separation between PAN and Corporate Network below firewall. Logical separation, at minimum, is mandatory for network connections above firewall”

○ Issue: The standards scope does not address remote areas (RTU’s) connectivity.

● SAES-J-902:

○ Scope: This Standard establishes the design criteria for the installation of power and wiring systems for electrical instrumentation.

○ Standard calls for:

12.3.2 “Data Links: Data links, including fiber optic cables, shall be specified and installed per system manufacturers' recommendations. When redundant data links are provided, the primary cable shall follow a different route from the back up cable. Primary and backup data link cables shall preferentially enter cabinets or consoles from opposite sides. Data link cables shall not be routed in the same conduit, duct, or tray with other instrument cables.”

○ Issue: The scope does not address SCADA or other remote sites’ Process Automation application related traffic. It is focused on the instrumentation.

● SAES-Z-001:

○ Scope: This Standard prescribes the minimum mandatory requirements and guidelines governing the engineering, design and installation of Process Control Systems (PCS) in Saudi Aramco plants (i.e., Oil/Gas Plants, Refineries, and Pump Stations).


Page 9 of 63


14.3 “Interface to Corporate Wide Area Networks (CWAN): The control system communication to Corporate Wide Area Network and other non-control computer systems shall be designed to ensure that no failure, no request for information, or network loading problem will impact the performance or availability of the PCS. Use of standard software and hardware protocols for interfaces, such as TCP/IP and OPC, are preferred.”

○ Issue: The standards calls for generic requirements (e.g., no failure, no loading problem will impact the performance, etc…). The standards does not address communication medium (e.g., physical separation vs. logical separation).

● 23-SAMSS-010:

○ Scope: all DCS equipment and associated software required to monitor and control a process plant.


11.2 “Communication at the control network level shall have redundant or fault tolerant paths. Communications from the controller to the I/O subsystem shall have redundant paths.”

11.3 “DCS internal communication shall be designed such that no single failure will degrade the performance of the system. This requirement applies to all communication between DCS modules, including communication between controllers to their I/O modules.”

○ Issue: The standards scope does not address SCADA or remote areas (RTU’s). Further, the standard calls for fault tolerant network but does not address the communication in detail.


Page 10 of 63

● 23-SAMSS-020:

○ Scope: This specification defines the minimum mandatory requirements for Supervisory Control and Data Acquisition (SCADA) systems for oil & gas and utility applications.

○ This SAMSS calls for:

13.1.4 “The system LAN shall be fault tolerant utilizing a network configuration that prevents a single point of failure.”

○ Issue: The standard calls for fault tolerant network but does not address the communication in detail.

3 Existing Remote Network Connectivity Design

There are three (3) different existing remote network connectivity designs in Saudi Aramco. These network design topologies have evolved over the past many years. These topologies are:

1. The earliest is SCADA network based on dedicated point-to-point digital circuits over digital transmission network and then evolved to be over Synchronous Digital Hierarchy (SDH) transmission network. OSPAS and PDD are the primary users for this network topology at this time.

2. SCADA using dedicated bandwidth on shared, transmission systems such as Synchronous Digital Hierarchy (SDH). This is typically implemented within oil and gas fields (e.g., Qatif, Haradh, etc.).

3. The third network design option is based on carrying SCADA traffic over IP then over ATM and then over SDH. This is limited to East /West Pipelines network design implementation and Shaybah’s inter GOSPs connectivity.

Detailed design overview for these different existing options is outlined in the subsequent sections.

3.1 SCADA Over Dedicated Transmission

The earliest SCADA network implementation was based on dedicated point-to-point digital circuits over Digital transmission transport network or Synchronous Digital Hierarchy (SDH) networks. This implementation is based on using dedicated point-to-point digital circuits with modems at both ends of the circuit or a radio link. The circuit provides connection between the Remote Terminal Unit (RTU) at the remote site to the SCADA Terminal Sever at the control room. Figure 4.1 depicts the network topology for this access method.


Page 11 of 63

Figure 3.1: Dedicated Point-to-Point Circuits

Typically, a RTU is either connected by a digital circuit (2 wire or 4 Wire E/M) or a radio link from remote sites to the nearest transport network node Digital Access Cross Connect System that is part of the Saudi Aramco SDH network. The majority of the dedicated lines operate at low speed, i.e., 1200 to 9600 bauds. Further, many of these dedicated lines utilize twisted telephone line pairs party line features; which can have up to six (6) RTUs connected.

Remote sites supported by leased lines circuits from Saudi Telecom Company (STC) are connected to the nearest Saudi Aramco SDH point of presence to STC (i.e., Jizan Bulk Plant, SSSP, etc...). The circuit is then backhauled to the SCADA terminal server at the control room.

The implementation of this design configuration is widely used in support of Oil Supply Planning & Scheduling Department (OSPAS), Power Distribution Department (PDD), and some of the pipelines sites.

OSPAS Center

5 0 0 P

P S 1

P S 2

O m n i S w i t c h

1 2 3 4 5 6 7 8 9

VACA

NT SL

OT

VACAN

T SLO

T

VACA

NT SL

OT

VACA

NT SL

OT

MPX

OMNI

SWITC

H 9WX

2 POR

TS OC

-3 SMF

-AT

M A

SX-155

RFS

1 2

32 UT

P 10/1

00 Mb

ps -E

therne

t

ESX

32 UT

P 10/1

00 Mb

ps -E

therne

t

ESX

Fiber Optics MUX/Modems connecting digital

Point to Point Circuits

UHF Radio

Fiber Optics Mux

RTU

5 0 0 P

P S 1

P S 2

O m n i S w i tc h

1 2 3 4 5 6 7 8 9

VACA

NT SL

OT

VACA

NT S

LOT

VACA

NT SL

OT

VACA

NT SL

OT

MPX

OMNI

SWITC

H 9W

X

2 POR

TS O

C-3 S

MF -

ATM

AS

X-15

5 RFS

1 2

32 U

TP 10

/100 M

bps

-Ethe

rnet

ESX

32 U

TP 10

/100 M

bps

-Ethe

rnet

ESX

5 0 0 P

P S 1

P S 2

O m n i S w i tc h

1 2 3 4 5 6 7 8 9

VACA

NT SL

OT

VACA

NT S

LOT

VACA

NT SL

OT

VACA

NT SL

OT

MPX

OMNI

SWITC

H 9W

X

2 POR

TS O

C-3 S

MF -

ATM

AS

X-15

5 RFS

1 2

32 U

TP 10

/100 M

bps

-Ethe

rnet

ESX

32 U

TP 10

/100 M

bps

-Ethe

rnet

ESX

Terminal Server

RTU

IT SDH Transport Network

Fiber Optics MUX/Modems connecting digital Point to Point Circuits

RTU

Fast Ethernet

Ethernet Switch


Page 12 of 63

OSPAS alone has over 400 point-to-point circuits companywide covering key oil and electric distribution systems offshore and onshore utilizing IT SDH transport network. Some of these connections are backhauled to Dhahran Core areas providing a centralized SCADA management system. The primary user of this architecture is OSPAS, where they monitor and control oil and gas flow from different wells and GOSPs. Also, electric power flow from SCECO to Saudi Aramco substations and distribution lines within Saudi Aramco facilities are managed by PDD through this design configuration.

Others are based on a distributed SCADA architecture where the remote sites are connected to a local control room site within the same serving area. An example of this is offshore to onshore SCADA connections.

3.2 SCADA Over Dedicated SDH Network

It must be noted in this document, the term Optical Transport Network (OTN) is used interchangeably with SDH as OTN is a vendor product solution deployed in several Saudi Aramco plants.

There are two approaches for using Synchronous Digital Hierarchy (SDH) transport network in carrying SCADA traffic. The first is based on circuit emulation for a point- to-point serial connection using the inherent multiplexing hierarchy of SDH systems. This approach is a proven concept that has been used in the industry for real time and non real time applications.

Figure 3.2.1 depicts the network topology for this design configuration. Ring network topology is the normal approach. For large fields with many scattered oil wells, multiple rings are implemented to provide robust network; while serving dispersed Remote Terminal Units.


Page 13 of 63

Figure 3.2.1: Dedicated Bandwidth over SDH Transmission Network

The configuration is typically deployed in a self healing SDH ring with a recovery time of close to 50 milliseconds. This is a powerful capability as long as the SCADA application’s communication time delay is compatible with distance between the RTU and the SCADA Master Sever on the two sides of the ring.

Oil & Gas Fields/Remote GOSP

Ethernet Switch

CCR

SCADA PAN Backbone Transmission Ring using SDH nodes

Operator Control Console

Fast Ethernet

ENG & OPR Console

Ethernet Switch

5 0 0 P

P S 1

P S 2

O m n i S w i tc h

1 2 3 4 5 6 7 8 9

VAC

ANT

SL

OT

VA

CAN

T SL

OT

VAC

ANT

SL

OT

VAC

ANT

SL

OT

MP

X

OM

NI S

WIT

CH 9

WX

2 PO

RTS

OC

-3 S

MF -

ATM

A

SX-1

55 R

FS

1

2

32

UTP

10/1

00

Mbp

s -

Eth

ern

et

ESX

32

UTP

10/1

00

Mbp

s -

Eth

ern

et

ESX

SCADA Server

WHESD/ESP/PDHM, etc..

PAN Access Transmission Ring Using SDH Nodes

5 0 0 P

P S 1

P S 2

O m n i S w i t c h

1 2 3 4 5 6 7 8 9

VACA

NT SL

OT

VAC

ANT S

LOT

VACA

NT SL

OT

VACA

NT SL

OT

MPX

OMN

I SW

ITCH 9

WX

2 PO

RTS O

C-3 S

MF -

ATM

AS

X-15

5 RF

S

1

2

32 U

TP 10

/100 M

bps

-Eth

ernet

ESX

32 U

TP 10

/100 M

bps

-Eth

ernet

ESX

Radio

Firewall Plant Network

RTU

Enterprise Office User

Work Station

Enterprise IT Wide Area Network

Office User

Enterprise PI Server (s)

GOSP: Eng & OPR Console/ Data Historian


Page 14 of 63

The second Synchronous Digital Hierarchy (SDH) transmission implementation approach is based on establishing a dedicated bandwidth on the SDH system, providing logical connection on shared SDH transport system. This eliminates the need for serial point to point connection emulation on the SDH and provides higher flexibility in allocating dedicated bandwidth based on actual bandwidth requirements. This option requires the introduction of Ethernet switches at the Remote Terminal Unit (RTU) site.

Figure 3.2.2 depicts a conceptual network design using SDH ring based on dedicated bandwidth allocating.

Figure 3.2.2: End-to-End Network Topology for a Typical Oil Well Work Site

ESP MOV

RTU=409Kbps/ 8M Buffer

Ethernet Access Switch

Ethernet Aggregator Switch

SDH

SDH

Ethernet Switch

Server

PDHM

CMS=2.4Kbps/0

CP=2.4Kbps/ 2M Buffer

MPFM=38.4kbps/ 4M Buffer

SWC= 19.4Kbps/Buffer = 4 Mbps

Phone Line=64kbps/ 0 Buffer

Load= Number of Oil Well Work Areas Interface Capacity Max= 10Mbps to 1 GE Single Mode Fiber Distant 2KM to >10 KM

Transmission Network SDH or OTN Ring

Effective Bandwidth Capacity = Dedicated Bandwidth

Capacity Provisioned on the SDH Ring Ranges from 0 to 100 Mbps

Oil Well Work

SDH

SDH

CCR Room

SDH Access Node / Ethernet Aggregator


Page 15 of 63

Access traffic, SCADA and others, is aggregated to each serving SDH transport node and then logically mapped to the SDH node hosting the SCADA servers at the control room. The term OTN (Optical Transport Network) is used interchangeably with SDH as OTN is a vendor product solution for SDH deployed in several Saudi Aramco plants. There are over 600 OTN nodes serving well heads, RTU’s and others in different Saudi Aramco facilities (e.g., Qatif, Haradh, etc.).

In this example, the peak load for each RTU is 516Kbps. This is the composite traffic resident at the oil well work area automation systems: RTU, MPFM, SWC, CMS, CP, Voice, etc. All of these services are connected to an Ethernet switch; an Access switch. Each access switch with a Giga Ethernet interface is connected to another Ethernet switch at the SDH node; an Aggregator switch. The Aggregator switch is then connected to the SDH network using Fast Ethernet connection.

A dedicated bandwidth channel is provisioned from the SDH node serving the Aggregator switch to the SDH node located at the CCR room. Therefore, the effective bandwidth allocation from the Aggregator switch to the CCR room is the dedicated bandwidth provisioned between the SDH node severing multiple oil well working areas and the SDH node at the CCR room. Careful considerations shall be taken when designing such a network from a bandwidth and redundancy perspectives. Sections 5.3 will outline engineering guidelines when design this type of network topology.

3.3 SCADA Over IT WAN Packet Network

The IT Wide Area Network (WAN) uses Asynchronous Transport Mode (ATM) cell switching technology as a backbone for Enterprise applications. There are some existing scenarios where the ATM backbone is used to support Process Automation application (i.e., SCADA) for linking remote sites to a central site. This is design was implemented for East/West Pipelines and Shaybah’s inter-GOSPs connectivity.

The SCADA traffic is interfaced via IP over Ethernet to IP over ATM WAN node in the local site (e.g., Pump station). The traffic is then routed to the Central SCADA server at the Control Room (CR) (e.g., Yanbu Crude Oil Terminal –YCOT). The process of carting the SCADA from the local site to the central site is supported by the IT ATM network based on best effort utilizing Local Area Network (LAN) ATM emulation capability. Both local SCADA and central sites are considered virtually on the same LAN using shared infrastructure.


Page 16 of 63

Each pump station is managed locally by its local SCADA Operation group. However, the YCOT SCADA centers can control the sectional valves in between pump stations and provide 2P

ndP level control support for the local

SCADA operation support groups.

Figure 3.3 depicts a conceptual network design using IP over ATM network.

Figure 3.3: SCADA over IP using ATM Transport Network

Fiber Optics with SDH Nodes

Pump Station #1

5 0 0 P

P S 1

P S 2

O m n i S w i t c h

1 2 3 4 5 6 7 8 9

VACA

NT S

LOT

VAC

ANT S

LOT

VACA

NT S

LOT

VACA

NT S

LOT

MPX

OMNI

SWIT

CH 9W

X

2 POR

TS OC

-3 SM

F -A

TM

ASX-

155 R

FS

1 2

32 U

TP 10

/100

Mbps

-Et

herne

t

ESX

32 U

TP 10

/100

Mbps

-Et

herne

t

ESX

Ethernet Switch with ATM Uplink


5 0 0 P

P S 1

P S 2

O m n i S w i t c h

1 2 3 4 5 6 7 8 9

VACA

NT S

LOT

VAC

ANT S

LOT

VACA

NT S

LOT

VACA

NT S

LOT

MPX

OMNI

SWIT

CH 9W

X

2 POR

TS OC

-3 SM

F -A

TM

ASX-

155 R

FS

1 2

32 U

TP 10

/100

Mbps

-Et

herne

t

ESX

32 U

TP 10

/100

Mbps

-Et

herne

t

ESX

S D

SY ST EM S

SE

RE

THN

EXT

SELE

CT

RESE

TTXC

RXL

PW

R

A S X - 10 0 0

B DB DB D B D

A CA CA CA C

S D

SY ST EM S

SER

ETH

NE

XT

SEL

ECT

RE

SE

TT

XCR

XLP

WR

A SX -10 0 0

B DB DB D B D

A CA CA CA C

IT ATM Backbone Network

ATM Switches

Router IP Router Network

IT SDH Transmission

Network

Local Pump Station SCADA Network

Ethernet Switch

Digital Transmission

Fiber Optics

Ethernet Switch with ATM Uplink

Pump Stations #2 thru 11

YCOT-Yanbu

PI Servers

Digital Transmission


Page 17 of 63

3.4 SCADA Over Radio Network

SCADA over Radio based on UHF band, 460MHZ, is used to establish point to point network connectivity between the RTU’s and the SCADA host. This technology approach was implemented in Zuluf, Marjan, and Safaniya offshore, Haradh III and several other remote sites.

Radio RTU’s are depicted into the previous figures 3.2.2 and 3.3. One of the key disadvantages of Radio is the shortage of available frequencies in the VHF/UHF (Very High Frequency / Ultra High Frequency) bands. Besides, the disadvantage of the Radio frequency is that the transmission distant is limited to a line of sight path between antennas. Radio is subject to atmospheric attenuation, distortion, and interference.

4 Industry Trends

There are two major organizations working on guidelines for plant and enterprise interconnectivity: Purdue Enterprise Reference Architecture (PERA) and the Instrumentation, Systems and Automation Society (ISA). Both organizations strive to provide a fundamental basis for good design and operational procedures; though their approaches are from different perspectives.

4.1 Purdue Enterprise Reference Architecture (PERA)

The PERA group looks at system / plant / infrastructure from a cradle to grave approach ensuring that the full life cycle is identified and implemented in the most efficient manner. PERA model defines the need to separate critical plant functions such as plant communications and control away from the enterprise. In addition to this logical and physical separation, PERA provides a conceptual (design philosophy) means of interconnecting the enterprise and plant communications at locations conducive operational requirements while maintaining security. The Physical System Architectures are best designed using certain rules. These rules are based on assessment of the 4R's of system design, specifically: Response; Reliability; Repairability; and Resolution.

Each industry is likely to have different functional requirements, as well as different 4R's, which will result in different system architectures. Hence, it is left to the user of this model to define their network outlook based on the above criteria.


Page 18 of 63

4.2 Instrumentation, Systems and Automation (ISA) Society

The ISA’s assessment is based on many industry experts that have long contribution to the success of ISA standards development activities. ISA has an extensive history of collaboration with the following organizations for securing control systems communications and forming best practices guidelines: US National Laboratories, the NIST Process Control Security Requirements forum, The Chemical industry Data Exchange, the North American Electric reliability Council and various others in order to stay ahead of the growing threats.

The two main ISA standards of relevance to this assessment are:

● ISA IS-95 Enterprise-Control System Integration

● ISA IS-99 Security Technologies for Manufacturing and Control Systems

ISA research generated numerous documents and guidelines discussing the benefits and concerns with the legacy designs. Five major themes were stressed in most of the material found during the research and they are as follows:

1. Process Control System Communication Network Security: Historically process control systems (PCS) have been essentially proprietary and isolated. Over the past decade and with the rapid advances in Information Technology, these systems have become more open and powerful. DCS systems are increasingly based on standard technologies and communications protocols. Most DCS systems in Saudi Aramco are now connected to the local area network (LAN) either directly or via data historian or advanced process control applications. Hence, process control system communication network security becomes vital to overall business operation. This network security theme is inline with the current Saudi Aramco practices. The practice in Saudi Aramco is to install a Firewall between the Corporate and the Process Automation network; refer to SAES-Z-010 for more details.

2. Network and Communication System Attacks: Network and communication system shall adopt a plan, design and implement a system that ensures a high level of separation or implement extensive firewall/ DMZ setups to prevent an authorized access to the control network. This theme is also inline with Saudi Aramco; refer to SAES-Z-010 for more details.


Page 19 of 63

3. Cyber Security: System and network attack analysis has shown that the current threats are coming from outside hackers trying to break into systems, just for the sake of doing it or for malicious intent. Numerous incidents reports show electrical power plants shutting down or experience temporary loss of control. There have been cases of nuclear power generation stations also experiencing such issues. This over emphasize the need for designing secure systems and keep them updated to minimize the security venerability. This theme is also inline with Saudi Aramco; refer to SAES-Z-010 for more details.

4. Control System Operation and Availability: The primary requirement for the plant control system is to stay operational and in control. Systems must be designed and implement with that point in mind. If a shared communications system is employed, all facility and cost savings will vanish if a control system / plant communications becomes compromised either by a failure due to poor design or exposed vulnerability due to lack of isolation / security.

5. Communication Infrastructure Weaknesses: Primary concerns fall into two main categories; communication system redundancy (hardware, fiber, copper, etc.), and communication systems operational control. As with any critical communication connection, all possible single points of failure must be eliminated by providing infrastructure redundancy and or alternate paths to ensure operational status. The other point of concern is control/ownership of communication infrastructure that your critical plant controls links are connected through.

4.3 Industry Trends Summary

Here are the key recommendations of the industry trends. These recommendations were evaluated as part of our final analysis and recommended network architecture for remote Process Automation applications.

● The plant control system with its dedicated, isolated, private control LAN / communications provides the best way to ensure security and control. This is the practice within Saudi Aramco as outlined in SAES-Z-010.


Page 20 of 63

● In today’s demanding business environment, it has become common to utilize a tightly controlled firewall / multi-zone DMZ secured communication connection that allows the plant control system to operate securely while providing a safe means of sharing required plant information with the enterprise proponents. This is the practice within Saudi Aramco as outlined in SAES-Z-010.

● Shared communications infrastructure is used to transport the remote plant

control information and mainline enterprise (voice, data and video) communications as long as designs provide congestion free, accurate, priority QoS based delivery of plant control data in a consistent manner.

Utilize separate communications systems if the facilities and funds are available or tightly control and secure a shared communications system to handle both plant process controls and enterprise requirements. This will be part of our recommendation since fiber optics and communication infrastructure is invested in the new fields (i.e.,Khurais, Khursaniyah, etc.).

● Plan to eliminate or minimize all single points of failure and if at all possible

procure, install and control your own communication infrastructure. This will be part of our recommendation since fiber optics and communication infrastructure is invested in the new fields (i.e.,Khurias, Khursaniyah, etc.).


Page 21 of 63

5 Future I-Field/SCADA/Remote Sites Network Architecture

An evaluation criterion was developed based on different performance issues outlined in section 5.1. The criteria a long with the impact was mapped and tabulated in section 5.2. A recommendation is provided based on this assessment in section 5.3.

5.1 Existing SCADA Performance & Reliability Issues

The table below summarizes the different performance issues for the SCADA systems in different areas within Saudi Aramco. The most common performance issues are enumerated below:

Network Design

Performance issues

Dedicated Lines Low speed/Daisy chained RTUs. Failure of one RTU will impact subsequent RTU’s

Primary: IP/ATM/SDH Backup: IP/ATM/Radio

Performance issues when failover from SDH to the back up radio links. Also, delay is noticeable when other bandwidth demanding applications (e.g., Video streaming) are pushed through the network on the E/W link Route

Dedicated Bandwidth/OTN

System freeze due to sever failover from primary to secondary. This issue is attributed to the point to point connection switch over at the Terminal server located in the control room. Self healing ring is not working when an intermediate transport node fails. This is attributed to the node technology specs where the node uses a shared CPU for both primary and backup links.

Dedicated lines over SDH

Synchronization and network timing issue in the transport communication network. Fiber cuts due to lack of coordination between plant project groups and plant operation. Also, fiber cables wiring are environmentally prone to attacks by rats.

Daisy chained Design

A failure of one RTU impact the performance of some RTU in the daisy chain link. A failure of one of the intermediate instruments impacts the subsequent instruments that are daisy chained on the same link.

Self healing transport network ring architecture

A node failure does not properly trigger the self healing ring capability. So, all nodes that is below the failed node become out of service.

System Servers: Primary to Backup

Intermittent access failure between the application servers (Primary and back up) and the Network Attached Storage (NAS). In addition, frequent failures of the primary and the back up servers. Intermittent failure in the switching between the Primary and the back up server.

Table 5.1: Existing Process Automation Performance Issues


Page 22 of 63

5.2 Analysis & Results

There are different design options that can be used to support Remote Process Automation Application traffic. These are:

1. Process Automation traffic over SDH Transport network using dedicated bandwidth or serial point-to-point circuit emulation utilizing shared infrastructure. This network design can be based on allocating a dedicated bandwidth for each specified service.

For example, a dedicated bandwidth for SCADA, a second for CCTV, etc. The second option is where all of the services are subscribed and queued to one dedicated bandwidth.

2. Process Automation traffic over IP/ATM based on using ATM best effort LAN emulation capability.

3. Process Automation traffic has a dedicated fiber optics strand where SCADA will have its own SDH/OTN network.

4. Process Automation traffic utilizing dedicated Giga Ethernet over dedicated fiber optics infrastructure (dedicated strands).

Five (5) performance criteria were used in a comparative analysis format for the stated above design options. These are: Traffic impacts, Security vulnerabilities, Cost, Support & Management operation, and System availability.

Each category for each network design was qualitatively mapped to either a Yes (for yes there is a negative impact) or a No (there are no negative impacts). The degree of impact was not reflected as this is table was used to qualitatively screen the most positive design option. Table 5.2 outlines the results.


Page 23 of 63

SCADA over SDH using

Shared Infrastructure

SCADA Over ATM

Shared Infrastruct-

ure

Dedicated over Dedicated Fiber

Strands

SCADA over GE Ethernet

Dedicated Strands

Network Design Impact Criteria

Option # 1

Dedicated Bandwidth

using Shared Infrastructure

Option #2

Subscription over shared

Infrastructure

Option #3 Option# 4

Single SDH Node

Option # 5 Dual SDH Nodes

Option # 6

Traffic Impacts No (1) Yes (7) Yes (13) No (19) No (25) No (31)

Security Yes (2) Yes (8) Yes (14) No (20) No (26) No (32)

Cost No (3) Yes (9) No (15) Yes (21) Yes ++

(27) No(33)

Support & Management Yes (4) Yes (10) Yes (16) Yes (22)

No (28) No (34)

System Availability Yes (5) Yes (11) Yes (17) No (23) No (29) No (35)

Results (# of No’s)

2 No's (6) 0 No's (12) 1 No (18) 3 No's (24) 4 No's, but very costly

(30) 5 No's (36)

+ = costly ++ = Very Costly No: Signify no impacts; positive. Yes: Signify an impact and a concern; negative. (#): Note number with details in Appendix 1.

Table 5.2: Comparative Analysis- Refer to Appendix 1 for details

The following is a summary of the alternatives that would provide sufficient reliability at an optimal cost. These are:

1. Dedicated bandwidth, option #4; using dedicated fiber optics stands based on single node SDH ring architecture.

2. Industrial Ethernet over dedicated fiber optics strands, option #6. This option eliminates the need for an SDH network layer thus improve cost and reduce network traffic encapsulations delays. This option is being trial tested by Saudi Aramco Project Management at this time.

3. Option #1 will be suitable for remote process automation application where dedicated infrastructure does not exist (e.g., East/West Pipelines, Water Injection Plants); since the company has already invested in a self healing SDH network.


Page 24 of 63

Implementing this solution for East/West pipelines for example, will substantiate the existing ATM performance issues and position the implementation of an efficient secure solution: implementing one (1) firewall at the Yanbu YCOT CCR Room rather than implementing a Firewall at each Pump station.

The following options are not recommended due to either manageability, availability, security concerns or they are at a higher cost. These are:

1. Option 3 (Utilizing ATM over SDH) is currently used in East/West Pipelines SCADA Network and will be replaced by option #1; upon BI-10-00179 implementation.

2. Option 2 (Subscription over shared) requires higher level of switching intelligence to support necessary quality of service settings. The complexity in managing and associated cost does not warrant this option.

3. Option 5 (Dual SDH nodes) provides higher level of redundancy but economically unjustifiable.

5.3 Remote Process Automation Application Network Architecture

The different network architectures outlined in this document and associated analysis has lead to two acceptable network technology and topology designs for future Process Automation networks. These are:

1. Industrial Ethernet using dedicated dark fiber in a self healing ring topology.

2. For existing implementations, utilize dedicated bandwidth based on Synchronous Digital Hierarchy (SDH) transport network self healing ring architecture supported by a dedicated fiber optics strands.

These two network toloplogies implemenation shall be based on the following general guidelines:

• A standalone fiber optics cable shall be utilized to connect remote sites (i.e., well sites) to a centralized Process Automation Network (PAN). Alternatively, Radio links, or Broadband wireless network solutions could be used to connect dispersed remote sites where fiber optics cable prove economically not feasible.


Page 25 of 63

• Self healing ring: is a ring topology based on a uni/bidirectional links between a set of stations providing optimum protection against fiber cable cut and equipment failure.

• Utilize point to point ring system for scattered sites and/or where a true closed loop physical ring topology is economically not justifiable.

• A PAN comprising of multiple scattered (PANs), shall interface with the Corporate Network via a centralized firewall(s). These scattered PANs shall be connected together to form a unified PAN utilizing the corporate SDH transmission infrastructure with a dedicated bandwidth of 10 Mbps. Alternatively, dark fiber can be utilized if available.

• SCADA over IP/ATM is a unique implementation for East/West Pipelines. It was recommended to PMT to move SCADA traffic from ATM and have it directly transported over a dedicated SDH transport network to fulfill the physical separation requirements as per SAES-Z-010. This would be part of the ongoing BI (BI-10-00179) that will replace the existing Microwave system to SDH.

Any future implementation similar to the East/West Pipelines such as Water Injection Plants, scattered GOSPs will be based on either one of the two design options stated above. This is mainly because remote GOSPs or WIPs operation is tightly coupled with centralized process control systems.

Detail design requirements for these two wireleine design options are reflected in section 5.3.1. These two design options will provide the required physical and logical separation between Plant Automation Networks (PAN) and other traffic.

The outcomes of each recommendation will be mapped to the appropriate Process Automation Standard to be included as part of the next revision. In the interim this report will serve as the guide for any future SCADA, I-Field, and remote site network connectivity until all the standards are updated with these recommendations.

5.3.1 Future Process Automation (SCADA, I-Field, Remote sites) Network Design Architecture

The architecture for I-Field, SCADA application, and Remote sites Process Automation Application communication network shall comply with the following guidelines:


Page 26 of 63

1. The remote Process Automation applications network architecture shall be based on either one of the following configurations:

i. Industrial Ethernet using dedicated dark fiber in a self healing ring topology for grass root projects such ,Khurias, Manifa, etc,

Figure 5.3.1.1 depicts the network topology.

ii. For existing implementations, utilize Synchronous Digital Hierarchy (SDH) dedicated bandwidth based on self healing ring architecture.

Figure 5.3.1.2 depicts the network topology.

iii. Radio links, or Broadband wireless network solution could be used to connect dispersed remote sites where fiber optics cable prove economically not feasible.

iv. Number of nodes in the Ring topology shall be carefully

calculated from a time delay, bandwidth utilization, and ring switch over perspective so that application performance is not hindered.

v. Self healing ring topology is based on optical ring with

physical route diversity that provides protection against fiber cable cut and equipment failures.

Also, the optical ring can be a linear system without a physical routed diversity. This system would protect against equipment failure for sites that are limited/scattered or where a true physically ring topology is not economically justifiable.

vi. Industrial Ethernet will be used as a migration path for the existing Optical Transport Node (OTN). Expansion for existing OTN fields’ networks are recommended to be supported by Industrial Ethernet; thus eliminating the need for communication shelters and the associated cooling requirements.


Page 27 of 63

2. The network design shall provide physical and logical separation between Plant Automation Networks and Corporate Network. However, logical separation utilizing the company’s provided transmission network (i.e., dedicated SDH bandwidth) is permitted for scattered and remote sites with a minimum SDH bandwidth of 10 Mbps.

3. Real-time control communications must be supported on plant owned

and dedicated infrastructure.

Supervisory control and remote engineering communications can utilize the company’s provided transmission network (i.e., dedicated SDH bandwidth, or dark fiber). However, a prior approval from P&CSD manager for SCADA applications that are directly linked to the Emergency Shutdown (ESD) systems.

4. Monitoring plants applications from the corporate network shall be allowed via only proxy servers.

5. Logical bandwidth separation between each remote site and the

Computer Control Room shall be achieved by provisioning dedicated bandwidth (i.e., trunk). The following shall be considered when supporting this network topology model:

i. Logical separation (i.e., dedicated SDH link) must be utilized to separate supervisory control and data acquisition traffic from other traffic such as voice, CCTV, etc.

Refer to Appendix B for Bandwidth Calculation of this report.

6. Industrial Ethernet over dark fiber network design ring topology

depicted in figure 5.3.1.1, is the direction for future remote Process Automation application network deployment as stated previosuly. This should eliminate the need for an SDH network layer; thus significantly minimize cost and network traffic encapsulations delays.

This network design shall adhere to the following:

i. Hardened industrial type communication equipment shall be used for all outdoor Process Automation networks.

Industrial switches have higher availability, require minimal software revision upgrades, and have longer life cycle.


Page 28 of 63

Moreover, these switches can interface directly to Ethernet enabled RTUs, PLCs, instruments, and services stations.

ii. Industrial Ethernet Access switches with 1 Gbps uplinks shall be installed at the remote sites in a ring network topology connecting to the backbone Giga Ethernet switches.

iii. Industrial Ethernet Access switches shall have VLAN capabilities to support traffic segregation.

iv. Industrial Ethernet Access interface ports at the remote site shall be physically and logically interoperable with the Remote Terminal Unit and I-Field Surface Units (IFSU; i.e., PDHM, MPFM, etc) instrumentations. This includes but not limited to port auto-sensing, Duplex/simplex, cable pin-out automatic configuration, etc.

v. Ethernet backbone switches shall be redundant and provide

minimum speed of 1 Gbps utilizing ring network topology architecture.

vi. Ethernet backbone switches shall be layer 3 non blocking switches; (carrier class core switches).

vii. Ethernet backbone switches shall perform the routing functionality for whole network.

viii. The uplink switch-over for for both access and backbone Ethernet switches shall not exceed applications maximum time out parameters.

ix. All TCP/IP addresses shall be obtained from Saudi Aramco network management group.

x. Provide integrated Domain Name Service (DNS) and Windows Intergated Name Service (WINS) to address the name IP address resolution.

xi. Tight integration (e.g., using OPC Tunneling) between

Network Management System and the Process Automation application (e.g., SCADA) is recommended for the ease of overall PAN network manageability.


Page 29 of 63

Figure 5.3.1.1: Future Network - Giga Ethernet over Dark Fiber; Eliminating the Need for SDH/OTN

Remote Sites Ring Size Dependent on # of RTU’s 10/100/1 GGE

Giga Ethernet Switches

Giga Ethernet Switches 10/100/10G

10

10 10

10

Redundant GE Ethernet Switches

Remote Radio

RTU=1, thru n

MIS User

CCR


Ethernet

ENG & OPR Console

PI Server (etc..)

Firewall

IT Wide Area Network

Host Radio

RTU=1, thru n

Oil & Gas Fields /Remote GOSP

Remote GOSP


Enterprise PI Server

Enterprise PI User


Page 30 of 63

Figure 5.3.1.2: Future Networks-Dedicated Bandwidth over SDH Transmission Network Using Dedicated Fiber Strands

Oil & Gas Fields Ethernet Switch

CCR

Fiber Optics Backbone Transmission Ring


Giga Ethernet

ENG & OPR Console

Ethernet Switch

5 0 0 P

P S 1

P S 2

O m n i S w it c h

1 2 3 4 5 6 7 8 9

VAC

ANT

SL

OT

VACA

NT

SLO

T

VAC

ANT

SL

OT

VAC

ANT

SL

OT

MP

X

OM

NI S

WIT

CH

9WX

2 P

ORTS

OC

-3 S

MF

-AT

M

ASX

-155

RFS

1

2

32

UTP

10/1

00

Mbps

-E

ther

net

ES

X

32

UTP

10/1

00

Mbps

-E

ther

net

ES

X

Data Acquestion Hostorian Server

WHESD/ESP/PDHM, etc..

Fiber optics Transmission Access Ring

REMOTE GOSP

Wireless

Firewall


Process Automation Network

Enterprise Office User


MIS Work Station



Page 31 of 63

8. Data acquisition frequency (e.g., polling cycle, deadband) shall be identeifed by the proponnet in the Funcational Specification Dcoment for each installation.

Comments: For I-Fields pressure transient analysis application data acquisition frequency rate shall be set to 1 second. If the communication protocol is based on report by exception, then the deadband shall be set to 0.01 for the Permanent Down Hole System Pressure and Temperature readings.

9. Redundant RTU and network interface requirements (Single vs.

Dual) to the Process Automation servers will be determined at the early design stage for each application. For example, high pressure gas wells may require complete end-to-end system, RTU, and network redundancy.

10. Each RTU shall have its own dedicated network connectivity. A failure of one RTU shall not impact the data acquisition from other RTUs. It shall also be equipped with local storage capacity that can support local data acquisition for 96 hours.

11. The design criteria for the installation of electrical instrumentations

are covered in SAES-J-902.

For I-Field Surface Units (IFSU; i.e., PDHM, MPFM, etc) network connectivity shall adhere to the following guidelines:

i. Communication interface between IFSU and the RTU/Ethernet switch shall be interoperable. Ethernet is preferred.

ii. The connectivity design between IFSU and the RTU/Etherenet switch shall be based on a dedicated point-to-point connection. Daisy chained IFSUs wiring design is not permitted to minimize failure impacts to the overall business operation.

iii. IFSUs shall be able to support standard Ethernet and/ or Serial interfaces. Ethernet is preferred.

iv. IFSU shall be equipped with local storage capacity that can support local data acquisition for 96 hours.


Page 32 of 63

12. Process Automation application servers shall be deployed as follow:

i. SCADA and other critical process applications such as I-Field computer system hardware (e.g., servers) shall adhere to 23-SAMSS-20.

ii. Non mission critical applications (e.g., Cathodic monitoring, Power System Automation, etc.) shall be supported by single servers.

Comments: It is recommended at the design stage for new projects to minimize the number of servers at the Process Automation control room by combining applications on the same set of servers; if feasible.

15. A Business Continuty Plan (BCP) shall be established to remedy possible network design deficiencies if any; but may help reduce the impacts and improve the response time upon a network failure.

5.3.2 Next Steps

Following are the next steps to be completed as part of Phase II of this report.

1. P&CSD will evaluate different industrial Ethernet vendor solutions, complete performance testing and test procedures.

2. The outcome of this effort will be used to develop the required material specifications for Industrial Ethernet that would accommodate Saudi Aramco Process Automation requirements for I-Field, SCADA, and Remote Site network connectivity.


Page 33 of 63

Part II: Process Automation Wireless Access

1 Background

This wireless Process Automation outlook was carried out as part of plant networks optimization efforts. The plan is to enable and promote efficient and cost effective and proven industrial wireless technologies for various process related applications such as I-Field, SCADA and other process remote monitoring. Also, this effort would help to streamline the different ongoing Process Automation wireless trials into uniform implementation for future projects.

In this effort, we emphasized on the importance of considering and complying with the requirements of critical and time-sensitive process information applications as well as considering wireless reliability, security and interoperability. Moreover, we avoided discussing proprietary wireless solution since it hinders capability, interoperability and flexibility.

2 Introduction

Wireless technology is becoming increasingly popular solution for a wide range of applications. After evaluating the technology, many users are convinced of its reliability, satisfied with its performance and are ready to use it for different applications in a large-scale and more complex wireless networks.

Wireless LANs (WLANs) differ from regular “WIRED” LANs in the means they use to transport data. WLANs use air instead of a network wired cable as a medium for data transport. During the last few years, WLANs have many applications in warehousing, manufacturing, airports, hotels and health-care.

Recently, wireless technology is becoming among the most significant cutting-edge technologies as a viable network connectivity option for Process Automation networks (PAN) as well as instrumentations networking. Different Wireless technologies are benchmarked in this assessment and were mapped to the suitable Process Automation application (s).

3 Wireless for Process Networks

In nearly all industrial plants, communication links carry vital information between field devices (instruments, sensors, actuators, etc.) and the process control center. From periodic updates to ongoing process and manufacturing management, reliable and secure data flow is critical operations.


Page 34 of 63

Control message and status information transferred in industrial settings—actuator position, temperature, or liquid levels, for example—is carried in short “bursts” which require relatively little bandwidth and connection speed. At the other extreme, large file transmission, such as activity logs from a production run, requires moving a lot of data very efficiently.

Whatever the specifics of the data being moved, all industrial communications share common critical requirements: Secure, Reliable and Timely data delivery.

The following applications have been identified as potential industrial applications for wireless access:

1. Exploration and Producing operation (Wireless-I-Field, data connectivity)

2. Remote Data Acquisition (temperature, pressure, flow, level)

3. Energy management & sub-metering

4. Process monitoring and control (Condition based, Power, Vibration, wellhead,…etc.)

5. Waste water management

4 Industrial Wireless vs. Non-Industrial Wireless

The Process Automation applications are considered mission critical applications and conventional wireless solution won’t be able to serve these applications in an active environment. Hence, industrial wireless is considered to be the alternate solution that would ensure security, reliably and real time delivery for the Process Automation data.

The Industrial Wireless solution should meet the required process networks reliability, availability and security as shown in Table 4.1.


Page 35 of 63

No Industrial requirements for Industrial Wireless solution

Office / consumer requirements for non-industrial wireless

1 Reservation of bandwidth for selected critical clients (e.g. PLC), Uguaranteed accessU to radio channel

Radio channel is a shared medium; everybody can access it (best efforts)

2 The communication cycles are predictable for UdeterministicU data traffic

The radio traffic is prioritized, but cannot be predicted

3 Redundant radio communication on two separate frequency bands for increased availability (URedundancyU)

Usually one radio channel

4 Metal casing, vibration-resistant connectors and cables Low-cost plastic casing

5 IP 65 degree of UprotectionU, +60°C and moisture-resistant for harsh industrial and outdoor use For home and office use

6 UIndustrial approvalsU: ATEX (EX area), Area Classification, EMC, UL, FM

No demand for increased requirements (CE, shock protection)

7 UFlexibleU use through Batteries, 18 - 57 VDC, 100 – 240 VAC, Power-over-Ethernet

Voltage supply 100 VAC – 240 VAC, PoE

8 Ethernet, antennae cables, casing Uresistant to chemicals U, halogen-free, silicon-free, flame-retardant

No demand for approvals with increased requirements

Table 4.1: Industrial Wireless vs. Non-Industrial Wireless

5 Industrial Wireless Classifications

Industrial wireless technologies can be classified into four main categories based on their coverage area as shown in Figure 5.1. These are:

1. Wireless Personal Area Network (WPAN) – few meters

2. Wireless Local Area Network (WLAN) – few 100’s meters up to few Km

3. Wireless Metropolitan Area Networks (WMAN) – few Km

4. Wireless Wide Area Networks (WWAN) – 10’s up to 100’s of Km


Page 36 of 63

Figure 5.0: Industrial Wireless Classifications Bandwidth vs. Distance

5.1 Wireless Personal Area Networks (WPAN)

The Wireless Personal Area Network (WPAN) technology uses a short-range radio link that has been optimized for power conscious, battery operated, small size, lightweight devices.

There are two dominant WPAN technologies namely Bluetooth (IEEE 802.15) and Zigbee technologies.

The UBluetooth Technology operates in the 2.4 GHz band and used in several computing devices such as notebook computers, phones, PDAs to exchange information. Bluetooth is not considered as potential industrial wireless solution for Process Automation due to the excessive latency and limited networking capabilities due to synchronization handshaking process.

UZigbee U Technology addresses the unique needs of low-cost, low-power, wireless instrumentation/sensor networks for remote monitoring & control and building automation network applications in the industrial plants at short distances (few 10s of meters). The zigbee standard was ratified in 2006..

WLAN802.11g

ZIGBEE 802.15.4

BlueTooth 802.15.1/1a

UWB(802.15.3a?)

WLAN 802.11b

2.5G 3G

10m

100m

2km

1Mb/s 50Mb/s

10km

20km+

100Mb/s

WPAN

WLAN

WMAN

WWAN

WLAN802.11a

M WiMAX 802.16e

4G

MWBA 802.20

FWBA802.16

3.5G

5km

22000088//99

22000066 22001100

22000066

22000055

WLAN802.11n

22000066

BlueTooth2.0

802.15.4a

22000055

22000044

22000055

22000033

11999999

11999999

Bandwidth

Distance


Page 37 of 63

The general specs of Zigbee along with IEEE 802.15.4 solution could be summarized as follows:

● Dual PHY (2.4GHz ISM and 868/915 MHz)

● Data rates of 250 kbps (@2.4 GHz), 40 kbps (@ 915 MHz), and 20 kbps (@868 MHz)

● Optimized for low duty-cycle applications (<0.1%)

● CSMA-CA channel access

● Yields high throughput and low latency for low duty cycle devices like sensors, instruments and controls

● Low power (battery life multi-month to years)

● Multiple topologies: star, peer-to-peer, mesh

● Addressing space of up to:

○ 18.45Exp+16 devices (64 bit IEEE address)

○ 65,535 networks

● Optional guaranteed time slot for applications requiring low latency

● Fully hand-shaked protocol for transfer reliability

● Range: 50m typical (5-100m based on environment)

In conclusion, the Zigbee solution (in conjunction with IEEE 802.15.4) is considered as a potential emerging short distance wireless technology (WPAN) for instrumentations networking for periodic, intermittent and slow latency applications. It must be noted that Zigbee has the major limitation with coverage area; distance coverage in the 10’s of meters. The overall Zigbee model is shown in Figure 5.1.1 and Zigbee Network Architecture is shown in Figure 5.1.2.


Page 38 of 63

Figure 5.1.1: ZigBee Network Model

Figure 5.1.2: ZigBee Network Architecture

5.2 Wireless Local Area Networks (WLAN)

IEEE 802.11x and HiperLAN1/2 are the two primary WLAN open standard technologies in the market. The HiperLAN (High Performance Radio LAN) solution is a European WLAN technology however; this technology did not see the light.


Page 39 of 63

On the other hand, IEEE 802.11(a,b,g) is a mature and proven wireless technology that already implemented in various applications. Currently, there are three main IEEE 802.11 WLAN standards each operate with different characteristics such as modulation type, data rate, frequency band, and transmit power (Table 5.2).

Table 5.2: IEEE 802.11 Standards

It should be noticed that IEEE 802.11b and g are backward compatible wireless technologies that operate in the 2.4GHz band (ISM) while IEEE 802.11a operates in the 5GHz band. The operating frequency, the data transfer rate, the transmission technology and compatibility of the present IEEE standardized WLANs is illustrated in Figure 5.1.


Page 40 of 63

Figure 5.1: IEEE 802.11 Standards Compatibilities

There are three main open standard frequency bands known as Industrial, Scientific and Medical (ISM) bands as shown in Figure 5. The 900 MHz band is becoming overcrowded due to various consumer products that operate at the same band. It does offer longer range (for the same gain antennas) than the 2.4 GHz band and 5GHz, but it has limitations on the maximum size of antennas that limits its overall range and the maximum data rate that is reliably obtained is under 1Mb, due to the limited frequency range.

At 2.4 GHz, the lower power transmitter allows very high gain antennas, which allows long distance communication (up 15km). The frequency range is also much wider than 900 MHz, allowing higher data rate (54Mbps) with a reliable range.

The 5 GHz band offers more bandwidth, allowing higher data rates; however, the nature of the higher frequency limits range. Typical range for 5 GHz band is much lower than the other ISM bands.


Page 41 of 63

Figure 5.2: WLAN Frequency Band Characteristic

For critical industrial applications, Access Point (AP) should support polling mechanism to guarantee application channel access moreover, redundant wireless connection in the 2.4 GHz and 5 GHz bands which is known as Tri-Mode Dual-Band solution. Both signals (2.4 GHz and 5 GHz) are transmitted simultaneously to achieve extremely high level of radio channel reliability and to guarantee interference-free wireless connection (Figure 5.3).

Figure 5.3: WLAN Radio Redundancy

900 MHz band 2.4 GHz band 5 GHz band

PROs

CONs

Greater range than 2.4 GHz band ( for in- building LANs)

Global market IEEE 802.11 Higher data rates (10+ Mbps)

Global market IEEE 802.11 Higher data rates (20+Mbps)

Less range than 900 MHz

Maximum data rate 1 Mbps Limited bandwidth Crowded band

Much less Range than 900 MHz or 2.4 GHz Higher cost RF components Large antenna required


Page 42 of 63

The IEEE 802.11a,b,g wireless technologies are considered a potential wireless solution for mid range (100’s of meters up to few Km) applications. Remote monitoring such as wellhead monitoring, motors monitoring, vibration monitoring interconnectivity are candidate applications for IEEE 802.11a/b/g WLAN solution as shown in Figure 5.4.

Figure 5.4: WLAN Industrial Applications

5.3 Broadband Wireless Connectivity

There are several wireless solutions falls under broadband wireless connectivity categories. However, WiMAX technology is the most promising last mile wireless connectivity solution. WiMAX is a worldwide wireless standard that addresses interoperability across IEEE 802.16 standard-based solution. WiMAX technology offers greater range and bandwidth than the WLAN family of standards and provides a wireless alternative to wired backhaul and last mile access (Figure 5.3.1).

Generally, WiMAX technology can reach 1 ~ 50km+ coverage radius and achieve data rates up to 75+ Mbps. Moreover, WiMax can provide a cost effective and flexible point-to-point connection.

FinFan Machine Tool

Central Control Room Roll

Pump

Access Point

PLCs


Page 43 of 63

Figure 5.3.1: Fixed WiMAX Deployment Models

WiMAX solution would require frequency approval from local authority (CITC) and should be requested through a 3 P

rdP party local service provider. This may

hinder WiMAX utilization for Process Automation applications.

5.4 Wireless Wide Area Networks (WWAN)

Cellular Network (GSM) is considered WWAN solutions. GSM network could be utilized to provide cost effective connectivity for various semi-real time, non critical applications. P&CSD had successfully deployed WWAN solution using GSM network (Short Message System/GPRS) to monitor the performance and status of the water treatment plant instruments, sensors and equipment (pumps, MOVs, Flow, Voltage, Temp,..) as shown in Figure 5.4.1. Customer was able to remotely monitor water treatment plant process and takes immediate corrective action as needed as well as eliminated the daily dispatch of the technician to the remote site.


Page 44 of 63

Figure 5.4.1: GSM Solution for Process Monitoring (SMS/GPRS)

This solution could be used for non critical applications that are covered with GSM network. Similar solutions could be developed using TETRA solution in the future.

Technology Summary: Hhttp://eway.aramco.com.sa/cops/coredocs_cop/Wireless Remote Monitoring and Control using SMS Technology for Water Treatment Plant _PCD-13_03-T.PDF

Technology Presentation: Hhttp://eway.aramco.com.sa/cops/coredocs_cop/Wireless Remote Monitoring and Control using SMS_Final.PDFH

Finally, Satellite communication (LEO, MEO, GEO/VSAT) is also considered as WWAN connectivity solution. This design option can be utilized in a point to point or point to multipoint connection. However, this design option can be costly. So, the concept of utilizing WLAN as an aggregator at the remote sites and then use Satellite communication to uplink the WLAN for the different remote sites back to a Centralized location.

Satellite communication typically is susceptible to latency, security and running cost that should be considered when selected in any future Saudi Aramco projects.

RIYADH REFINERY Remote Site (STP)

PPrreessssuurree

TTeemmpp

AAmmppss

MTU MTU PPuummpp

GSM Wireless Network


Page 45 of 63

6 Wireless for Process Network Conclusion

Industrial Wireless solution provides cost effective and efficient connectivity for different plant process systems, instruments and devices. This benchmark was carried out as part of plant networks optimization efforts by enabling and promoting efficient and cost effective mature and proven industrial wireless technologies for various process related applications such as I-Field, SCADA and process remote monitoring.

It is vital to consider and comply with open standard wireless solution that would meet the requirements of critical and time-sensitive process information applications as well as considering wireless reliability, security and interoperability.

The Zigbee solution (in conjunction with IEEE 802.15.4) is considered as a potential emerging short distance wireless technology (WPAN) for instrumentations networking, subsystems networking of periodic, intermittent and slow latency applications.

Industrial WLAN solution (IEEE 802.11a,b,g) is considered the current mature and proven wireless technologies for mid range (100’s of meters up to few Km) applications. Wellhead monitoring, motors monitoring, vibration monitoring, PLC interconnectivity are ideal applications for the IEEE 802.11a/b/g WLAN solution. Hybrid wired/wireless solution is the current practical model where sensors and instrumentations are connect to a concentrator through short cables (bus) then the acquired data are sent through the Wireless Access Point (WAP) to a centralized location for processing, analysis and decision taking.

For semi-real time non-critical applications, WWAN (GSM/SMS/GPRS/TETRA) could be utilized to provide efficient and cost effective process monitoring solution.

In short, Industrial WLAN option is the current viable and mature wireless solution that would meet the requirement of sensitive Process Automation applications through a hybrid wired/wireless configuration. Total throughput of around 22Mpbs would be available in a single wireless cell/link when using IEEE 802.11a,g technology.

7 Next Steps

P&CSD will evaluate different implementations of Wireless solutions. The outcome will be used to benchmark these different wireless technology implementations against wired communication.

The results will be used to develop the required material specifications for wireless solution that would accommodate Saudi Aramco Process Automation requirements for I-Field, SCADA, and Remote Site network connectivity.


Page 46 of 63

Part III: Network Timing & Synchronization Architecture

1 Synchronization Architecture

There are several methods for synchronizing a complex digital network, computer systems, and end devices. The most common method, and the one used by Saudi Aramco, is a master-slave arrangement as shown in Figure 1.1 of this section.

Figure 1.0: Saudi Aramco Atomic Clock System Architecture

This method is based on the principle of designating one clock as the master clock and locking (slaving) all other clocks to it. The master clock for the Saudi Aramco network has been installed in the Dhahran EOB digital central office and is based on an atomic (Cesium) standard with a lifetime stability of one part in 10EXP-12. Three clocks are installed together and outputs are constantly analyzed on a two-out-of-three comparison algorithm. A caution alarm is generated when the frequency difference is over 1 x 10EXP-11, and a frequency alarm when the frequency difference is over 1 x 10EXP-9.

Atomic Clock Dhahran EOB

DCO: Digital Central Office

JGP DCO RT DCO JD DCO

Field Systems

Field Systems

Field Systems

Dha


Page 47 of 63

Clocking signals from the Dhahran EOB are imposed on the digital microwave and fiber optics facilities, which in turn synchronize the other digital systems in the company. If these digital links (or the Dhahran clocks) were lost, the local digital systems at a site provide local synchrony until the master clocks could be reconnected.

The second most common method is based the Global Positioning System (GPS), Figure 1.2. GPS provides an extremely precise time reference, required for telecommunications and some scientific research, including the study of earthquakes. Hence, it is considered as an optimal solution in synchronous systems that require high timing precision.

GPS is now widely used in closely coupled real time control systems that require synchronization in the range of mili to micro-seconds. An excellent example is power system distributions and control, Distributed Control System (DCS) for petrochemicals, and other manufacturing applications.

GPS system is based on 24 satellites rotating on 6 different orbital planes with 12hour orbital period. The satellites are spaced so that from any point on Earth, at least four satellites will be above the horizon. GPS receiver calculates position and time based on signal received from 4 satellites.

Figure 1.2: GPS Network Architecture

The GPS signal is typically feed into a Network Time Protocol (NTP) server (s). The NTP protocol feeds the time synchronizations clocking to all digital systems that reside in the same communication infrastructure. Figure 1.3 outlines the system hierarchy for NTP protocol based on GPS feeder clocks.


Page 48 of 63

Figure 1.3: GPS/NTP System Architecture

GPS is currently installed at different Saudi Aramco facilities (e.g., JGP, Abqaiq, etc.) to synchronize the time between DCS nodes within a plant.

• Due to GPS accuracy, the strategy is to synchronize the remote sites with the application server based on GPS technology for all the impacted sites relevant to Process Automation.

• To minimize cost, existing GPS/Network Protocol Timing (NPT) will be extended to the remote sites via the communication infrastructure.

• All process automation system components (PDHM, MPFM, RTU, SCADA, PI systems and including communication infrastructure) shall be synchronized to minimize system slippage and data acquisition coloration quality issues.

2 Next Steps

P&CSD will evaluate different implementations of Time synchronization and develop guidelines or standards to support future implementation.


Page 49 of 63

APPENDICES

Appendix 1: Performance Criteria Analysis

Appendix 2: Bandwidth Calculations for Wireline Network

Appendix 3: Wireless Technology Classifications

Appendix 4: Frequency band set aside for general use industry, scientific, Medical (ISM)

Appendix 5: IEEE 802.11 WLAN standards model

Appendix 6: IEEE 802.11 WLAN Standards Model Appendix 7: Support Structure


Page 50 of 63

Appendix 1: Design Performance Analysis

Four (4) different design options that can be used to support I-Field. SCADA, etc..traffic from the Remote Terminal Unit to the application server (s). These are:

1. Company’s SDH transmission network; using dedicated bandwidth or serial point-to-point circuit emulation utilizing shared infrastructure; Figure 3.1 and Figure 3.2.1

2. IP/ATM; based on using ATM best effort LAN emulation capability; Figure 3.3

3. Dedicated Giga Ethernet over dedicated dark fiber optics infrastructure (dedicated strands); Figure 5.3.1.1

4. Plant owned SDH transmission network (i.e., Siemens OTN); Figure 5.3.1.2 Five (5) performance criteria were used in a comparative analysis format. These are: 1) Traffic impacts, 2) Security vulnerability 3) Cost, Support & Management operation and 5) System availability. Each criteria for each network design was qualitatively mapped to either Yes (for yes there is a negative impact) or No (there are no negative impacts). The degree of impact was not reflected as this is table was used to qualitatively screen the most positive design option. Table below outlines the results.

SCADA over SDH using

Shared Infrastructure

SCADA Over ATM

Shared Infrastruc-

ture

Dedicated Bandwidth over Dedicated Fiber

Strands Using

SCADA over GE Ethernet Dedicated Strands

Network Design

Impact Criteria

Dedicated Bandwidth

using Shared Infrastructure

Option # 1

Subscription over shared

Infrastructure Option

#2

Option #3 Single SDH Node Option

# 4

Dual SDH Nodes

option # 5

Option # 6

Traffic Impacts

No (1) Yes (7) Yes (13) No (19) No (25) No (31)

Security Yes (2) Yes (8) Yes (14) No (20) No (26) No (32)

Cost No (3) Yes (9) No (15) Yes (21) Yes ++ (27) No(33)

Support & Management

Yes (4) Yes (10) Yes (16) Yes (22) No (28) No (34)

System Availability

Yes (5) Yes (11) Yes (17) No (23) No (29) No (35)

Results (# of No’s)

2 No's (6) 0 No's (12) 1 No (18) 3 No's (24) 4 No's, but very costly

(30)

5 No's (36)

+ = costly ++ = Very Costly No: Signify no impacts; positive. Yes: Signify an impact and a concern; negative. (#): Note number with details in outlined in the next page

Table Appendix 1.0 : Comparative Analysis Note#1: No- There is no bandwidth traffic impacts when using dedicated bandwidth on a shared infrastructure. For

example, allocating 2 Mbps on an SDH system for a Process Automation application and 4 Mbps for Closed Circuit Television, these two dedicated bandwidth do not compete with each others.


Page 51 of 63

Note#2: Yes-There is either a potential of physical or logical security threat to the nodes and the traffic traversing this infrastructure.

Note#3: No-There are no incremental costs of using this option. Note#4: Yes-There is a system availability issue since the infrastructure is shared with other traffic. The lack of

system availability can be triggered by schedule or unscheduled maintenance windows. Note #6: the overall score is 2 areas of no impacts. The design option with higher number of No’s is considered a

favorable option. Note #7: Yes. The traffic impact is high when several applications are queued over a shared bandwidth. This is not

a favorable option since it creates extended delay. Note #8: Yes. Security is venerable in this design option since IP address translation will be required between the

intermediate nodes in a shared translation environment. Note #9: Yes. Cost is higher since this will require special switches for priority queuing and quality of service. Note #10: Yes. Manageability and support for this design option is more cumbersome and will require additional

network management systems/integrations to ensure reliable performance is achieved. Note #11: Yes. Availability is threatened by the need for closure network manageability and support. Hence, this

will require more maintenance windows that scheduled and unscheduled. Further, traffic congestion has direct impact on the overall system real-time availability.

Note #12: the overall score is 0 areas of no impacts. This design option is considered the worst design that can be used for real-time systems.

Note #13: Yes. Traffic impact is high since process automaton will be mixed with other utilizing ATM as a transport network. ATM network is susceptible to delay since it is based on Best Effort Network.

Note #14: Yes. A Venerable network solution since traffic routing translations is required on the access and backbone ATM network.

Note #15: No. A cost effective since it is existing network Note #16: Yes. Requires scheduled and unscheduled maintenance windows. Further, this will require integration in

management system for both the Process Automation network and ATM which will increase overall complexity and cost.

Note #17: Yes. Network availability is threatened by the overall performance of the ATM best effort network design and maintenance and support requirements.

Note #18: the overall score is 1 area of no impacts. This design option is considered the least design that can be used for real-time systems.

Note #19: No. These are no impact since this design is based on dedicated bandwidth allocation. Note #20: No. There are no impacts since this design is based on logical and physical separation. Note #21: Yes. There is additional cost since this option is based on dedicated access nodes and dedicated

infrastructure. Note #22: Yes. This requires support and maintenance utilization. Note #23: No. There no impacts on system availability as this provide an excellent option for real-time system. Note #24: the overall score is 3 areas of no impacts. This design option is considered a favorable design option. Note #25: No. These are no impact since this design is based on dedicated bandwidth allocation. Note #26: No. There are no impacts since this design is based on logical and physical separation. Note #27: Yes. There is significant additional cost since this option has dual dedicated access nodes and

dedicated infrastructure. Note #28: No. This does require less support and maintenance utilization. Note #29: No. There no impacts on system availability as this provide a fault tolerant network for real-time system. Note #30: the overall score is 4 areas of no impacts. This design option is considered too expensive and is not

recommended.


Page 52 of 63

Note #31: No. These are no impact since this design is based on dedicated bandwidth allocation. Note #32: No. There are no impacts since this design is based on logical and physical separation. Note #33: No. There is additional cost since this Ethernet switches are considered as part of the existing design

options. Further, these Ethernet switches can eliminate the need for the SDH transport node. Note #34: No. This does require less support and maintenance utilization. Note #35: No. There no impacts on system availability as this provide an excellent option for real-time system. Note #36: the overall score is 5 areas of no impacts. This design option is considered the most favorable design

option.


Page 53 of 63

Appendix 2: Bandwidth Calculations for Wireline Network

Utilizing system thinking and Kepner-Tregoe problem solving methods, the bandwidth performance requirements for process automation network was assessed and results were summarized in this appendix.

Process Automation network bandwidth calculation is contingent on the overall application requirements (e.g., puling cycle, packet size, delay sensitivity, etc). In addition, the nature of the application is a key factor in sizing the bandwidth requirements. Process control type applications require ample bandwidth on the range of 10 times the effective bandwidth utilization (i.e., bandwidth utilization at 10%). This is commonly seen in DCS environments.

On the other hand, semi real time application (e.g., SCADA, I-Field, etc.) transport network utilization can run as high as 75% utilization. The 25% overhead bandwidth is to provide enough capacity to accommodate traffic peaks or primary to backup reroutes.

P&CSD/PID/CCNU shall be involved in future project network design and sizing to ensure required capacity is optimized to accommodate immediate and future demands. The following is a generic methodology for bandwidth calculations:

1. Bandwidth calculations per node shall utilize this formula:

Bandwidth/Node = Available bandwidth *.75/N,

where N is the total number of nodes deployed as part of the system and the Bandwidth/Node ≥ RTU aggregate bandwidth.

The remaining 25 % is reserved as a spare capacity for future expansion.

The RTU traffic aggregate is calculated as follows:

1. Identify the data rate and buffer depth based on the field devices that will be installed at the remote site (e.g., an oil well work area). This can be obtained from the vendor specs. An example is illustrated in the table listed below where the break down for each field device is outlined.


Page 54 of 63

Field Device Data Rate Peak RTU (Remote Terminal Unit) which supports SWC, PDHM, ESP, and a link to MPFM

409Kbps

CMS (Corrosion Monitoring Systems)

2.4Kbps

CP (Cathodic Protection) 2.4Kbps

MPFM ((Multiphase Flow Meter) 38.4Kbps

Voice over Data Phone Line 64Kbps

Total 516Kbps

Table Appendix 2.1: Traffic Mix per Each Oil Well Work Area

2. Identify the interface type for the field devices and map their aggregate to a network access node concentrator, which can be an Ethernet node, modem, radio modem, or SDH transport multiplexer.

3. The peak load is expected to occur when the communication link or the application server (s) are down, during which the traffic resident at the Oil well automation systems (RTU, MPFM, SWC, CMS, CP, Voice, etc.) buffer will be waiting on the communication or server to be back online to empty their buffer. In this example, the peak is estimated to be 516Kbps (Column #2 in the above table).

4. Network access node concentrator shall be connected to the backbone node with a minimum data rate of the total peak load.

Due to technology interface modularity (64Kbps, 128Kbps, 256Kbps, 512Kbps, etc).or vendor’s bandwidth provisioning system, the link speed should be rounded up to the next modulator. For example, if the total peak load is 516Kbps then use 768Kbps link. This will ensure minimize delay and provide a scalable solution that would accommodate future demands.

5. Multiple access node concentrators are typically connected to a backbone network node. Hence, network connection between backbone network nodes shall be based on a link speed that can provide at least 50% overhead capacity. (e.g., 50% utilization). This would accommodate traffic reroutes upon backbone node failure and provide scalability that would support future demands.

6. Figure, next page, depicts the dedicated 768 Kbps channel is provisioned from the OTN node serving the Concentrator switch to the OTN backbone node located at the CCR room. Therefore, the effective bandwidth allocation from the OTN node severing multiple oil well working areas to the OTN node at the CCR room is 768 Kbps.


Page 55 of 63

7. Virtual Local Area Network can be designed to build logical mapping of the different sub-services at the Concentrator node to the backbone connecting different Servers supported at the CCR room.

Figure Appendix 2.1: End-to-End Network Topology for a Remote Site

ESP

RTU=409Kbps/ 8M Buffer

Ethernet Access Switch

Ethernet Aggregator Switch

SDH/ OTN

SDH/ OTN

Ethernet Switch

Servers

PDHM

CMS=2.4Kbps/0

CP=2.4Kbps/ 2M

MPFM=38.4kbps/ 4M

SWC=19.4kbps/ 5M Buffer

Phone Line=64kbps/ 0 Buffer

Load=0.516M Capacity= 10Mbps Utilization=5.518% Single Mode Fiber

OTN allocated Capacity=768Kbps

OTN Ring

Effective Bandwidth Capacity=768Kbps

Concentrator Node

OTN/ SDH

SDH/ OTN

Backbone Nodes

Oil Well Work Area

Ethernet Switch Trunk Utilization at 50%

Backbone Node (can serve multiple access node concentrators)


Page 56 of 63

Appendix 3: Wireless Technology Comparison

Wireless Category Technology Distance Capacity Spectrum Comments

WPAN 802.15.4 Zigbee 10’s m 250 Kbps 900Mhz/

2.4Ghz Low bit rate for

sensors

WLAN 802.11a,b,g

Few meters ~15km

1Mbps - 108Mbps 2.4Ghz / 5GHz Industrial WLAN

WMAN 802.16 WiMax

1-20km+ radius 75Mbps 2-6Ghz

Fixed, Nomadic and Mobile Users (Non-

LOS)

WWAN GSM/Tetra 5-20km

144Kbps/Vehicle 384Kbps/ Ped’n 2Mbps

Fixed

850Mhz/ 900Mhz/ 1800Mhz/ 1900Mhz

Deployment worldwide

accelerating


Page 57 of 63

Appendix 4: Wireless Technology Classifications

WWAN (GSM/3G, TETRA, etc.)

Low throughput, short range

Low throughput, Long range

High throughput, short range WLAN (Wi-Fi)

WPAN (Zigbee)

WMAN (Wi-Max)

Bluetooth, RFID,ZigBee (802.15.4)


Page 58 of 63

Appendix 5: ISM Frequency Band 1 2 3 4 5 6 GHz

5.1cm 125MHz 5.725 - 5.850 GHz 5.7cm 300MHz 5.10 - 5.40 GHz 12cm 84MHz 2.40 - 2.484 GHz 33cm 26MHz 902 - 928MHz λ Bandwidth Spectrum


Page 59 of 63

Appendix 6: IEEE 802.11 WLAN Standards Model


Page 60 of 63

Appendix 7: Support Structure All Process Automation networks and systems shall comply with Saudi Aramco Process Control Engineering Standards; supported by Process & Control Department. Operational and maintenances’ requirements are supported by the Plants operations based on field survey. The table below outlines the different categories illustrating the responsibility matrix for some of the Northern and Southern Areas There are cases where the plant operations may sign up a service level agreement for the maintenance and operations of the communication network (e.g., Haradh). This does not preclude these areas from being standard compliant to Saudi Aramco Process Control Engineering Standards nor decouple the ultimate responsibilities of the plants operation for that area of responsibility. All Engineering requirements will continue to be supported by P&CSD.


Page 61 of 63

Glossary

Asynchronous Transfer Mode (ATM): ATM is very high speed communication transmission protocol technology. ATM is a high bandwidth, low-delay, connection-oriented, packet-like switching and multiplexing technique.

Backbone: A network configuration that connects various Networks / Systems together into an integrated network. In a Plant-wide network, that part of the network whose primary function is to forward data packets between the other smaller networks.

Bandwidth: In digital communications, describes the amount of data that can be transmitted over a channel in bits-per-seconds

Ethernet: A local-area network architecture based on IEEE 802.3. It uses a bus or star topology and supports data transfer rates of 10, 100, 1000, and 10,000 Mbps.

Firewall: A firewall is a set of related software / hardware, to protect the resources of a private network from users from other networks.

Gigabit Ethernet: Ethernet that operates at 1000 Megabits per second.

Industrial Ethernet Access Switch: Industrial Ethernet Access Switch is a switch used to connect remote sites to Control Room. These switches are typically required to tolerate outdoor environmental conditions (temperature, humidity, etc).

Ethernet Backbone Switch: Ethernet backbone switch connects multiple Industrial Ethernet Access switches to the Computer Control Room servers and workstations. These switches are typically installed indoor.

Process Automation Network (PAN): is a plant wide network interconnecting Process Control / Monitoring Networks and provides an interface to the WAN.

Process Control Network (PCN): A network that provides real time control communications.

Wide Area Network (WAN): an extension of LAN technology to include more nodes and greater distances between nodes; can be formed by interconnection of individual LANs.


Page 62 of 63

Glossary (Cont.)

Supervisory Control: A telemetry based Process Control command initiated from a Master Central Station over a bidirectional communications link using specific communication protocols to initiate an action and/or change analog set point in a remotely located Control Stations. Such commands follow timely bidirectional confirmation and acknowledgment executing sequences between the master and the station known as select/check before Operate.

Server: A server is a dedicated un-manned data provider.

Abbreviations:

CCTV - Closed Circuit Television

DCS - Distributed Control Systems

SCADA - Supervisory Control & Data Acquisition

TMS - Terminal Management System

VMS - Vibration Monitoring System

WAN - Wide Area Network


Page 63 of 63

Report Working Team

Lead Engineers: Soliman Almadi P&CSD/PID/CCNU

Abdullah Al-Khalifah P&CSD/PCD/PASU

Soliman Al-Walaie P&CSD/PID/CCNU

Acknowldegment

Kamal Banjar, P&CSD/PID/CCNU

Fouad Al-Khabbaz, P&CSD/PID/CCNU

Khalid AL-Ghamdi , P&CSD/PID/CCNU

Zakarya Abualsaud, P&CSD/PID/CCNU

Saad AL-Harbi, P&CSD/PID/CCNU

Salem AL-Elwi, P&CSD/PID/CCNU

Hussain AL-Salem, P&CSD/PID/CCNU

Nasser Y. Assiry, P&CSD/PCD/PASU

Ghalib AL-Hashim, P&CSD/PCD/PASU

Saleh A. AL-Qaffas, P&CSD/PCD/PASU

Ralph Hartman, P&CSD/PID/IU

Mohammed Subaii, EW Pipleline Engineering

Mohammad Shenqiti, SAEPED

Daniel Foglesong, FPD

Ibrahim AL-Hashem, PMT

John F. Grainger, EW Pipleline Engineering

Ali H. AL-Ahmad , OSPAS

Ali Sufair, IT Communication Engineering

Hadi Z. AL-Shammari, IT Communication Engineering

Adel A. AL-Fayez, IT Communication Engineering

Osama A. Alaidarous, QATIF Enginerring

George J. Hirezi, NAPED/RTTSU

Documents

SAER-6114