Assessing the Effectiveness of Attack Detection at aHackfest on Industrial Control Systems

Sridhar Adepu and Aditya MathuriTrust Center for Research in Cyber Security

Singapore University of Technology and DesignSingapore

Email adepu sridharmymailsutdedusg aditya mathursutdedusg

AbstractmdashA hackfest named SWaT Security Showdown (S3)has been organized consecutively for two years S3 has enabledresearchers and practitioners to assess the effectiveness of meth-ods and products aimed at detecting cyber attacks launchedin real-time on an operational water treatment plant namelySecure Water Treatment (SWaT) In S3 independent attack teamsdesign and launch attacks on SWaT while defence teams protectthe plant passively and raise alarms upon attack detectionAttack teams are scored according to how successful they arein performing attacks based on specific intents while the defenseteams are scored based on the effectiveness of their methods todetect the attacks This paper focuses on the first two instances ofS3 and summarizes the benefits of hackfest and the performanceof an attack detection mechanism named Water Defense thatwas exposed to attackers during S3

Index TermsmdashAttack Detection Capture-The-Flag (CTF)Cyber-physical Attacks Cyber-Physical Systems Cyber SecurityIndustrial Control Systems Hackfest Water Defense WaterTreatment Plant

I INTRODUCTION

Industrial Control Systems (ICS) [39] considered in thiswork are complex interconnected systems deployed to controland monitor among others critical infrastructures such aswater treatment and electric power systems [17] The increasein successful cyber attacks on public infrastructure [14] [29][48] and other mostly unsuccessful attempts [25] has raisedthe importance of deploying cyber defense mechanisms inICS Attackers are often bypassing the defense mechanisms(prevention and detection) by exploiting software and hard-ware vulnerabilities or through social engineering Thereforeit becomes important to look for ways of detecting processanomalies in an ICS caused by an attacker who has gainedunauthorized entry Water Defense [5] referred to in this paperas WD is one such anomaly detection mechanism

The technology underlying WD has been described andexperimentally evaluated by the authors [1] [4] This pa-per describes an independent supplementary assessment ofWD based on attacks launched by teams of attackers Theassessment was carried out over two consecutive years inan event labeled ldquohackfest (S3)rdquo [9] [35] [36] The eventwas organized by a team of faculty students and staff atthe Singapore University of Technology and Design Several

This work was supported by research grant 9013102373 from the Ministryof Defense and NRF2014-NCR-NCR001-040 from the National ResearchFoundation Singapore

attack and defense teams [9] [35] [36] participated in S3The attack teams designed and launched attacks on SWaTndashthe system used for assessment of WD by the authors Thispaper is focused on analysis of data used for assessing theperformance of WD in detecting process anomalies resultingfrom the attacks Details of S3 are in Section IV

The following two research questions were the focus of S3

with respect to WD

RQ1 How do attackers compromise the security of an ICS

RQ2 How effective is WD in detecting attacks launched byindependent attack teams

Contributions This paper (a) summarizes the performanceof the WD mechanism during two consecutive S3 eventsand (b) reports on observations and lessons learned fromthe experience all attacks launched during S3 are reportedInformation presented in this work will likely be valuable toresearchers attempting to assess the performance of processanomaly detection methods other than WD A complete listof the invariants (see Section III) used in WD and the S3

dataset are also available for download [41]

Organization The remainder of this paper is organized asfollows Section II contains material to aid in understanding theremainder of this paper An overview of the WD is presented inSection III Section IV contains a description of S3 Preparationfor the event is described in Section V Attacks performed byattacker teams during both S3 events are in Section VI Resultsfrom the events are presented in Section VII Section VIIIreturns to the research questions mentioned above and answersthem with respect to the data generated during S3 Section IXis a summary of the work Conclusions based on S3 aresummarized in Section X

II PRELIMINARIES AND BACKGROUND

This section is a brief introduction to Industrial ControlSystems (ICS) [39] in the context of SWaT An exampleillustrates the nature of a cyber attack on SWaT and itspotential impact on system response

A Industrial Control Systems

An ICS [39] consists of physical control and networkdevices (Figure 1) Physical devices include sensors and actua-

arX

iv1

809

0478

6v1

[cs

CR

] 1

3 Se

p 20

18

tors Control devices include Programmable Logic Controllers(PLCs) Supervisory Control and Data Acquisition (SCADA)workstations and Human-Machine-Interface (HMI) devicesNetwork devices include network switches and access pointsThe PLCs SCADA and HMI monitor and control the physicalprocess Communication channels in the network act as abridge between the physical process and the control devicesCommunication channels communicate the state of physicalprocess to controllers and control signals to the actuatorsPLCs receive data from sensors compute control actions andapply these actions to specific devices The PLCs in an ICScan be viewed collectively as a distributed control system thattransforms the state of the process through the use of sensorsand actuators

Fig 1 High level view of an ICS

B SWaT Architecture and components

SWaT [30] [40] is a testbed for water treatment It is usedto investigate the response to cyber attacks and experimentwith novel designs of defense mechanisms such as the onesdescribed in [3] The architecture and components in SWaTare described next

Stages in SWaT As shown in Figure 2 SWaT consists ofsix stages labeled Stage 1 through Stage 6 Each stage iscontrolled by its own set of PLCs Stage 1 controls the inflowof water to be treated by opening or closing a valve thatconnects the inlet pipe to the raw water tank (T101) Waterfrom the raw water tank is pumped via a chemical dosing(Stage 2) station to another Ultrafiltration (UF) feed watertank (T301) in Stage 3 In Stage 3 a UF feed pump sendswater via the UF unit to a Reverse Osmosis (RO) feed watertank (T401) in Stage 4 In Stage 4 an RO feed pump sendswater through an ultraviolet dechlorination unit controlled bya PLC Differential pressure sensors in Stage 3 measure thepressure drop across the UF unit A backwash cycle is initiatedwhen the pressure drop exceeds 04 bar indicating that themembranes need immediate cleaning Stage 5 contains a PLCto control the Reverse Osmosis (RO) unit that further filtersthe water using a 2-stage RO process The output of the ROunit enters storage tanks T601 and T602 in Stage 6 Tank T601contains the reject from RO and is used to clean the UF unitusing a backwash process Water in tank T602 contains thepermeate which is recycled into tank T101 in Stage 1

Sensors and actuators SWaT contains 42 sensors and actu-ators across the six stages These include sensors that relateto the dynamics of the process such as water level in tanksflow indicators and pressure indicators as well as thosefor measuring chemical properties of water including pHconductivity and hardness Each PLC has its own set of sensorsand actuators connected through a ring network Thus whena PLC needs to obtain state information from another PLCit must request such information via a suitable commandthe requested data is sent over level 1 network as shown inFigure 3

Communications Figure 3 shows the architecture of thecommunications infrastructure in SWaT Each PLC obtainsdata from sensors associated with the corresponding stageand controls pumps and valves in its domain Flow of wateracross various stages is controlled through opening and closingof valves and turning pumps ON or OFF Level sensors ineach tank enable the PLCs to decide when to turn a pumpON or OFF Several other sensors are available to check onthe physical and chemical properties of water flowing throughthe six stages PLCs communicate with each other through aseparate network Communications among sensors actuatorsand PLCs can be via either wired or wireless links controlledmanually Both wired and wireless networks connect PLCsto the physical process and to the HMI and engineeringworkstation ie the SCADA workstation

The control network is connected to the SCADA worksta-tion [40] through a wired network using a 16-port switch ThePLCs and SCADA workstation are also connected througha wireless network configured using star topology Switcheslocated at each stage enable the use of either wired or wirelesscommunications The communications network is layered intotwo levels For each PLC level 0 refers to the communicationlayer between sensors actuators and the PLC Level 0 net-work is implemented as a ldquodevice level ringrdquo [9] [43] whichincludes a Remote IO (RIO) device The RIO is connectedto the physical sensors and actuators Monitoring and controlinformation is exchanged between the PLC sensors andactuators across a Distributed Logical Router (DLR) Level 1refers to the communication layer among PLCs This layeris implemented in a star topology and includes a SCADAworkstation an HMI and a Historian

Controllers (PLCs) SWaT is equipped with Allen BradleyControlLogix PLCs Therefore some of the attacks describedin the remainder of this paper require consideration of theprotocols used by EtherNetIP [32] for Allen Bradley PLCsSCADA software is developed with tools from Rockwellautomation [8]

Attack detection mechanisms As shown in Figure 3 SWaTcontains four attack detection mechanisms D1 D2 D3 andOrthogonal Defense Mechanism (ODM) [6] D1 is based ona modified version of the open source Bro intrusion detectiontool [10] D2 is a set of three commercially available intrusiondetection systems that use a mix of process dynamics and

Raw Waterinput UltraltrationChemical

dosing Dechlorination

PLC1

Reverse Osmosis

PLC5PLC4PLC3PLC2

Backwash

PLC6

S=LIT101 FIT101

A=MV101 P101

S=AIT202

A=MV201

S=LIT301 FIT301 DPIT301

A=MV201 MV302 P301

S=LIT401 FIT401

A= P401

S=FIT501 FIT502 AIT501

A=MV501 P501

T101 T301Storage tanks T401[No storage]

FromT501From

T502

MV201P101MV101 FIT101

P6

Water fromexternalsource

DPIT301LIT301FIT301P301

MV302P302

Stage 1 Stage 2 Stage 3 Stage 4 Stage 5

Stage 6T601 T602

Fig 2 Six stages in SWaT with corresponding PLCs sensors and actuators Five water storage tanks as shown are labeled Txxx Water level in each tankis measured by the corresponding level indicator labeled as LITxxx AITxxx FITxxx and DPITxxx measure respectively chemical properties of water flowrate in a pipe and differential pressure across the ultrafiltration unit Pxxx denote pumps at various stages

Historian SCADASwitch

PLC 2

AS

PLC 6

AS

PLC 1

AS

HMI

Externaldevices

D1 D1 D1

D2

D3 D3D3

ODM ODMODM

Level 0

Level 1

Level 2

S SensorA ActuatorODM Orthogonal Defense Mechanism

Fig 3 Communications structure of SWaT D1 D2 and D3 denote setsof defense mechanisms in SWaT ODM (Orthogonal Defense Mechanism) isindependent of these mechanisms

other techniques for anomalous process behavior [12] [24][27] D3 (WD) sits inside PLCs and implements a distributedattack detection mechanism that relies exclusively on processdynamics [1] The ODM has direct access to sensors and anal-yses the data received for the existence of process anomaly

C An illustrative attack on SWaT

Consider Stage 1 of SWaT in Figure 2 This stage has amotorized valve labeled MV101 which when open causeswater to flow into tank T101 The inflow into T101 is measuredby flow meter FIT101 and the water level by a level sensorlabeled LIT101 Pump P101 sends water to the next stageFlow meter FIT201 measures the outflow of water fromStage 1 to Stage 3 PLC1 receives the LIT101 reading and

controls the motorized valve MV101 Similarly PLC1 receivesLIT301 readings from PLC3 and controls pump P101

Tanks T101 and T301 have four markers each labeled Low(L) Low Low (LL) High (H) and High High (HH) Eachmarker corresponds to a specific value of water level in thetank These markers are used by the corresponding PLCs tocontrol the states of motorized valves and pumps Thus forexample when the water level in T101 reaches L PLC1 opensMV101 and closes it when the level reaches H When waterlevel in T301 reaches L PLC1 turns P101 ON and turns it OFFwhen the level reaches H The following example illustrates theimpact of compromising level sensor LIT101 with the intentof damaging pump P101

Example Consider an attack where the attackerrsquos intentionis to underflow T101 and damage P101 by making it runwithout any incoming water The attack is launched on LIT101with Stage 1 in the following state LIT301 955mm MV101Closed P101 OFF UF is operational and therefore waterlevel in tank T301 is decreasing Assume now that the attackersets LIT101 reading to a constant value of 790mm In thisattack even though the water level in T101 is changing(decreasing) PLC1 receives a constant value After a whilewhen LIT301 reaches L pump P101 is turned ON by PLC1However the actual water level in tank T101 is lower thanL say at LL This leads to the outflow from the pump beingreduced to less than the intended flow rate Pump P101 runsdry when there is no water in T101 and will eventually getdamaged unless a corrective action is taken

Figure 4 shows the water level in tank T101 during theattack It can be observed that the outflow increases graduallywhen the attack is removed Note that the sudden drop in thevalue of LIT101 soon after attack removal corresponds to thefact that the PLC begins to receive the correct measurement

Fig 4 Water level in tank T101 when LIT101 is attacked LIT101readings are observed by PLC1

Fig 5 Level sensor LIT101 is under attack The attackerrsquos intention isto underflow T101 tank and damage P101 The first arrow indicatesthe outflow reducing time second arrow indicates the pump noisestarting time

of water level in T101 When the water level goes down to150 mm tank T101 does not have enough water to send totank T301 Figure 5 shows the change in flow rate duringthe attack as measured by flow meter FIT201 The two arrowsindicate the start of reduction of outflow from T101 At around10 seconds there is no water flowing from P101 even thoughthe pump is ON At this point the pump becomes noisy and theflow rate reduces to zero If not removed this attack may leadto pump damage due to overheating Of course a mechanicalcut off at the pump would avoid such damage

The above example shows how an attacker could potentiallydamage a pump by changing the sensor values and actuatorstates More complex attacks mentioned in Section VI can bedesigned and launched to reduce the chances of being detected

III OVERVIEW OF WDWD is a mechanism to detect process anomalies A process

is considered anomalous when it deviates from its expectedbehavior WD detects such anomalies through the use of in-variants An invariant [4] is a condition among physical andorchemical properties of the process that must hold wheneveran ICS is in a given state At a given time instant sensormeasurements of a suitable set of such properties constitutethe observable state of the physical process as known to theICS

The invariants serve as checkers of the system state Theseare coded and the code placed inside each PLC used for attack

detection The checker code is added to the control code thatalready exists in each PLC The PLC executes the code in acyclic manner In each cycle data from the sensors is obtainedcontrol actions computed and applied when necessary and theinvariants checked against the state variables or otherwise Dis-tributing the attack detection code among various controllersadds to the scalability of the proposed method During S3 theimplementation was located inside the Programmable LogicControllers (PLCs) as well as embedded in the communicationnetwork

Two types of invariants were considered state dependent(SD) and state agnostic (SA) While both types use statesto define relationships that must hold the SA invariants areindependent of any state based guard while SD invariants areAn SD invariant is true when the plant is in a given state anSA invariant is always true

A State-Dependent (SD) invariants

Consider for example the case when the motorized valveMV101 is Open In this case the flow rate indicator FIT101must provide a non-zero reading to the PLC This phys-ical fact leads to the following state-dependent invariantMV101=Open =rArr FIT101lt δ where δ denotes a thresholdindicating flow Note that an SD invariant may include con-ditions from across the various stages of SWaT thus enablingdistributed detection of attacks Derivation of SD invariants isbased on the design of the ICS and is described in [4]

B State-Agnostic (SA) invariants

Under normal system operation an SA invariant mustalways be true regardless of the system state One SA invariantwas derived for each tank in SWaT to detect attacks that affectthe flow of water into and out of a tank These invariants arebased on the flow of water and water level in a tank andhence are identical in terms of the mathematical relationshipthat they capture

As an example of an SA invariant consider the water levelin a tank At time instant k+1 the water level in T101 dependson the level at time k and the inflow and outflow at instant kThis relationship is captured in the following idealized discretetime model of the tank

x(k + 1) = x(k) + α(ui(k)minus uo(k)) (1)

where ui(k) and uo(k) denote the inflow and outflow ratesat time k and α is a proportionality constant that convertsflow rate to change in level using the tank dimensions x(k)is the true state of the water level Let y(k) denote the sensormeasurement of the water level x(k) an estimate of the levelsensor reading and ε a threshold based on experimentationBased on Eqn 1 the statistics obtained experimentally andconverting the true states to their estimates the followinginvariant is derived to test whether or not the tank fillingprocess is anomalous

sumn

i=1|(x(i)minus y(i))|n

gt ε under attack (2)

le ε normal (3)

Fig 6 Invariant to detect anomalous behavior of LIT 101

IV SWAT SECURITY SHOWDOWN (S3)

This section presents details of the two S3 [36] eventsincluding guidelines and selected information on participantsIn S3 the attackers are challenged to realise concrete goals inSWaT Points earned by an attack team are weighted based onthe capabilities needed to launch the attack and the number ofdefence mechanisms successfully bypassed during the attackThe goal was to meet as many pre-defined challenges aspossible within the pre-allocated time

Information disclosed to the attack teams Technical detailson SWaT such as network architecture protocols and devicesused are released to the attackers one month prior to theirarrival for participation in the event Publicly available whitepapers on mechanisms deployed by the defence teams areshared with each attack team

Information disclosed to the defenders S3 organizers workedclosely with the defense teams to integrate their defence mech-anisms into SWaT Information about the normal operation ofSWaT was disclosed to the defenders to enable them to fine-tune their detection systems and reduce false alarms as muchas they could

Attacker profiles Attack teams were asked to select from aset of attacker profiles [34] The following attacker profileswere available cyber-criminal insider or a combination ofboth An attacker profile is intended to restrict availability ofresources and limit the access rights of the attackers as shownin Table I

A S3-2016

Attack teams included three from industry and three fromacademia Similarly there were three defense teams from theindustry and three from academia During the live phase heldat the SWaT testbed all six [35] defence mechanisms weresimultaneously in place Each team was given 12 hours forpassive reconnaissance and team was assigned a 3-hour slotduring which they were able to launch attacks

B S3-2017

Attack teams included one from industry and four fromacademia There were two defense teams from the industryand two from academia Each attack team was given two ses-sions [36] of four hours each to conduct reconnaissance on thetestbeds During these sessions various attacks were preparedand tested with the assistance of the SWaT laboratory engineerDuring the actual event each team was given two hours todemonstrate their attacks that were prepared previously Attack

TABLE IRESOURCES AND ACCESS RIGHTS FOR ATTACKER PROFILES

Profile Constraints

Cyber-criminal Limited number of attempts to realize a goal

Physical access not allowed manual manipulation ofthe sensors and actuators are not allowed

Direct connection to PLCs using any software suchas Allen Bradleyrsquos Studio5000 not allowed

Insider Physical access to SWaT allowed manual manipula-tion of the sensors and actuators are allowed

Allowed to alter the network topology

Direct connection to PLCs using any software suchas Allen Bradleyrsquos Studio5000 allowed

TABLE IITARGETS OF ATTACKS IN S3

Target Description

Physical Process Attacks

Valves Control the motorized valves

Pumps Disrupt pump control operations

Pressure Alter the pressure in pipes

Tank fill level Alter water level in a tank

Chemical dosing Alter chemical dosing

Sensor Data Attacks

Historian Alter data in the Historian

HMISCADA Alter the sensor actuator values at HMI orSCADA DoS Attacks on SCADA HMI

PLC Reprogram PLC DoS attacks on PLCsChange the commands and values in which thePLC receives and sends

RIODisplay Control of the RIO through disconnected ana-logue InputOutput pin

teams were also given a separate network for Internet up-linkand up to three Virtual Machines (VMs) running either Linuxor Windows operating system

C Attack targets

The attack teams were given a list of components andsubsystems in SWaT that could serve as the target of theirattacks Table II lists the targets available to the attack teamsTable II has two kinds of attacks physical process attacks andsensor data attacks In physical process attacks an attackerrsquosobjective is to alter the physical process In the case of sensordata attacks an attackerrsquos objective is to alter the sensor oractuator tags during communication or in the Historian

V PREPARATION FOR S3

To prepare for S3-2016 an earlier version of WD wasextended to all six stages of SWaT This extension requiredthe generation of invariants across all stages coding of theinvariants and placement of the code inside [1] the six PLCsThe modified WD was tested on SWaT by running the plantunder various operating conditions

Based on lessons learned during S3-2016 several newinvariants were generated coded and added to the PLCs ForS3-2017 we decided to use an additional monitoring systemplaced outside the PLCs This system collects data from theHistorian and evaluates the invariants All invariants wereimplemented in a Linux environment using a Piwebclient APIto talk to the Historian This new implementation is referredto as WDH

The invariants in WD are coded using ladder logic andstructured text while those in WDH in Python Both imple-mentations use the same set of invariants the difference is intheir placement The Historian may not get all the data andcommands that flow across the PLCs sensors and actuatorsHowever as WDH gets its data directly from the Historian ithas access to information flowing across SCADA workstationand the Historian This information may be compromised byan attacker and is not available to the PLC

A Scope of WD

WD is designed to detect process anomalies Thus anyabnormal behavior in the water treatment process in SWaTought to be detected by WD However there could be attacksthat do not cause the process to deviate from its normalbehavior but lead to undesirable consequences An exampleof such an attack is one intended to deface the screen on theSCADA workstation or the HMI Such an attack will not bedetected by WD Attacks that may cause process anomaly butonly after an attack has been removed from the system mayalso not be detected by WD Denial of Service is one suchattack

B Scope of WDH

WDH and WD use the same set of invariants Howeverthe placement of WDH could lead to a difference in detectioncapabilities of the two defense mechanisms WDH gets its datafrom Historian while WD directly from the PLC Data that isnot programmed to be logged in the Historian will not beaccessible to WDH Thus any anomaly that requires such datawill likely not be detected by WDH Similarly attacks thatmanipulate data entering the Historian or SCADA may not bevisible to WD Thus while the two invariant-based processanomaly detection mechanisms are identical in the invariantsthey use their placement in SWaT is expected to result indifferent performance in detecting attacks

VI S3 ATTACKS

The attacks launched by teams participating in the two S3

events are described next

A S3-2016 Attacks

All attacks designed and launched during S3-2016 areenumerated in Table III Three attacks selected from Table IIIare described next Details of all attacks are available in [9]Of the 18 attacks in Table III 4 and 16 are cyber criminalattacks and the remaining are insider attacks

DoS attack on SCADA In this attack (attack 4 in Table III)the attackerrsquos intention was to deface the SCADA workstationscreen and hence prevent the operator from observing plantstate The cyber-criminal attacker model was used to designthis attack To realize the intention the attacker launched anARP poisoning Man-in-the-Middle attack in two steps In thefirst step all traffic intended for HMI was redirected to theSCADA workstation In the second step this redirected trafficwas dropped and thus no packets were received at the SCADAworkstation This led to the screen on the workstation becom-ing completely gray and no state information was displayedThis attack was not detected by WD as it did not lead toany process anomaly It is an ARP spoofing attack and not atraditional DoS attack As part of the DoS attack the attackertargeted the PLC and sent millions of packets at a time Thisled to the same effect as would be the case when an ARPspoofing attack is performed on SCADA

Manipulation of the chemical dosing pump Intention of theattacker in this case (attack 14 in Table III) was to manipulatethe pH of water entering Stage 3 of SWaT The insider-attackermodel was used in the design of this attack This attack wasexecuted in two steps In the first step PLC 2 was set to manualmode Note that in manual mode the plant operator can directlycontrol the actuators eg the dosing pumps in this case In thesecond step the attacker altered the chemical dosing processin the Pre-treatment Stage 2 of SWaT by interacting directlywith the HMI interface and overriding the commands sent bythe PLC WD was able to detect this attack because the set-points changed by the attacker were different from those setin WD

DoS to PLC by SYN flooding The intention of the attackerin this case (attack 16 in Table III) was to disable the HMIso that an operator is unable to view or control the plantoperation The insider-attacker model was used in the designof this attack In this way the attacker had an access to theadministrator account and the associated tools The attackerperformed a SYN flooding attack on EthernetIP server ofPLC1

As a result of this DoS attack the HMI was unable toobtain the current state values to display and would insteaddisplay 0 or characters WD was unable to detect this attackphysical process as not affected During the attack period PLCwas controlling the process as expected Such attacks whilenot altering process behavior may impede supervision of theprocess in an operational plant

B S3-2017 Attacks

All attacks designed and launched during S3-2017 areenumerated in Table IV Selected attacks from Table III aredescribed next Details of all attacks are available in [20] Ofthe 31 attacks in Table IV 17 can be classified as cybercriminal attacks and the remaining as insider attacks (Figure I)All attacks launched during S3-2016 and S3-2017 are listedand categorized in Table V

TABLE IIIATTACKS LAUNCHED DURING S3-2016

SNO Target Method Attack Tool

1 Tank fill levelLIT101

Use HMI access Close MV101 and Stop P101 andP102

HMI

2 HMISCADA ARP spoofing Attack HMI DoS attack Ettercap

3 PLC Manual access Removed the cable at the ring atlevel 0

Manual

4 HMISCADA DoS on HMI by droppingall packets between PLC andSCADAHMI

DoS attack on SCADA wide DoSattack took a while to restore SWaTto its normal state

Ettercap

5 Tank fill levelLIT101

Use HMI access Attack on LIT101 ManualHMI

6 Valve MV301 Use SCADA access Attack on MV301 manually openfrom the SCAD workstation

ManualSCADA

7 Pump P101 Use SCADA access Attack pump manually open it fromthe SCADA workstation

ManualSCADA

8 Historian DoS attack using CPPPO andloop

Attack between HMI and PLC CPPPO

9 Valve MV101 Use SCADA access MV101 attacked using SCADAchanged the valve state from Opento Closed

ManualSCADA

10 Pump P101 Use SCADA access LIT301 set point changed ManualSCADA

11 Tank fill levelLIT301

Using SCADA access LIT301 set point altered ManualSCADA

12 Chemical dosingP201

Control MV101 and AIT503 setpoints of LIT301 to ensure flowthis triggered chemical dosing

Dosing pump attack on P201 ManualSCADA

13 HMISCADALIT101

Functional block introduce newconstant tag tie that to output tagcould only do zero

LIT101 set to zero from PLC Studio5000

14 Chemical dosingpump P205

Use SCADA access Manipulation of the chemical dosingpump (P205)

ManualSCADA

15 HMISCADA DoS on HMI using Level 1 net-work

Attack on HMI EttercapPycomm

16 Historian SYN flood ENIP port at PLC1 DoS to PLC by SYN flooding (attackon HMI)

Ettercap

17 Chemical dosingpump P203

HMI-based direct manipulation Attack on P203 while the four dosingpumps are running

ManualHMI

18 HMISCADALIT101

Re-program PLC to fix LIT101value to an arbitrary value

Attack on LIT101 Studio5000

416 are cyber criminal attacks in S3-2016

Control of the chemical dosing system through a Pythonscript (Pycomm) The objective of this attack (attack 15 inTable IV) was to change chemical dosing at the end of the de-chlorination system (Stage 4) First the attackers compromisedVirtual Network Computing (VNC) Then they used a Pythonscript (Pycomm) and Wireshark to gain access to the HMIAfter gaining access to the HMI through the compromisedVNC the cybercriminal attacker used Wireshark to capturethe packets flowing between the HMI and PLC4 The con-troller tags were retrieved by an analysis of the packets Theattackers changed the data associated with these tags to controlthe chemical dosing function using the Pycomm framework

Control of PLC through the Bridged Man-in-the-Middle(MiTM) at Level 0 the objective of this attack was (attack 16in Table IV) to change the commands and values that PLC1receives and sends First the attackers configured a bridgebetween the RIO and PLC1 using Netfilterqueue andScapy The attack was launched at two network levels Ananalysis on the network traffic revealed the packets that theattackers should edit As the target of this attack was thewater level in T101 the attackers set it to a constant valueto hide from PLC1 the rise in water level in T101 Before apacket was forwarded Netfilterqueue rerouted it into aqueue which can be read and modified by the Python script

TABLE IVATTACKS LAUNCHED DURING S3-2017

SNo Target Method Attack Tool

1 HMISCADA LIT401 HMI simulation insider attack Change the value of LIT401 in the HMI Manual HMI

2 Historian ARP and drop Change the value stored at the Historian Ettercap

3 Valve MV201 Reprogram PLC Change the status of the MV201 Studio 5000

4 Tank fill level LIT301 420to 320

Manual Lower the water tank level from 820mm to420mm without raising any alarm LIT301decreased till 320mm

Manual HMI

5 Pump P101 Manual mode of pump Alternate the state [OnOff] of the pump P101 Manual HMI

6 Chemical dosing P205 Manually dosing chemical pump Change the chemical dosage of sodiumhypochlorite (NaOCl) in P2

Manual SCADA

7 PLC Disconnect cable Disrupt sensor values from remote inputoutput(RIO) to the PLC

Manual

8 RIO Display Disconnect IO PIN manual Disrupt the sensor reading send to PLC throughRemote IO (RIO)

Manual

9 Chemical dosing P404 MiTM Python script to control Increase chemical dosage in pre-treatment Python script

10 LIT101 (476mm to 540mm ) Reprogram PLC Falsify water level display at SCADA Studio 5000

11 Pump P101 HMI simulation insider attack Alternate the state [OnOff] of the pump P101 Manual HMI

12 HMISCADA AIT 504 ARP+rewriting Increase AIT504 Ettercap

13 PLC LIT401 Reprogram PLC Falsify water level display at SCADA Studio 5000

14 RIODisplay Disconnect specific IO PIN basedon manual

Disrupt the sensor reading send to PLC throughremote IO (RIO)

Manual

15 Chemical dosing pumpP403 AIT501

Based on captured traffic betweenHMI and PLC4

Change chemical dosing function VNC Python script PycommWireshark

16 PLC LT101 from 742mm to500mm

Level 0 MITM Change the commands and values that the PLCreceives and sends

Aircrack Airodump AireplayNetfilterqueue Scapy

17 Historian LT101 tag Aircrack WiFi ARP spoofingEttercap

Compromise historian data Ettercap Aircrack

18 Pressure sensor DPIT30130MV301-4

SMB to EW get project files runFT

Disrupt valves operation of Ultrafiltration andBackwash (P3)

SMB

19 MV201 LT101 metasploit+vnc Change the water level of the tank LIT101 Metasploit+vnc

20 Pump P501 Rogue AP disassociated Telnetwith default credentials to turnoff original AP Scapy rewrite

Disrupt pump control operation KisMAC Password crackingtool 3vilTwinAttacker TelnetScapy

21 PLC LIT101 Reprogram PLC Change level indicator value Studio 5000

22 Pump P101 LIT301 Using back-door connection Establish back-door connection Mimikatz malicious VBAMacro SOCKS proxy

23 HMISCADA P201 Netfilterqueue Scapy Change the display value of the HMI Netfilterqueue Scapy

24 Historian LIT101 Overwrote specific data stored at the Historian Microsoft PsExec ipconfig

25 RIODisplay Manual Control of the RIO through disconnected Ana-logue InputOutput pin

Manual

26 Valve MV201 Manual Permanently closed the motorised valve re-gardless of commands issued

Manual SCADA

27 RIODisplay AIT202203 Manual Change the pH value shown at HMI Manual HMI

28 PLC MV201 P101 ARP poisoning MiTM Increase the pressure at P1 Ettercap

29 Tank fill level LIT101 Lower Falsify the water level reading of the tankdisplayed at SCADA

Pycomm

30 Chemical dosing PLC2 Use Studio 5000 Change the level of the chemical used fordosing

Studio 5000

31 Pressure MV302 P3012 Using Pycomm script Change the pump state sent to the PLC Pycomm

TABLE VCYBER CRIMINAL ATTACKS IN S3

Cyber Criminal Attacks Insider Attacks

S3-2016 4 16 1 2 3 5 6 7 8 910 11 12 13 14 1517 18

S3-2017 2 9 10 12 13 1516 17 18 19 20 2122 24 28 29 30

1 3 4 5 6 7 8 1114 23 25 26 27 31

To prevent all packets from entering the queue in order notto disrupt other processes iptables was used to identifythe targeted packets entering the queue Using Scapy and acustom dissector the attacker edited the payload of the targetedpacket which was then forwarded to its original destination

Control of Historian through the Aircrack WiFi The objectiveof this attack was (attack 17 in Table IV) to compromise thedata stored in the Historian Attackers performed crack WiFipassword ARP poisoning and MiTM payload manipulationusing Aircrack and Ettercap As PLC1 was operating inthe wireless mode the cybercriminal attacker used Aircrackto obtain the password for connecting to the ICS Access Point(AP) ARP poisoning was executed to reroute traffic betweenPLC1 and the Historian through the attackerrsquos rogue terminalThe attackers then used an Ettercap filter to manipulate thenetwork packets The attackers changed the tag correspondingto LIT101 to an arbitrary value before releasing the packetsto the Historian

Control of pressure through the Server Message Block (SMB)The objective of this attacks was (attack 18 in Table IV) todisrupt the state of four motorized valves in Stage 3 to affectthe differential pressure in UF Vulnerability CVE-2008-21601

in Factory Talk software from Rockwell and in MicrosoftrsquosServer Message Block (SMB) was used by the attackersto obtain files from the HMI As the HMI was runningWindows CE it has a vulnerability that allows an attackerrsquosterminal to execute arbitrary code on the HMI Thus theattackers were able to retrieve the files to create a copy ofthe workstation From the copied workstation the attackersmanually changed the state of the valves in Stage 3 suchthat the differential pressure across the UF unit as measuredby DPIT301 became dangerously high The attackers closedvalves MV301 MV302 and MV303 and opened MV304

Control of water level in the tank through the MetasploitVNC Scanner Objective of this attack was (attack 19 inTable IV) to change the water level in tank T101 The attackersused Metasploit VNC authentication None scanner to ob-tain access to the VNC server without password protection andto check for nodes running a VNC Server Once the scannerdetected the VNC Server running without any authenticationthe attackers penetrated into the server through a VNC Clientconnection As the VNC Server was hosting the HMI which

1httpswwwcvedetailscomcveCVE-2008-2160

controlled the ICS the attackers changed the simulation tagassociated with water level in T101

Control of a pump through a rogue router The objective ofthis attack (attack 20 in Table IV) was to disrupt the controlof pump P501 The attackers used Evil twin (rogue accesspoint) method using KisMAC a password cracking tool3vilTwinAttacker Telnet and Scapy The attackersused KisMAC to scan for wireless networks in the ICS Oncethe targeted wireless network was identified the attackers useddictionary attack to crack the password After the passwordwas cracked the attackers created a rogue wireless routerwith a similar SSID and configuration They then sent a de-authentication packet to disassociate PLC5 and the originalrouter The attackers used Telnet to log into the originalrouter and shut it down Scapy was then used to modify thepackets to turn the pump on

VII RESULTS

Tables VI and VII summarize the response of WD andWDH to the attacks launched during the two S3 events Recallthat both WD and WDH contain exactly the same set ofinvariants In WD the invariants are coded and placed insidethe PLCs whereas in WDH the invariants are coded and placedat the Historian WDH did not exist during S3-2016 and hencethe response of WDH is available only for attacks launchedduring S3-2017

A S3-2016 results

We note from Table VI that 10 out of 18 attacks weredetected immediately while the remaining eight attacks werenot detected Six of the eight undetected attacks did not leadto process anomaly during the observation period and hencedid not violate any invariant This outcome is expected as theinvariants in WD are designed to detect process anomaly

Consider attack 2 ARP spoofing in Table III This is aDoS attack on HMI It leads to defacing the screen on theHMI or displaying incorrect information thereby preventingan operator from knowing the actual plant state Howeverthe attack does not cause process anomaly and hence is notdetected as it does not violate any invariant Similar logic canbe used to explain why the other attacks in Table VI are notdetected

It is important to note that a DoS attack when given enoughtime to evolve and be launched at an appropriate state of theplant may impact physical process behavior In such a caseone or more invariants may detect the attack One such attackis 16 in Table VI This attack prevented the Historian fromreceiving data from PLC1 However if this attack was leftactive for a longer period it would prevent PLC1 from sendingappropriate commands to the actuators eg to MV101 orP101 In turn this would have led to process anomaly Notenough data is available to conclude with certainty whether ornot this attack would be detected by WD if active for sufficienttime

Two single point [2] attacks were not detected by WD Inone attack (attack 6 in Table III) the adversary altered the status

of valve MV301 Under normal circumstances this valve isopened during the backwash process However the attackeropened it when there was no backwash Hence the attackdid not affect the physical process except in changing thevalve status No invariant was violated due to this attackbecause the backwash process ie Stage 6 is not includedin this case study The second single point attack (attack 17 inTable III) was performed on chemical dosing pump P203 whilethe other pump P204 was running Note that under normalcircumstances only one of these two pumps is supposed to berunning while the other remains as a backup Subsequently theattacker shut down pump P204 This attack was not detectedbecause there were no invariants that related to the chemicalproperties of water

Although the overall performance of WD was below 100it did detect all attacks within its scope except two (attacks 6and 17 in Table III) as mentioned earlier

B S3-2017 results

Table VI indicates that 21 out of 31 attacks were detectedby WD while 24 out of 31 attacks were detected by WDHConsidering only the attacks within its scope as mentionedin Section V-A WD detected 21 out of 28 attacks (75)Similarly WDH detected 24 out of 31 attacks (7741) withinits scope mentioned in Section V-B Three attacks on theHistorian are not in the scope of WD All attack targets relatedto RIODisplay (in Table II and in Table IV) are not detectedby both WD and WDH This is because registers inside aPLC save the previous values received from the sensors andthe PLC continues to execute the control code The invariantsalso use the same values stored in the PLC registers and hencedo not raise an alert

In general PLCs send to the Historian via the SCADAworkstation the data received from the sensors When a PLCdoes not have updated values during the attack period it isobvious that the Historian also receives the same stale valuesThis is the reason why WDH also did not detect attacksrelated to RIODisplay Note that the RIODisplay attacks werelaunched and remained active only for a few seconds Duringthis period the PLC did not update the current sensor valuescoming through the RIO If the same attack is performed for alonger duration the PLC would update the data received fromthe sensors Doing so would likely lead to WD and WDHdetecting the RIO attacks

Attacks launched on the Historian were detected by WDHbut not by WD This variance is due to the fact that data inthese attacks is manipulated at the Historian Thus invariantsin a PLC do not have access to the manipulated data andhence the invariants in WD do not raise any alert All attackstargeting a PLC are detected by WD and WDH

WD Detection of physical process attacks All attacks onvalves pressure sensor and level sensors were detected Threeout of four attacks on the chemical dosing process pumpswere detected An example of a detected attack is when theattackers took control of pump P301 (attack 20 in Table IV)

TABLE VIPERFORMANCE OF WD AND WDH

S3-2016 S3-2017

WD WD WDH

Detected 1 5 7 910 11 1213 14 18

3 4 7 9 1011 12 13 1516 18 19 2021 22 23 2628 29 30 31

2 3 4 7 9 1011 12 13 15 1617 18 19 20 2122 23 24 26 2829 30 31

Not detected 2 3 4 68 15 1617

1 2 5 6 8 1417 24 25 27

1 5 6 8 14 2527

through a Python script (Pycomm) to raise the pressure in theUF unit measured by sensor DPIT301 to a dangerous levelWD immediately raised an alarm This invariant ensured thatpump P301 must be OFF when the pressure at DPIT301 wasabove a threshold During the attack the invariant was violatedas the pump was not turned off while DPIT301 indicatedreadings that were above the threshold Consequently an alarmwas raised immediately In certain cases multiple alarmswere raised due to the violation of one or more invariantsFor example when level sensor LIT101 was compromisedthe invariants corresponding to this sensor were violated andraised alarms

WD Detection of sensor data attack WD detected attacks onHMISCADA and PLC values because these attacks directlycompromised the physical processes These attacks eithercompromised chemical dosing water tank levels or pumpstatus through hacking of the HMISCADA or PLC Hencethe robustness of WD in detecting unusual physical processbehavior was found effective in these attacks On the otherhand WD was unable to detect insider attacks that pulled outRIO cables This is because WD triggers an alarm only whenthe invariants are violated Under normal circumstance for aperiod of time a PLC continues to execute its control codeand any invariant code based on the last known state andorvalues Thus the invariants located inside the PLCs are unableto observe this anomalous behavior

WDH Detection of physical process attacks WDH detected14 out of 16 physical process attacks

WDH Detection of sensor data attacks WDH detected theattacks on HMISCADA and PLC values because these attacksdirectly compromised the physical processes albeit with aslightly lower detection rate when compared with the rate ofdetecting physical process attacks As with WD WDH did notdetect any attack launched against the Remote IO by pullingthe cables that connect it to the corresponding PLC WDHfared better in the detection of attacks against the Historian asit was directly accessing data on the Historian server

If the Historian itself or data that is input to the Historian iscompromised WDH takes the decision based on the input itreceives A clever and powerful attacker can attack the physical

TABLE VIIRESULTS FROM S3 2017

Target of Attack Noofattacks

WD WDH

Physical Process Attacks

State of motorised valves 2 100 100

State of water pumps 4 75 75

Pressure in UF 2 100 100

Water tank level 4 100 100

Chemical dosing 4 75 75

Sensor Data Attacks

Data in historian 3 0 100

Data in HMISCADA 3 67 67

Tampering PLC communi-cations

5 100 100

Tampering Remote IO 4 0 0

Total Attacks 31 6774 7741

process and modify values entering the Historian and thusdeceive WDH In general such a situation may arise in allbehavioral intrusion detection systems where the detector takesthe decision based on incorrect input data

Indeed data that appears to be ldquolegitimaterdquo could lead theWDH into believing that there is nothing wrong with thephysical process though there actually is However doingso requires the attacker to continuously manipulate a largenumber of state variables For example consider an attackwhere the attacker turns a pump say P101 ON when it shouldbe OFF and (continually) sends the state of the pump as OFFto the Historian and the corresponding PLC If the pump isOFF then the level of the source and destination tanks must berespectively decreasing and increasing at rates determined bythe pump characteristics Creating ldquolegitimate-lookingrdquo datathus requires an attacker to manipulate several state variablesas explained next (a) Two state variables that correspondto tank levels Two sensors (in SWaT) measure these statevariables (see Figure 2) Thus the attacker must have accessto these level sensors (b) If pump P101 is actually ON whilethe Historian receives its state as OFF then FIT201 must showno flow Thus the attacker will also need to manipulate FIT201to avoid detection This argument can be carried forward tosubsequent stages to show that many sensors will need to bemanipulated by an attacker to ldquohiderdquo a simple attack such asldquochange the state of a pumprdquo In summary yes incorrect dataat the Historian could prevent detection though doing so wouldbe a significant challenge for the attacker due primarily to thedistributed nature of the invariants

VIII DISCUSSION

A Challenges faced

We faced several challenges during S3 For example aftereach teamrsquos performance the operator was required to bringSWaT back to a predefined normal state It was necessary to

keep SWaT in a normal state before another team launched at-tacks Bringing SWaT to its normal state required (a) resettingnetwork communications to ensure that all the communicationchannels are operating as expected (b) the operator to ensurethat all physical processes in SWaT are stable with respectto the control logic (c) the operator to bring back SWaT tothe normal state of that particular device such as a pump or amotorized valve in the case of any physical or manual attacksby the previous team and (d) that the Historian and SCADAservers were reverted to their original state ie the state thatexisted prior to the launch of attacks

B Research questions

RQ1 How do attackers compromise the security of an ICS InSection VI we presented and categorized the attacks based onattacker profiles An attacker can launch physical attacks wheninside the plant such as manually operating a motorized valveor tampering with network cabling Several attacks launchedby the attack teams had not been launched by the authorsin their evaluation of WD [1] and WDH Thus S3 raisedour confidence in the effectiveness of the attack detectionmechanisms based on invariants derived from plant designs

RQ2 How effective is WD in detecting attacks launched byindependent attack teams As mentioned earlier while bothWD and WDH were found to detect a number of attacksthey did fail in several cases Given that the invariants derivedare intended to detect process anomalies it is clear that suchmechanisms must be used in conjunction with other attackdetection tools such as those in [24] [27] [21]

C Assessment by the authors and by independent teamsTable VIII lists the number of attacks launched by the

authors in an experimental evaluation performed prior to S3-2016 [1] Note that the WD detection rate observed by theauthors (89) was higher than the combined rate observedduring the two S3 events (6326) The difference in perfor-mance is due to different attack vectors used in the three setsof experiments WDH detection rate observed during S3 eventis (7741) which is much higher than the WD detection rateSome of these attack vectors are explained in Section VI andthe remaining may be found in [20]

TABLE VIIIPERFORMANCE OF WD AS EVALUATED BY THE AUTHORS AGAINST THOSE

BY PARTICIPANTS IN S3

Experiments by Attacks

Launched Detected (WD) Detected (WDH)

Authors 37 33 (89) NA

S3-2016 18 10 (555) NA

S3-2017 31 21 (677) 24 (774)

NA WDH did not exist at the time of experimentation by the author andduring S3-2016

The data in Table VIII is indicative of the value of orga-nizing S3 events Specifically in the case described in this

paper the two S3 events led to an increased confidence inthe effectiveness of the invariant-based approach in detectingcyber attacks The hackfests also led to the creation of newtypes of attack vectors that were not used earlier to assess theperformance of WD and WDH in detecting cyber attacks

D False alarms

The performance of any attack detection method ought tobe assessed using its detection accuracy ie how many of thelaunched attacks it detects as well as the rate at which falsealarms are raised During S3 each team attempted to launchseveral attacks The attacks listed in Tables III and IV are theones that were successful in realizing the stated attacker intentand were scored by the judges The remaining attacks werenot recorded and hence any alarm generated by such attackswas not considered Some of these unrecorded alarms couldbe false though no specific claims can be made about theirnature

Since S3-2017 the authors have observed no false alarmsfrom WD during normal operation of SWaT WDH has beenin operation since a few weeks prior to S3-2017 Againduring the normal operation of SWaT no alarm has beengenerated by WDH This observation should not be construedto imply that an invariant-based attack detection mechanismwill not generate any false alarmndash in fact it could Howeverif the invariants generated are complete in the sense that theyaccurately capture all aspects of process behavior and theirimplementation is correct and tuned properly the likelihoodof false alarms is low

Even though SWaT is a relatively new plant (2-years sinceits inauguration at the time of writing this paper) we doobserve intermittent failures in a few motorized valves Forexample sometimes MV101 in Stage 1 takes much longer toopen than expected by its controlling PLC1 The PLC itselfdetects such cases In such a case WD or WDH dependingon the time it takes for the valve to finally open will raisean alarm We do not consider this as a false positive simplybecause whether an anomalous behavior is due to a naturalcause or a cyber attack cannot be distinguished by WD orWDH While such distinction is important to make additionalresearch is needed to distinguish process anomalies due tocyber attacks and those arising due to natural componentfailures

E Benefits of S3

S3 exposed the organisers participants and researchers tohow an attacker might design and launch attacks on ICS Bene-fits of S3 include the following 1) An improved understandingof how an ICS operates and the consequent formulation ofnew research directions 2) Opportunity for participants fromindustry and academia to learn from the event and focus onthe limitations of their work 3) An aid to the ICS managementteam to observe the defense teams thus leading to possibleadoption of technology embedded in WD or WDH

F Placement of WD

The placement of WD is another question that ought tobe looked into carefully In this work WD is placed insidePLCs However an exceptionally large number of invariantsmay prevent adding code to the existing control code in a PLCThis may happen due to the computational load requirementson a PLC This aspect led us to create WDH that is placedon the plant network and gets its data from the Historian toevaluate the invariants

G Forensics

One advantage of the invariant-based approach for attackdetection appears while determining the area of impact ofan attack When a single invariant is violated it indicatesclearly the source of process anomaly For example an alertis generated if valve MV101 is closed when the water intank T101 is at or below the L level marker While this alertdoes not indicate how an attacker entered the system or ifthe valve or the level sensor is defective it does assist inlocalising the reason for the alert The analysis becomes abit more complex when multiple invariants raise alerts Thisaspect of an invariant-based detection mechanisms remains tobe analyzed in further detail

H Attacker capabilities

We do not have any validation of the professionalism of theS3 attack teams As mentioned earlier [20] [35] [36] attackteams were from a variety of backgrounds including fromthe industry and academia from Europe and Asia During S3-2017 one team consisting of four membersndashall from outsideof Singaporendash focuses on ethical hacking and cyber-warsinvolving critical infrastructure This team is part of a globalalliance The other teams consist of hackers interested inknowing how vulnerabilities in software can be exploitedand passes this information to others for improving systemssecurity Coverage of attacks launched by the attack teams andattacker profiles is discussed in Section IV and summarizedin Tables I II V and VII

I Attack trees

It is possible to use attack trees [37] [42] to model attackslaunched during the two hackfests reported in this paper Doingso would enable mapping each attack to a specific path inthe attack tree and reveal which attack paths in SWaT weretraversed Such modeling and analysis has not been attemptedin this work and is a possible subject for future research

IX RELATED WORK

S3 is a Capture-The-Flag [15] event on ICS TraditionalCTF events generally attract the attention of both industrialand academic teams and currently enjoy increasing popularityas indicated in [15] The number of such events is graduallyincreasing [13] [16] Such events aid in learning about secu-rity vulnerabilities how these could be exploited nature ofattacks and strength of the deployed [18] [33] [45] defensemechanisms To the best of our knowledge S3 is the first CTF

style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed

The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks

There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc

The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies

Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS

X CONCLUSION

There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense

While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the

actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure

It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously

The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants

The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]

It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question

ACKNOWLEDGMENTS

A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript

REFERENCES

[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016

[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016

[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016

[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016

[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016

[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016

[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017

[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012

[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017

[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and

S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011

[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml

[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010

[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015

[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security

under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler

Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016

[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017

[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016

[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016

[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010

[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for

safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016

[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical

systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013

[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111

[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016

[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013

[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx

[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007

[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016

[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3

[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event

[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008

[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016

[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011

[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf

[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment

of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007

[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016

[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016

[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003

[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014

[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014

[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011

BIOGRAPHY

Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems

Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research

tors Control devices include Programmable Logic Controllers(PLCs) Supervisory Control and Data Acquisition (SCADA)workstations and Human-Machine-Interface (HMI) devicesNetwork devices include network switches and access pointsThe PLCs SCADA and HMI monitor and control the physicalprocess Communication channels in the network act as abridge between the physical process and the control devicesCommunication channels communicate the state of physicalprocess to controllers and control signals to the actuatorsPLCs receive data from sensors compute control actions andapply these actions to specific devices The PLCs in an ICScan be viewed collectively as a distributed control system thattransforms the state of the process through the use of sensorsand actuators

Fig 1 High level view of an ICS

B SWaT Architecture and components

SWaT [30] [40] is a testbed for water treatment It is usedto investigate the response to cyber attacks and experimentwith novel designs of defense mechanisms such as the onesdescribed in [3] The architecture and components in SWaTare described next

Stages in SWaT As shown in Figure 2 SWaT consists ofsix stages labeled Stage 1 through Stage 6 Each stage iscontrolled by its own set of PLCs Stage 1 controls the inflowof water to be treated by opening or closing a valve thatconnects the inlet pipe to the raw water tank (T101) Waterfrom the raw water tank is pumped via a chemical dosing(Stage 2) station to another Ultrafiltration (UF) feed watertank (T301) in Stage 3 In Stage 3 a UF feed pump sendswater via the UF unit to a Reverse Osmosis (RO) feed watertank (T401) in Stage 4 In Stage 4 an RO feed pump sendswater through an ultraviolet dechlorination unit controlled bya PLC Differential pressure sensors in Stage 3 measure thepressure drop across the UF unit A backwash cycle is initiatedwhen the pressure drop exceeds 04 bar indicating that themembranes need immediate cleaning Stage 5 contains a PLCto control the Reverse Osmosis (RO) unit that further filtersthe water using a 2-stage RO process The output of the ROunit enters storage tanks T601 and T602 in Stage 6 Tank T601contains the reject from RO and is used to clean the UF unitusing a backwash process Water in tank T602 contains thepermeate which is recycled into tank T101 in Stage 1

Sensors and actuators SWaT contains 42 sensors and actu-ators across the six stages These include sensors that relateto the dynamics of the process such as water level in tanksflow indicators and pressure indicators as well as thosefor measuring chemical properties of water including pHconductivity and hardness Each PLC has its own set of sensorsand actuators connected through a ring network Thus whena PLC needs to obtain state information from another PLCit must request such information via a suitable commandthe requested data is sent over level 1 network as shown inFigure 3

Communications Figure 3 shows the architecture of thecommunications infrastructure in SWaT Each PLC obtainsdata from sensors associated with the corresponding stageand controls pumps and valves in its domain Flow of wateracross various stages is controlled through opening and closingof valves and turning pumps ON or OFF Level sensors ineach tank enable the PLCs to decide when to turn a pumpON or OFF Several other sensors are available to check onthe physical and chemical properties of water flowing throughthe six stages PLCs communicate with each other through aseparate network Communications among sensors actuatorsand PLCs can be via either wired or wireless links controlledmanually Both wired and wireless networks connect PLCsto the physical process and to the HMI and engineeringworkstation ie the SCADA workstation

The control network is connected to the SCADA worksta-tion [40] through a wired network using a 16-port switch ThePLCs and SCADA workstation are also connected througha wireless network configured using star topology Switcheslocated at each stage enable the use of either wired or wirelesscommunications The communications network is layered intotwo levels For each PLC level 0 refers to the communicationlayer between sensors actuators and the PLC Level 0 net-work is implemented as a ldquodevice level ringrdquo [9] [43] whichincludes a Remote IO (RIO) device The RIO is connectedto the physical sensors and actuators Monitoring and controlinformation is exchanged between the PLC sensors andactuators across a Distributed Logical Router (DLR) Level 1refers to the communication layer among PLCs This layeris implemented in a star topology and includes a SCADAworkstation an HMI and a Historian

Controllers (PLCs) SWaT is equipped with Allen BradleyControlLogix PLCs Therefore some of the attacks describedin the remainder of this paper require consideration of theprotocols used by EtherNetIP [32] for Allen Bradley PLCsSCADA software is developed with tools from Rockwellautomation [8]

Attack detection mechanisms As shown in Figure 3 SWaTcontains four attack detection mechanisms D1 D2 D3 andOrthogonal Defense Mechanism (ODM) [6] D1 is based ona modified version of the open source Bro intrusion detectiontool [10] D2 is a set of three commercially available intrusiondetection systems that use a mix of process dynamics and

Raw Waterinput UltraltrationChemical

dosing Dechlorination

PLC1

Reverse Osmosis

PLC5PLC4PLC3PLC2

Backwash

PLC6

S=LIT101 FIT101

A=MV101 P101

S=AIT202

A=MV201

S=LIT301 FIT301 DPIT301

A=MV201 MV302 P301

S=LIT401 FIT401

A= P401

S=FIT501 FIT502 AIT501

A=MV501 P501

T101 T301Storage tanks T401[No storage]

FromT501From

T502

MV201P101MV101 FIT101

P6

Water fromexternalsource

DPIT301LIT301FIT301P301

MV302P302

Stage 1 Stage 2 Stage 3 Stage 4 Stage 5

Stage 6T601 T602

Fig 2 Six stages in SWaT with corresponding PLCs sensors and actuators Five water storage tanks as shown are labeled Txxx Water level in each tankis measured by the corresponding level indicator labeled as LITxxx AITxxx FITxxx and DPITxxx measure respectively chemical properties of water flowrate in a pipe and differential pressure across the ultrafiltration unit Pxxx denote pumps at various stages

Historian SCADASwitch

PLC 2

AS

PLC 6

AS

PLC 1

AS

HMI

Externaldevices

D1 D1 D1

D2

D3 D3D3

ODM ODMODM

Level 0

Level 1

Level 2

S SensorA ActuatorODM Orthogonal Defense Mechanism

Fig 3 Communications structure of SWaT D1 D2 and D3 denote setsof defense mechanisms in SWaT ODM (Orthogonal Defense Mechanism) isindependent of these mechanisms

other techniques for anomalous process behavior [12] [24][27] D3 (WD) sits inside PLCs and implements a distributedattack detection mechanism that relies exclusively on processdynamics [1] The ODM has direct access to sensors and anal-yses the data received for the existence of process anomaly

C An illustrative attack on SWaT

Consider Stage 1 of SWaT in Figure 2 This stage has amotorized valve labeled MV101 which when open causeswater to flow into tank T101 The inflow into T101 is measuredby flow meter FIT101 and the water level by a level sensorlabeled LIT101 Pump P101 sends water to the next stageFlow meter FIT201 measures the outflow of water fromStage 1 to Stage 3 PLC1 receives the LIT101 reading and

controls the motorized valve MV101 Similarly PLC1 receivesLIT301 readings from PLC3 and controls pump P101

Tanks T101 and T301 have four markers each labeled Low(L) Low Low (LL) High (H) and High High (HH) Eachmarker corresponds to a specific value of water level in thetank These markers are used by the corresponding PLCs tocontrol the states of motorized valves and pumps Thus forexample when the water level in T101 reaches L PLC1 opensMV101 and closes it when the level reaches H When waterlevel in T301 reaches L PLC1 turns P101 ON and turns it OFFwhen the level reaches H The following example illustrates theimpact of compromising level sensor LIT101 with the intentof damaging pump P101

Example Consider an attack where the attackerrsquos intentionis to underflow T101 and damage P101 by making it runwithout any incoming water The attack is launched on LIT101with Stage 1 in the following state LIT301 955mm MV101Closed P101 OFF UF is operational and therefore waterlevel in tank T301 is decreasing Assume now that the attackersets LIT101 reading to a constant value of 790mm In thisattack even though the water level in T101 is changing(decreasing) PLC1 receives a constant value After a whilewhen LIT301 reaches L pump P101 is turned ON by PLC1However the actual water level in tank T101 is lower thanL say at LL This leads to the outflow from the pump beingreduced to less than the intended flow rate Pump P101 runsdry when there is no water in T101 and will eventually getdamaged unless a corrective action is taken

Figure 4 shows the water level in tank T101 during theattack It can be observed that the outflow increases graduallywhen the attack is removed Note that the sudden drop in thevalue of LIT101 soon after attack removal corresponds to thefact that the PLC begins to receive the correct measurement

Fig 4 Water level in tank T101 when LIT101 is attacked LIT101readings are observed by PLC1

Fig 5 Level sensor LIT101 is under attack The attackerrsquos intention isto underflow T101 tank and damage P101 The first arrow indicatesthe outflow reducing time second arrow indicates the pump noisestarting time

of water level in T101 When the water level goes down to150 mm tank T101 does not have enough water to send totank T301 Figure 5 shows the change in flow rate duringthe attack as measured by flow meter FIT201 The two arrowsindicate the start of reduction of outflow from T101 At around10 seconds there is no water flowing from P101 even thoughthe pump is ON At this point the pump becomes noisy and theflow rate reduces to zero If not removed this attack may leadto pump damage due to overheating Of course a mechanicalcut off at the pump would avoid such damage

The above example shows how an attacker could potentiallydamage a pump by changing the sensor values and actuatorstates More complex attacks mentioned in Section VI can bedesigned and launched to reduce the chances of being detected

III OVERVIEW OF WDWD is a mechanism to detect process anomalies A process

is considered anomalous when it deviates from its expectedbehavior WD detects such anomalies through the use of in-variants An invariant [4] is a condition among physical andorchemical properties of the process that must hold wheneveran ICS is in a given state At a given time instant sensormeasurements of a suitable set of such properties constitutethe observable state of the physical process as known to theICS

The invariants serve as checkers of the system state Theseare coded and the code placed inside each PLC used for attack

detection The checker code is added to the control code thatalready exists in each PLC The PLC executes the code in acyclic manner In each cycle data from the sensors is obtainedcontrol actions computed and applied when necessary and theinvariants checked against the state variables or otherwise Dis-tributing the attack detection code among various controllersadds to the scalability of the proposed method During S3 theimplementation was located inside the Programmable LogicControllers (PLCs) as well as embedded in the communicationnetwork

Two types of invariants were considered state dependent(SD) and state agnostic (SA) While both types use statesto define relationships that must hold the SA invariants areindependent of any state based guard while SD invariants areAn SD invariant is true when the plant is in a given state anSA invariant is always true

A State-Dependent (SD) invariants

Consider for example the case when the motorized valveMV101 is Open In this case the flow rate indicator FIT101must provide a non-zero reading to the PLC This phys-ical fact leads to the following state-dependent invariantMV101=Open =rArr FIT101lt δ where δ denotes a thresholdindicating flow Note that an SD invariant may include con-ditions from across the various stages of SWaT thus enablingdistributed detection of attacks Derivation of SD invariants isbased on the design of the ICS and is described in [4]

B State-Agnostic (SA) invariants

Under normal system operation an SA invariant mustalways be true regardless of the system state One SA invariantwas derived for each tank in SWaT to detect attacks that affectthe flow of water into and out of a tank These invariants arebased on the flow of water and water level in a tank andhence are identical in terms of the mathematical relationshipthat they capture

As an example of an SA invariant consider the water levelin a tank At time instant k+1 the water level in T101 dependson the level at time k and the inflow and outflow at instant kThis relationship is captured in the following idealized discretetime model of the tank

x(k + 1) = x(k) + α(ui(k)minus uo(k)) (1)

where ui(k) and uo(k) denote the inflow and outflow ratesat time k and α is a proportionality constant that convertsflow rate to change in level using the tank dimensions x(k)is the true state of the water level Let y(k) denote the sensormeasurement of the water level x(k) an estimate of the levelsensor reading and ε a threshold based on experimentationBased on Eqn 1 the statistics obtained experimentally andconverting the true states to their estimates the followinginvariant is derived to test whether or not the tank fillingprocess is anomalous

sumn

i=1|(x(i)minus y(i))|n

gt ε under attack (2)

le ε normal (3)

Fig 6 Invariant to detect anomalous behavior of LIT 101

IV SWAT SECURITY SHOWDOWN (S3)

This section presents details of the two S3 [36] eventsincluding guidelines and selected information on participantsIn S3 the attackers are challenged to realise concrete goals inSWaT Points earned by an attack team are weighted based onthe capabilities needed to launch the attack and the number ofdefence mechanisms successfully bypassed during the attackThe goal was to meet as many pre-defined challenges aspossible within the pre-allocated time

Information disclosed to the attack teams Technical detailson SWaT such as network architecture protocols and devicesused are released to the attackers one month prior to theirarrival for participation in the event Publicly available whitepapers on mechanisms deployed by the defence teams areshared with each attack team

Information disclosed to the defenders S3 organizers workedclosely with the defense teams to integrate their defence mech-anisms into SWaT Information about the normal operation ofSWaT was disclosed to the defenders to enable them to fine-tune their detection systems and reduce false alarms as muchas they could

Attacker profiles Attack teams were asked to select from aset of attacker profiles [34] The following attacker profileswere available cyber-criminal insider or a combination ofboth An attacker profile is intended to restrict availability ofresources and limit the access rights of the attackers as shownin Table I

A S3-2016

Attack teams included three from industry and three fromacademia Similarly there were three defense teams from theindustry and three from academia During the live phase heldat the SWaT testbed all six [35] defence mechanisms weresimultaneously in place Each team was given 12 hours forpassive reconnaissance and team was assigned a 3-hour slotduring which they were able to launch attacks

B S3-2017

Attack teams included one from industry and four fromacademia There were two defense teams from the industryand two from academia Each attack team was given two ses-sions [36] of four hours each to conduct reconnaissance on thetestbeds During these sessions various attacks were preparedand tested with the assistance of the SWaT laboratory engineerDuring the actual event each team was given two hours todemonstrate their attacks that were prepared previously Attack

TABLE IRESOURCES AND ACCESS RIGHTS FOR ATTACKER PROFILES

Profile Constraints

Cyber-criminal Limited number of attempts to realize a goal

Physical access not allowed manual manipulation ofthe sensors and actuators are not allowed

Direct connection to PLCs using any software suchas Allen Bradleyrsquos Studio5000 not allowed

Insider Physical access to SWaT allowed manual manipula-tion of the sensors and actuators are allowed

Allowed to alter the network topology

Direct connection to PLCs using any software suchas Allen Bradleyrsquos Studio5000 allowed

TABLE IITARGETS OF ATTACKS IN S3

Target Description

Physical Process Attacks

Valves Control the motorized valves

Pumps Disrupt pump control operations

Pressure Alter the pressure in pipes

Tank fill level Alter water level in a tank

Chemical dosing Alter chemical dosing

Sensor Data Attacks

Historian Alter data in the Historian

HMISCADA Alter the sensor actuator values at HMI orSCADA DoS Attacks on SCADA HMI

PLC Reprogram PLC DoS attacks on PLCsChange the commands and values in which thePLC receives and sends

RIODisplay Control of the RIO through disconnected ana-logue InputOutput pin

teams were also given a separate network for Internet up-linkand up to three Virtual Machines (VMs) running either Linuxor Windows operating system

C Attack targets

The attack teams were given a list of components andsubsystems in SWaT that could serve as the target of theirattacks Table II lists the targets available to the attack teamsTable II has two kinds of attacks physical process attacks andsensor data attacks In physical process attacks an attackerrsquosobjective is to alter the physical process In the case of sensordata attacks an attackerrsquos objective is to alter the sensor oractuator tags during communication or in the Historian

V PREPARATION FOR S3

To prepare for S3-2016 an earlier version of WD wasextended to all six stages of SWaT This extension requiredthe generation of invariants across all stages coding of theinvariants and placement of the code inside [1] the six PLCsThe modified WD was tested on SWaT by running the plantunder various operating conditions

Based on lessons learned during S3-2016 several newinvariants were generated coded and added to the PLCs ForS3-2017 we decided to use an additional monitoring systemplaced outside the PLCs This system collects data from theHistorian and evaluates the invariants All invariants wereimplemented in a Linux environment using a Piwebclient APIto talk to the Historian This new implementation is referredto as WDH

The invariants in WD are coded using ladder logic andstructured text while those in WDH in Python Both imple-mentations use the same set of invariants the difference is intheir placement The Historian may not get all the data andcommands that flow across the PLCs sensors and actuatorsHowever as WDH gets its data directly from the Historian ithas access to information flowing across SCADA workstationand the Historian This information may be compromised byan attacker and is not available to the PLC

A Scope of WD

WD is designed to detect process anomalies Thus anyabnormal behavior in the water treatment process in SWaTought to be detected by WD However there could be attacksthat do not cause the process to deviate from its normalbehavior but lead to undesirable consequences An exampleof such an attack is one intended to deface the screen on theSCADA workstation or the HMI Such an attack will not bedetected by WD Attacks that may cause process anomaly butonly after an attack has been removed from the system mayalso not be detected by WD Denial of Service is one suchattack

B Scope of WDH

WDH and WD use the same set of invariants Howeverthe placement of WDH could lead to a difference in detectioncapabilities of the two defense mechanisms WDH gets its datafrom Historian while WD directly from the PLC Data that isnot programmed to be logged in the Historian will not beaccessible to WDH Thus any anomaly that requires such datawill likely not be detected by WDH Similarly attacks thatmanipulate data entering the Historian or SCADA may not bevisible to WD Thus while the two invariant-based processanomaly detection mechanisms are identical in the invariantsthey use their placement in SWaT is expected to result indifferent performance in detecting attacks

VI S3 ATTACKS

The attacks launched by teams participating in the two S3

events are described next

A S3-2016 Attacks

All attacks designed and launched during S3-2016 areenumerated in Table III Three attacks selected from Table IIIare described next Details of all attacks are available in [9]Of the 18 attacks in Table III 4 and 16 are cyber criminalattacks and the remaining are insider attacks

DoS attack on SCADA In this attack (attack 4 in Table III)the attackerrsquos intention was to deface the SCADA workstationscreen and hence prevent the operator from observing plantstate The cyber-criminal attacker model was used to designthis attack To realize the intention the attacker launched anARP poisoning Man-in-the-Middle attack in two steps In thefirst step all traffic intended for HMI was redirected to theSCADA workstation In the second step this redirected trafficwas dropped and thus no packets were received at the SCADAworkstation This led to the screen on the workstation becom-ing completely gray and no state information was displayedThis attack was not detected by WD as it did not lead toany process anomaly It is an ARP spoofing attack and not atraditional DoS attack As part of the DoS attack the attackertargeted the PLC and sent millions of packets at a time Thisled to the same effect as would be the case when an ARPspoofing attack is performed on SCADA

Manipulation of the chemical dosing pump Intention of theattacker in this case (attack 14 in Table III) was to manipulatethe pH of water entering Stage 3 of SWaT The insider-attackermodel was used in the design of this attack This attack wasexecuted in two steps In the first step PLC 2 was set to manualmode Note that in manual mode the plant operator can directlycontrol the actuators eg the dosing pumps in this case In thesecond step the attacker altered the chemical dosing processin the Pre-treatment Stage 2 of SWaT by interacting directlywith the HMI interface and overriding the commands sent bythe PLC WD was able to detect this attack because the set-points changed by the attacker were different from those setin WD

DoS to PLC by SYN flooding The intention of the attackerin this case (attack 16 in Table III) was to disable the HMIso that an operator is unable to view or control the plantoperation The insider-attacker model was used in the designof this attack In this way the attacker had an access to theadministrator account and the associated tools The attackerperformed a SYN flooding attack on EthernetIP server ofPLC1

As a result of this DoS attack the HMI was unable toobtain the current state values to display and would insteaddisplay 0 or characters WD was unable to detect this attackphysical process as not affected During the attack period PLCwas controlling the process as expected Such attacks whilenot altering process behavior may impede supervision of theprocess in an operational plant

B S3-2017 Attacks

All attacks designed and launched during S3-2017 areenumerated in Table IV Selected attacks from Table III aredescribed next Details of all attacks are available in [20] Ofthe 31 attacks in Table IV 17 can be classified as cybercriminal attacks and the remaining as insider attacks (Figure I)All attacks launched during S3-2016 and S3-2017 are listedand categorized in Table V

TABLE IIIATTACKS LAUNCHED DURING S3-2016

SNO Target Method Attack Tool

1 Tank fill levelLIT101

Use HMI access Close MV101 and Stop P101 andP102

HMI

2 HMISCADA ARP spoofing Attack HMI DoS attack Ettercap

3 PLC Manual access Removed the cable at the ring atlevel 0

Manual

4 HMISCADA DoS on HMI by droppingall packets between PLC andSCADAHMI

DoS attack on SCADA wide DoSattack took a while to restore SWaTto its normal state

Ettercap

5 Tank fill levelLIT101

Use HMI access Attack on LIT101 ManualHMI

6 Valve MV301 Use SCADA access Attack on MV301 manually openfrom the SCAD workstation

ManualSCADA

7 Pump P101 Use SCADA access Attack pump manually open it fromthe SCADA workstation

ManualSCADA

8 Historian DoS attack using CPPPO andloop

Attack between HMI and PLC CPPPO

9 Valve MV101 Use SCADA access MV101 attacked using SCADAchanged the valve state from Opento Closed

ManualSCADA

10 Pump P101 Use SCADA access LIT301 set point changed ManualSCADA

11 Tank fill levelLIT301

Using SCADA access LIT301 set point altered ManualSCADA

12 Chemical dosingP201

Control MV101 and AIT503 setpoints of LIT301 to ensure flowthis triggered chemical dosing

Dosing pump attack on P201 ManualSCADA

13 HMISCADALIT101

Functional block introduce newconstant tag tie that to output tagcould only do zero

LIT101 set to zero from PLC Studio5000

14 Chemical dosingpump P205

Use SCADA access Manipulation of the chemical dosingpump (P205)

ManualSCADA

15 HMISCADA DoS on HMI using Level 1 net-work

Attack on HMI EttercapPycomm

16 Historian SYN flood ENIP port at PLC1 DoS to PLC by SYN flooding (attackon HMI)

Ettercap

17 Chemical dosingpump P203

HMI-based direct manipulation Attack on P203 while the four dosingpumps are running

ManualHMI

18 HMISCADALIT101

Re-program PLC to fix LIT101value to an arbitrary value

Attack on LIT101 Studio5000

416 are cyber criminal attacks in S3-2016

Control of the chemical dosing system through a Pythonscript (Pycomm) The objective of this attack (attack 15 inTable IV) was to change chemical dosing at the end of the de-chlorination system (Stage 4) First the attackers compromisedVirtual Network Computing (VNC) Then they used a Pythonscript (Pycomm) and Wireshark to gain access to the HMIAfter gaining access to the HMI through the compromisedVNC the cybercriminal attacker used Wireshark to capturethe packets flowing between the HMI and PLC4 The con-troller tags were retrieved by an analysis of the packets Theattackers changed the data associated with these tags to controlthe chemical dosing function using the Pycomm framework

Control of PLC through the Bridged Man-in-the-Middle(MiTM) at Level 0 the objective of this attack was (attack 16in Table IV) to change the commands and values that PLC1receives and sends First the attackers configured a bridgebetween the RIO and PLC1 using Netfilterqueue andScapy The attack was launched at two network levels Ananalysis on the network traffic revealed the packets that theattackers should edit As the target of this attack was thewater level in T101 the attackers set it to a constant valueto hide from PLC1 the rise in water level in T101 Before apacket was forwarded Netfilterqueue rerouted it into aqueue which can be read and modified by the Python script

TABLE IVATTACKS LAUNCHED DURING S3-2017

SNo Target Method Attack Tool

1 HMISCADA LIT401 HMI simulation insider attack Change the value of LIT401 in the HMI Manual HMI

2 Historian ARP and drop Change the value stored at the Historian Ettercap

3 Valve MV201 Reprogram PLC Change the status of the MV201 Studio 5000

4 Tank fill level LIT301 420to 320

Manual Lower the water tank level from 820mm to420mm without raising any alarm LIT301decreased till 320mm

Manual HMI

5 Pump P101 Manual mode of pump Alternate the state [OnOff] of the pump P101 Manual HMI

6 Chemical dosing P205 Manually dosing chemical pump Change the chemical dosage of sodiumhypochlorite (NaOCl) in P2

Manual SCADA

7 PLC Disconnect cable Disrupt sensor values from remote inputoutput(RIO) to the PLC

Manual

8 RIO Display Disconnect IO PIN manual Disrupt the sensor reading send to PLC throughRemote IO (RIO)

Manual

9 Chemical dosing P404 MiTM Python script to control Increase chemical dosage in pre-treatment Python script

10 LIT101 (476mm to 540mm ) Reprogram PLC Falsify water level display at SCADA Studio 5000

11 Pump P101 HMI simulation insider attack Alternate the state [OnOff] of the pump P101 Manual HMI

12 HMISCADA AIT 504 ARP+rewriting Increase AIT504 Ettercap

13 PLC LIT401 Reprogram PLC Falsify water level display at SCADA Studio 5000

14 RIODisplay Disconnect specific IO PIN basedon manual

Disrupt the sensor reading send to PLC throughremote IO (RIO)

Manual

15 Chemical dosing pumpP403 AIT501

Based on captured traffic betweenHMI and PLC4

Change chemical dosing function VNC Python script PycommWireshark

16 PLC LT101 from 742mm to500mm

Level 0 MITM Change the commands and values that the PLCreceives and sends

Aircrack Airodump AireplayNetfilterqueue Scapy

17 Historian LT101 tag Aircrack WiFi ARP spoofingEttercap

Compromise historian data Ettercap Aircrack

18 Pressure sensor DPIT30130MV301-4

SMB to EW get project files runFT

Disrupt valves operation of Ultrafiltration andBackwash (P3)

SMB

19 MV201 LT101 metasploit+vnc Change the water level of the tank LIT101 Metasploit+vnc

20 Pump P501 Rogue AP disassociated Telnetwith default credentials to turnoff original AP Scapy rewrite

Disrupt pump control operation KisMAC Password crackingtool 3vilTwinAttacker TelnetScapy

21 PLC LIT101 Reprogram PLC Change level indicator value Studio 5000

22 Pump P101 LIT301 Using back-door connection Establish back-door connection Mimikatz malicious VBAMacro SOCKS proxy

23 HMISCADA P201 Netfilterqueue Scapy Change the display value of the HMI Netfilterqueue Scapy

24 Historian LIT101 Overwrote specific data stored at the Historian Microsoft PsExec ipconfig

25 RIODisplay Manual Control of the RIO through disconnected Ana-logue InputOutput pin

Manual

26 Valve MV201 Manual Permanently closed the motorised valve re-gardless of commands issued

Manual SCADA

27 RIODisplay AIT202203 Manual Change the pH value shown at HMI Manual HMI

28 PLC MV201 P101 ARP poisoning MiTM Increase the pressure at P1 Ettercap

29 Tank fill level LIT101 Lower Falsify the water level reading of the tankdisplayed at SCADA

Pycomm

30 Chemical dosing PLC2 Use Studio 5000 Change the level of the chemical used fordosing

Studio 5000

31 Pressure MV302 P3012 Using Pycomm script Change the pump state sent to the PLC Pycomm

TABLE VCYBER CRIMINAL ATTACKS IN S3

Cyber Criminal Attacks Insider Attacks

S3-2016 4 16 1 2 3 5 6 7 8 910 11 12 13 14 1517 18

S3-2017 2 9 10 12 13 1516 17 18 19 20 2122 24 28 29 30

1 3 4 5 6 7 8 1114 23 25 26 27 31

To prevent all packets from entering the queue in order notto disrupt other processes iptables was used to identifythe targeted packets entering the queue Using Scapy and acustom dissector the attacker edited the payload of the targetedpacket which was then forwarded to its original destination

Control of Historian through the Aircrack WiFi The objectiveof this attack was (attack 17 in Table IV) to compromise thedata stored in the Historian Attackers performed crack WiFipassword ARP poisoning and MiTM payload manipulationusing Aircrack and Ettercap As PLC1 was operating inthe wireless mode the cybercriminal attacker used Aircrackto obtain the password for connecting to the ICS Access Point(AP) ARP poisoning was executed to reroute traffic betweenPLC1 and the Historian through the attackerrsquos rogue terminalThe attackers then used an Ettercap filter to manipulate thenetwork packets The attackers changed the tag correspondingto LIT101 to an arbitrary value before releasing the packetsto the Historian

Control of pressure through the Server Message Block (SMB)The objective of this attacks was (attack 18 in Table IV) todisrupt the state of four motorized valves in Stage 3 to affectthe differential pressure in UF Vulnerability CVE-2008-21601

in Factory Talk software from Rockwell and in MicrosoftrsquosServer Message Block (SMB) was used by the attackersto obtain files from the HMI As the HMI was runningWindows CE it has a vulnerability that allows an attackerrsquosterminal to execute arbitrary code on the HMI Thus theattackers were able to retrieve the files to create a copy ofthe workstation From the copied workstation the attackersmanually changed the state of the valves in Stage 3 suchthat the differential pressure across the UF unit as measuredby DPIT301 became dangerously high The attackers closedvalves MV301 MV302 and MV303 and opened MV304

Control of water level in the tank through the MetasploitVNC Scanner Objective of this attack was (attack 19 inTable IV) to change the water level in tank T101 The attackersused Metasploit VNC authentication None scanner to ob-tain access to the VNC server without password protection andto check for nodes running a VNC Server Once the scannerdetected the VNC Server running without any authenticationthe attackers penetrated into the server through a VNC Clientconnection As the VNC Server was hosting the HMI which

1httpswwwcvedetailscomcveCVE-2008-2160

controlled the ICS the attackers changed the simulation tagassociated with water level in T101

Control of a pump through a rogue router The objective ofthis attack (attack 20 in Table IV) was to disrupt the controlof pump P501 The attackers used Evil twin (rogue accesspoint) method using KisMAC a password cracking tool3vilTwinAttacker Telnet and Scapy The attackersused KisMAC to scan for wireless networks in the ICS Oncethe targeted wireless network was identified the attackers useddictionary attack to crack the password After the passwordwas cracked the attackers created a rogue wireless routerwith a similar SSID and configuration They then sent a de-authentication packet to disassociate PLC5 and the originalrouter The attackers used Telnet to log into the originalrouter and shut it down Scapy was then used to modify thepackets to turn the pump on

VII RESULTS

Tables VI and VII summarize the response of WD andWDH to the attacks launched during the two S3 events Recallthat both WD and WDH contain exactly the same set ofinvariants In WD the invariants are coded and placed insidethe PLCs whereas in WDH the invariants are coded and placedat the Historian WDH did not exist during S3-2016 and hencethe response of WDH is available only for attacks launchedduring S3-2017

A S3-2016 results

We note from Table VI that 10 out of 18 attacks weredetected immediately while the remaining eight attacks werenot detected Six of the eight undetected attacks did not leadto process anomaly during the observation period and hencedid not violate any invariant This outcome is expected as theinvariants in WD are designed to detect process anomaly

Consider attack 2 ARP spoofing in Table III This is aDoS attack on HMI It leads to defacing the screen on theHMI or displaying incorrect information thereby preventingan operator from knowing the actual plant state Howeverthe attack does not cause process anomaly and hence is notdetected as it does not violate any invariant Similar logic canbe used to explain why the other attacks in Table VI are notdetected

It is important to note that a DoS attack when given enoughtime to evolve and be launched at an appropriate state of theplant may impact physical process behavior In such a caseone or more invariants may detect the attack One such attackis 16 in Table VI This attack prevented the Historian fromreceiving data from PLC1 However if this attack was leftactive for a longer period it would prevent PLC1 from sendingappropriate commands to the actuators eg to MV101 orP101 In turn this would have led to process anomaly Notenough data is available to conclude with certainty whether ornot this attack would be detected by WD if active for sufficienttime

Two single point [2] attacks were not detected by WD Inone attack (attack 6 in Table III) the adversary altered the status

of valve MV301 Under normal circumstances this valve isopened during the backwash process However the attackeropened it when there was no backwash Hence the attackdid not affect the physical process except in changing thevalve status No invariant was violated due to this attackbecause the backwash process ie Stage 6 is not includedin this case study The second single point attack (attack 17 inTable III) was performed on chemical dosing pump P203 whilethe other pump P204 was running Note that under normalcircumstances only one of these two pumps is supposed to berunning while the other remains as a backup Subsequently theattacker shut down pump P204 This attack was not detectedbecause there were no invariants that related to the chemicalproperties of water

Although the overall performance of WD was below 100it did detect all attacks within its scope except two (attacks 6and 17 in Table III) as mentioned earlier

B S3-2017 results

Table VI indicates that 21 out of 31 attacks were detectedby WD while 24 out of 31 attacks were detected by WDHConsidering only the attacks within its scope as mentionedin Section V-A WD detected 21 out of 28 attacks (75)Similarly WDH detected 24 out of 31 attacks (7741) withinits scope mentioned in Section V-B Three attacks on theHistorian are not in the scope of WD All attack targets relatedto RIODisplay (in Table II and in Table IV) are not detectedby both WD and WDH This is because registers inside aPLC save the previous values received from the sensors andthe PLC continues to execute the control code The invariantsalso use the same values stored in the PLC registers and hencedo not raise an alert

In general PLCs send to the Historian via the SCADAworkstation the data received from the sensors When a PLCdoes not have updated values during the attack period it isobvious that the Historian also receives the same stale valuesThis is the reason why WDH also did not detect attacksrelated to RIODisplay Note that the RIODisplay attacks werelaunched and remained active only for a few seconds Duringthis period the PLC did not update the current sensor valuescoming through the RIO If the same attack is performed for alonger duration the PLC would update the data received fromthe sensors Doing so would likely lead to WD and WDHdetecting the RIO attacks

Attacks launched on the Historian were detected by WDHbut not by WD This variance is due to the fact that data inthese attacks is manipulated at the Historian Thus invariantsin a PLC do not have access to the manipulated data andhence the invariants in WD do not raise any alert All attackstargeting a PLC are detected by WD and WDH

WD Detection of physical process attacks All attacks onvalves pressure sensor and level sensors were detected Threeout of four attacks on the chemical dosing process pumpswere detected An example of a detected attack is when theattackers took control of pump P301 (attack 20 in Table IV)

TABLE VIPERFORMANCE OF WD AND WDH

S3-2016 S3-2017

WD WD WDH

Detected 1 5 7 910 11 1213 14 18

3 4 7 9 1011 12 13 1516 18 19 2021 22 23 2628 29 30 31

2 3 4 7 9 1011 12 13 15 1617 18 19 20 2122 23 24 26 2829 30 31

Not detected 2 3 4 68 15 1617

1 2 5 6 8 1417 24 25 27

1 5 6 8 14 2527

through a Python script (Pycomm) to raise the pressure in theUF unit measured by sensor DPIT301 to a dangerous levelWD immediately raised an alarm This invariant ensured thatpump P301 must be OFF when the pressure at DPIT301 wasabove a threshold During the attack the invariant was violatedas the pump was not turned off while DPIT301 indicatedreadings that were above the threshold Consequently an alarmwas raised immediately In certain cases multiple alarmswere raised due to the violation of one or more invariantsFor example when level sensor LIT101 was compromisedthe invariants corresponding to this sensor were violated andraised alarms

WD Detection of sensor data attack WD detected attacks onHMISCADA and PLC values because these attacks directlycompromised the physical processes These attacks eithercompromised chemical dosing water tank levels or pumpstatus through hacking of the HMISCADA or PLC Hencethe robustness of WD in detecting unusual physical processbehavior was found effective in these attacks On the otherhand WD was unable to detect insider attacks that pulled outRIO cables This is because WD triggers an alarm only whenthe invariants are violated Under normal circumstance for aperiod of time a PLC continues to execute its control codeand any invariant code based on the last known state andorvalues Thus the invariants located inside the PLCs are unableto observe this anomalous behavior

WDH Detection of physical process attacks WDH detected14 out of 16 physical process attacks

WDH Detection of sensor data attacks WDH detected theattacks on HMISCADA and PLC values because these attacksdirectly compromised the physical processes albeit with aslightly lower detection rate when compared with the rate ofdetecting physical process attacks As with WD WDH did notdetect any attack launched against the Remote IO by pullingthe cables that connect it to the corresponding PLC WDHfared better in the detection of attacks against the Historian asit was directly accessing data on the Historian server

If the Historian itself or data that is input to the Historian iscompromised WDH takes the decision based on the input itreceives A clever and powerful attacker can attack the physical

TABLE VIIRESULTS FROM S3 2017

Target of Attack Noofattacks

WD WDH

Physical Process Attacks

State of motorised valves 2 100 100

State of water pumps 4 75 75

Pressure in UF 2 100 100

Water tank level 4 100 100

Chemical dosing 4 75 75

Sensor Data Attacks

Data in historian 3 0 100

Data in HMISCADA 3 67 67

Tampering PLC communi-cations

5 100 100

Tampering Remote IO 4 0 0

Total Attacks 31 6774 7741

process and modify values entering the Historian and thusdeceive WDH In general such a situation may arise in allbehavioral intrusion detection systems where the detector takesthe decision based on incorrect input data

Indeed data that appears to be ldquolegitimaterdquo could lead theWDH into believing that there is nothing wrong with thephysical process though there actually is However doingso requires the attacker to continuously manipulate a largenumber of state variables For example consider an attackwhere the attacker turns a pump say P101 ON when it shouldbe OFF and (continually) sends the state of the pump as OFFto the Historian and the corresponding PLC If the pump isOFF then the level of the source and destination tanks must berespectively decreasing and increasing at rates determined bythe pump characteristics Creating ldquolegitimate-lookingrdquo datathus requires an attacker to manipulate several state variablesas explained next (a) Two state variables that correspondto tank levels Two sensors (in SWaT) measure these statevariables (see Figure 2) Thus the attacker must have accessto these level sensors (b) If pump P101 is actually ON whilethe Historian receives its state as OFF then FIT201 must showno flow Thus the attacker will also need to manipulate FIT201to avoid detection This argument can be carried forward tosubsequent stages to show that many sensors will need to bemanipulated by an attacker to ldquohiderdquo a simple attack such asldquochange the state of a pumprdquo In summary yes incorrect dataat the Historian could prevent detection though doing so wouldbe a significant challenge for the attacker due primarily to thedistributed nature of the invariants

VIII DISCUSSION

A Challenges faced

We faced several challenges during S3 For example aftereach teamrsquos performance the operator was required to bringSWaT back to a predefined normal state It was necessary to

keep SWaT in a normal state before another team launched at-tacks Bringing SWaT to its normal state required (a) resettingnetwork communications to ensure that all the communicationchannels are operating as expected (b) the operator to ensurethat all physical processes in SWaT are stable with respectto the control logic (c) the operator to bring back SWaT tothe normal state of that particular device such as a pump or amotorized valve in the case of any physical or manual attacksby the previous team and (d) that the Historian and SCADAservers were reverted to their original state ie the state thatexisted prior to the launch of attacks

B Research questions

RQ1 How do attackers compromise the security of an ICS InSection VI we presented and categorized the attacks based onattacker profiles An attacker can launch physical attacks wheninside the plant such as manually operating a motorized valveor tampering with network cabling Several attacks launchedby the attack teams had not been launched by the authorsin their evaluation of WD [1] and WDH Thus S3 raisedour confidence in the effectiveness of the attack detectionmechanisms based on invariants derived from plant designs

RQ2 How effective is WD in detecting attacks launched byindependent attack teams As mentioned earlier while bothWD and WDH were found to detect a number of attacksthey did fail in several cases Given that the invariants derivedare intended to detect process anomalies it is clear that suchmechanisms must be used in conjunction with other attackdetection tools such as those in [24] [27] [21]

C Assessment by the authors and by independent teamsTable VIII lists the number of attacks launched by the

authors in an experimental evaluation performed prior to S3-2016 [1] Note that the WD detection rate observed by theauthors (89) was higher than the combined rate observedduring the two S3 events (6326) The difference in perfor-mance is due to different attack vectors used in the three setsof experiments WDH detection rate observed during S3 eventis (7741) which is much higher than the WD detection rateSome of these attack vectors are explained in Section VI andthe remaining may be found in [20]

TABLE VIIIPERFORMANCE OF WD AS EVALUATED BY THE AUTHORS AGAINST THOSE

BY PARTICIPANTS IN S3

Experiments by Attacks

Launched Detected (WD) Detected (WDH)

Authors 37 33 (89) NA

S3-2016 18 10 (555) NA

S3-2017 31 21 (677) 24 (774)

NA WDH did not exist at the time of experimentation by the author andduring S3-2016

The data in Table VIII is indicative of the value of orga-nizing S3 events Specifically in the case described in this

paper the two S3 events led to an increased confidence inthe effectiveness of the invariant-based approach in detectingcyber attacks The hackfests also led to the creation of newtypes of attack vectors that were not used earlier to assess theperformance of WD and WDH in detecting cyber attacks

D False alarms

The performance of any attack detection method ought tobe assessed using its detection accuracy ie how many of thelaunched attacks it detects as well as the rate at which falsealarms are raised During S3 each team attempted to launchseveral attacks The attacks listed in Tables III and IV are theones that were successful in realizing the stated attacker intentand were scored by the judges The remaining attacks werenot recorded and hence any alarm generated by such attackswas not considered Some of these unrecorded alarms couldbe false though no specific claims can be made about theirnature

Since S3-2017 the authors have observed no false alarmsfrom WD during normal operation of SWaT WDH has beenin operation since a few weeks prior to S3-2017 Againduring the normal operation of SWaT no alarm has beengenerated by WDH This observation should not be construedto imply that an invariant-based attack detection mechanismwill not generate any false alarmndash in fact it could Howeverif the invariants generated are complete in the sense that theyaccurately capture all aspects of process behavior and theirimplementation is correct and tuned properly the likelihoodof false alarms is low

Even though SWaT is a relatively new plant (2-years sinceits inauguration at the time of writing this paper) we doobserve intermittent failures in a few motorized valves Forexample sometimes MV101 in Stage 1 takes much longer toopen than expected by its controlling PLC1 The PLC itselfdetects such cases In such a case WD or WDH dependingon the time it takes for the valve to finally open will raisean alarm We do not consider this as a false positive simplybecause whether an anomalous behavior is due to a naturalcause or a cyber attack cannot be distinguished by WD orWDH While such distinction is important to make additionalresearch is needed to distinguish process anomalies due tocyber attacks and those arising due to natural componentfailures

E Benefits of S3

S3 exposed the organisers participants and researchers tohow an attacker might design and launch attacks on ICS Bene-fits of S3 include the following 1) An improved understandingof how an ICS operates and the consequent formulation ofnew research directions 2) Opportunity for participants fromindustry and academia to learn from the event and focus onthe limitations of their work 3) An aid to the ICS managementteam to observe the defense teams thus leading to possibleadoption of technology embedded in WD or WDH

F Placement of WD

The placement of WD is another question that ought tobe looked into carefully In this work WD is placed insidePLCs However an exceptionally large number of invariantsmay prevent adding code to the existing control code in a PLCThis may happen due to the computational load requirementson a PLC This aspect led us to create WDH that is placedon the plant network and gets its data from the Historian toevaluate the invariants

G Forensics

One advantage of the invariant-based approach for attackdetection appears while determining the area of impact ofan attack When a single invariant is violated it indicatesclearly the source of process anomaly For example an alertis generated if valve MV101 is closed when the water intank T101 is at or below the L level marker While this alertdoes not indicate how an attacker entered the system or ifthe valve or the level sensor is defective it does assist inlocalising the reason for the alert The analysis becomes abit more complex when multiple invariants raise alerts Thisaspect of an invariant-based detection mechanisms remains tobe analyzed in further detail

H Attacker capabilities

We do not have any validation of the professionalism of theS3 attack teams As mentioned earlier [20] [35] [36] attackteams were from a variety of backgrounds including fromthe industry and academia from Europe and Asia During S3-2017 one team consisting of four membersndashall from outsideof Singaporendash focuses on ethical hacking and cyber-warsinvolving critical infrastructure This team is part of a globalalliance The other teams consist of hackers interested inknowing how vulnerabilities in software can be exploitedand passes this information to others for improving systemssecurity Coverage of attacks launched by the attack teams andattacker profiles is discussed in Section IV and summarizedin Tables I II V and VII

I Attack trees

It is possible to use attack trees [37] [42] to model attackslaunched during the two hackfests reported in this paper Doingso would enable mapping each attack to a specific path inthe attack tree and reveal which attack paths in SWaT weretraversed Such modeling and analysis has not been attemptedin this work and is a possible subject for future research

IX RELATED WORK

S3 is a Capture-The-Flag [15] event on ICS TraditionalCTF events generally attract the attention of both industrialand academic teams and currently enjoy increasing popularityas indicated in [15] The number of such events is graduallyincreasing [13] [16] Such events aid in learning about secu-rity vulnerabilities how these could be exploited nature ofattacks and strength of the deployed [18] [33] [45] defensemechanisms To the best of our knowledge S3 is the first CTF

style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed

The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks

There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc

The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies

Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS

X CONCLUSION

There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense

While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the

actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure

It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously

The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants

The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]

It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question

ACKNOWLEDGMENTS

A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript

REFERENCES

[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016

[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016

[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016

[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016

[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016

[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016

[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017

[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012

[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017

[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and

S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011

[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml

[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010

[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015

[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security

under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler

Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016

[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017

[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016

[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016

[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010

[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for

safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016

[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical

systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013

[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111

[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016

[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013

[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx

[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007

[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016

[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3

[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event

[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008

[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016

[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011

[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf

[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment

of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007

[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016

[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016

[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003

[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014

[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014

[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011

BIOGRAPHY

Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems

Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research

Raw Waterinput UltraltrationChemical

dosing Dechlorination

PLC1

Reverse Osmosis

PLC5PLC4PLC3PLC2

Backwash

PLC6

S=LIT101 FIT101

A=MV101 P101

S=AIT202

A=MV201

S=LIT301 FIT301 DPIT301

A=MV201 MV302 P301

S=LIT401 FIT401

A= P401

S=FIT501 FIT502 AIT501

A=MV501 P501

T101 T301Storage tanks T401[No storage]

FromT501From

T502

MV201P101MV101 FIT101

P6

Water fromexternalsource

DPIT301LIT301FIT301P301

MV302P302

Stage 1 Stage 2 Stage 3 Stage 4 Stage 5

Stage 6T601 T602

Fig 2 Six stages in SWaT with corresponding PLCs sensors and actuators Five water storage tanks as shown are labeled Txxx Water level in each tankis measured by the corresponding level indicator labeled as LITxxx AITxxx FITxxx and DPITxxx measure respectively chemical properties of water flowrate in a pipe and differential pressure across the ultrafiltration unit Pxxx denote pumps at various stages

Historian SCADASwitch

PLC 2

AS

PLC 6

AS

PLC 1

AS

HMI

Externaldevices

D1 D1 D1

D2

D3 D3D3

ODM ODMODM

Level 0

Level 1

Level 2

S SensorA ActuatorODM Orthogonal Defense Mechanism

Fig 3 Communications structure of SWaT D1 D2 and D3 denote setsof defense mechanisms in SWaT ODM (Orthogonal Defense Mechanism) isindependent of these mechanisms

other techniques for anomalous process behavior [12] [24][27] D3 (WD) sits inside PLCs and implements a distributedattack detection mechanism that relies exclusively on processdynamics [1] The ODM has direct access to sensors and anal-yses the data received for the existence of process anomaly

C An illustrative attack on SWaT

Consider Stage 1 of SWaT in Figure 2 This stage has amotorized valve labeled MV101 which when open causeswater to flow into tank T101 The inflow into T101 is measuredby flow meter FIT101 and the water level by a level sensorlabeled LIT101 Pump P101 sends water to the next stageFlow meter FIT201 measures the outflow of water fromStage 1 to Stage 3 PLC1 receives the LIT101 reading and

controls the motorized valve MV101 Similarly PLC1 receivesLIT301 readings from PLC3 and controls pump P101

Tanks T101 and T301 have four markers each labeled Low(L) Low Low (LL) High (H) and High High (HH) Eachmarker corresponds to a specific value of water level in thetank These markers are used by the corresponding PLCs tocontrol the states of motorized valves and pumps Thus forexample when the water level in T101 reaches L PLC1 opensMV101 and closes it when the level reaches H When waterlevel in T301 reaches L PLC1 turns P101 ON and turns it OFFwhen the level reaches H The following example illustrates theimpact of compromising level sensor LIT101 with the intentof damaging pump P101

Example Consider an attack where the attackerrsquos intentionis to underflow T101 and damage P101 by making it runwithout any incoming water The attack is launched on LIT101with Stage 1 in the following state LIT301 955mm MV101Closed P101 OFF UF is operational and therefore waterlevel in tank T301 is decreasing Assume now that the attackersets LIT101 reading to a constant value of 790mm In thisattack even though the water level in T101 is changing(decreasing) PLC1 receives a constant value After a whilewhen LIT301 reaches L pump P101 is turned ON by PLC1However the actual water level in tank T101 is lower thanL say at LL This leads to the outflow from the pump beingreduced to less than the intended flow rate Pump P101 runsdry when there is no water in T101 and will eventually getdamaged unless a corrective action is taken

Figure 4 shows the water level in tank T101 during theattack It can be observed that the outflow increases graduallywhen the attack is removed Note that the sudden drop in thevalue of LIT101 soon after attack removal corresponds to thefact that the PLC begins to receive the correct measurement

Fig 4 Water level in tank T101 when LIT101 is attacked LIT101readings are observed by PLC1

Fig 5 Level sensor LIT101 is under attack The attackerrsquos intention isto underflow T101 tank and damage P101 The first arrow indicatesthe outflow reducing time second arrow indicates the pump noisestarting time

of water level in T101 When the water level goes down to150 mm tank T101 does not have enough water to send totank T301 Figure 5 shows the change in flow rate duringthe attack as measured by flow meter FIT201 The two arrowsindicate the start of reduction of outflow from T101 At around10 seconds there is no water flowing from P101 even thoughthe pump is ON At this point the pump becomes noisy and theflow rate reduces to zero If not removed this attack may leadto pump damage due to overheating Of course a mechanicalcut off at the pump would avoid such damage

The above example shows how an attacker could potentiallydamage a pump by changing the sensor values and actuatorstates More complex attacks mentioned in Section VI can bedesigned and launched to reduce the chances of being detected

III OVERVIEW OF WDWD is a mechanism to detect process anomalies A process

is considered anomalous when it deviates from its expectedbehavior WD detects such anomalies through the use of in-variants An invariant [4] is a condition among physical andorchemical properties of the process that must hold wheneveran ICS is in a given state At a given time instant sensormeasurements of a suitable set of such properties constitutethe observable state of the physical process as known to theICS

The invariants serve as checkers of the system state Theseare coded and the code placed inside each PLC used for attack

detection The checker code is added to the control code thatalready exists in each PLC The PLC executes the code in acyclic manner In each cycle data from the sensors is obtainedcontrol actions computed and applied when necessary and theinvariants checked against the state variables or otherwise Dis-tributing the attack detection code among various controllersadds to the scalability of the proposed method During S3 theimplementation was located inside the Programmable LogicControllers (PLCs) as well as embedded in the communicationnetwork

Two types of invariants were considered state dependent(SD) and state agnostic (SA) While both types use statesto define relationships that must hold the SA invariants areindependent of any state based guard while SD invariants areAn SD invariant is true when the plant is in a given state anSA invariant is always true

A State-Dependent (SD) invariants

Consider for example the case when the motorized valveMV101 is Open In this case the flow rate indicator FIT101must provide a non-zero reading to the PLC This phys-ical fact leads to the following state-dependent invariantMV101=Open =rArr FIT101lt δ where δ denotes a thresholdindicating flow Note that an SD invariant may include con-ditions from across the various stages of SWaT thus enablingdistributed detection of attacks Derivation of SD invariants isbased on the design of the ICS and is described in [4]

B State-Agnostic (SA) invariants

Under normal system operation an SA invariant mustalways be true regardless of the system state One SA invariantwas derived for each tank in SWaT to detect attacks that affectthe flow of water into and out of a tank These invariants arebased on the flow of water and water level in a tank andhence are identical in terms of the mathematical relationshipthat they capture

As an example of an SA invariant consider the water levelin a tank At time instant k+1 the water level in T101 dependson the level at time k and the inflow and outflow at instant kThis relationship is captured in the following idealized discretetime model of the tank

x(k + 1) = x(k) + α(ui(k)minus uo(k)) (1)

where ui(k) and uo(k) denote the inflow and outflow ratesat time k and α is a proportionality constant that convertsflow rate to change in level using the tank dimensions x(k)is the true state of the water level Let y(k) denote the sensormeasurement of the water level x(k) an estimate of the levelsensor reading and ε a threshold based on experimentationBased on Eqn 1 the statistics obtained experimentally andconverting the true states to their estimates the followinginvariant is derived to test whether or not the tank fillingprocess is anomalous

sumn

i=1|(x(i)minus y(i))|n

gt ε under attack (2)

le ε normal (3)

Fig 6 Invariant to detect anomalous behavior of LIT 101

IV SWAT SECURITY SHOWDOWN (S3)

This section presents details of the two S3 [36] eventsincluding guidelines and selected information on participantsIn S3 the attackers are challenged to realise concrete goals inSWaT Points earned by an attack team are weighted based onthe capabilities needed to launch the attack and the number ofdefence mechanisms successfully bypassed during the attackThe goal was to meet as many pre-defined challenges aspossible within the pre-allocated time

Information disclosed to the attack teams Technical detailson SWaT such as network architecture protocols and devicesused are released to the attackers one month prior to theirarrival for participation in the event Publicly available whitepapers on mechanisms deployed by the defence teams areshared with each attack team

Information disclosed to the defenders S3 organizers workedclosely with the defense teams to integrate their defence mech-anisms into SWaT Information about the normal operation ofSWaT was disclosed to the defenders to enable them to fine-tune their detection systems and reduce false alarms as muchas they could

Attacker profiles Attack teams were asked to select from aset of attacker profiles [34] The following attacker profileswere available cyber-criminal insider or a combination ofboth An attacker profile is intended to restrict availability ofresources and limit the access rights of the attackers as shownin Table I

A S3-2016

Attack teams included three from industry and three fromacademia Similarly there were three defense teams from theindustry and three from academia During the live phase heldat the SWaT testbed all six [35] defence mechanisms weresimultaneously in place Each team was given 12 hours forpassive reconnaissance and team was assigned a 3-hour slotduring which they were able to launch attacks

B S3-2017

Attack teams included one from industry and four fromacademia There were two defense teams from the industryand two from academia Each attack team was given two ses-sions [36] of four hours each to conduct reconnaissance on thetestbeds During these sessions various attacks were preparedand tested with the assistance of the SWaT laboratory engineerDuring the actual event each team was given two hours todemonstrate their attacks that were prepared previously Attack

TABLE IRESOURCES AND ACCESS RIGHTS FOR ATTACKER PROFILES

Profile Constraints

Cyber-criminal Limited number of attempts to realize a goal

Physical access not allowed manual manipulation ofthe sensors and actuators are not allowed

Direct connection to PLCs using any software suchas Allen Bradleyrsquos Studio5000 not allowed

Insider Physical access to SWaT allowed manual manipula-tion of the sensors and actuators are allowed

Allowed to alter the network topology

Direct connection to PLCs using any software suchas Allen Bradleyrsquos Studio5000 allowed

TABLE IITARGETS OF ATTACKS IN S3

Target Description

Physical Process Attacks

Valves Control the motorized valves

Pumps Disrupt pump control operations

Pressure Alter the pressure in pipes

Tank fill level Alter water level in a tank

Chemical dosing Alter chemical dosing

Sensor Data Attacks

Historian Alter data in the Historian

HMISCADA Alter the sensor actuator values at HMI orSCADA DoS Attacks on SCADA HMI

PLC Reprogram PLC DoS attacks on PLCsChange the commands and values in which thePLC receives and sends

RIODisplay Control of the RIO through disconnected ana-logue InputOutput pin

teams were also given a separate network for Internet up-linkand up to three Virtual Machines (VMs) running either Linuxor Windows operating system

C Attack targets

The attack teams were given a list of components andsubsystems in SWaT that could serve as the target of theirattacks Table II lists the targets available to the attack teamsTable II has two kinds of attacks physical process attacks andsensor data attacks In physical process attacks an attackerrsquosobjective is to alter the physical process In the case of sensordata attacks an attackerrsquos objective is to alter the sensor oractuator tags during communication or in the Historian

V PREPARATION FOR S3

To prepare for S3-2016 an earlier version of WD wasextended to all six stages of SWaT This extension requiredthe generation of invariants across all stages coding of theinvariants and placement of the code inside [1] the six PLCsThe modified WD was tested on SWaT by running the plantunder various operating conditions

Based on lessons learned during S3-2016 several newinvariants were generated coded and added to the PLCs ForS3-2017 we decided to use an additional monitoring systemplaced outside the PLCs This system collects data from theHistorian and evaluates the invariants All invariants wereimplemented in a Linux environment using a Piwebclient APIto talk to the Historian This new implementation is referredto as WDH

The invariants in WD are coded using ladder logic andstructured text while those in WDH in Python Both imple-mentations use the same set of invariants the difference is intheir placement The Historian may not get all the data andcommands that flow across the PLCs sensors and actuatorsHowever as WDH gets its data directly from the Historian ithas access to information flowing across SCADA workstationand the Historian This information may be compromised byan attacker and is not available to the PLC

A Scope of WD

WD is designed to detect process anomalies Thus anyabnormal behavior in the water treatment process in SWaTought to be detected by WD However there could be attacksthat do not cause the process to deviate from its normalbehavior but lead to undesirable consequences An exampleof such an attack is one intended to deface the screen on theSCADA workstation or the HMI Such an attack will not bedetected by WD Attacks that may cause process anomaly butonly after an attack has been removed from the system mayalso not be detected by WD Denial of Service is one suchattack

B Scope of WDH

WDH and WD use the same set of invariants Howeverthe placement of WDH could lead to a difference in detectioncapabilities of the two defense mechanisms WDH gets its datafrom Historian while WD directly from the PLC Data that isnot programmed to be logged in the Historian will not beaccessible to WDH Thus any anomaly that requires such datawill likely not be detected by WDH Similarly attacks thatmanipulate data entering the Historian or SCADA may not bevisible to WD Thus while the two invariant-based processanomaly detection mechanisms are identical in the invariantsthey use their placement in SWaT is expected to result indifferent performance in detecting attacks

VI S3 ATTACKS

The attacks launched by teams participating in the two S3

events are described next

A S3-2016 Attacks

All attacks designed and launched during S3-2016 areenumerated in Table III Three attacks selected from Table IIIare described next Details of all attacks are available in [9]Of the 18 attacks in Table III 4 and 16 are cyber criminalattacks and the remaining are insider attacks

DoS attack on SCADA In this attack (attack 4 in Table III)the attackerrsquos intention was to deface the SCADA workstationscreen and hence prevent the operator from observing plantstate The cyber-criminal attacker model was used to designthis attack To realize the intention the attacker launched anARP poisoning Man-in-the-Middle attack in two steps In thefirst step all traffic intended for HMI was redirected to theSCADA workstation In the second step this redirected trafficwas dropped and thus no packets were received at the SCADAworkstation This led to the screen on the workstation becom-ing completely gray and no state information was displayedThis attack was not detected by WD as it did not lead toany process anomaly It is an ARP spoofing attack and not atraditional DoS attack As part of the DoS attack the attackertargeted the PLC and sent millions of packets at a time Thisled to the same effect as would be the case when an ARPspoofing attack is performed on SCADA

Manipulation of the chemical dosing pump Intention of theattacker in this case (attack 14 in Table III) was to manipulatethe pH of water entering Stage 3 of SWaT The insider-attackermodel was used in the design of this attack This attack wasexecuted in two steps In the first step PLC 2 was set to manualmode Note that in manual mode the plant operator can directlycontrol the actuators eg the dosing pumps in this case In thesecond step the attacker altered the chemical dosing processin the Pre-treatment Stage 2 of SWaT by interacting directlywith the HMI interface and overriding the commands sent bythe PLC WD was able to detect this attack because the set-points changed by the attacker were different from those setin WD

DoS to PLC by SYN flooding The intention of the attackerin this case (attack 16 in Table III) was to disable the HMIso that an operator is unable to view or control the plantoperation The insider-attacker model was used in the designof this attack In this way the attacker had an access to theadministrator account and the associated tools The attackerperformed a SYN flooding attack on EthernetIP server ofPLC1

As a result of this DoS attack the HMI was unable toobtain the current state values to display and would insteaddisplay 0 or characters WD was unable to detect this attackphysical process as not affected During the attack period PLCwas controlling the process as expected Such attacks whilenot altering process behavior may impede supervision of theprocess in an operational plant

B S3-2017 Attacks

All attacks designed and launched during S3-2017 areenumerated in Table IV Selected attacks from Table III aredescribed next Details of all attacks are available in [20] Ofthe 31 attacks in Table IV 17 can be classified as cybercriminal attacks and the remaining as insider attacks (Figure I)All attacks launched during S3-2016 and S3-2017 are listedand categorized in Table V

TABLE IIIATTACKS LAUNCHED DURING S3-2016

SNO Target Method Attack Tool

1 Tank fill levelLIT101

Use HMI access Close MV101 and Stop P101 andP102

HMI

2 HMISCADA ARP spoofing Attack HMI DoS attack Ettercap

3 PLC Manual access Removed the cable at the ring atlevel 0

Manual

4 HMISCADA DoS on HMI by droppingall packets between PLC andSCADAHMI

DoS attack on SCADA wide DoSattack took a while to restore SWaTto its normal state

Ettercap

5 Tank fill levelLIT101

Use HMI access Attack on LIT101 ManualHMI

6 Valve MV301 Use SCADA access Attack on MV301 manually openfrom the SCAD workstation

ManualSCADA

7 Pump P101 Use SCADA access Attack pump manually open it fromthe SCADA workstation

ManualSCADA

8 Historian DoS attack using CPPPO andloop

Attack between HMI and PLC CPPPO

9 Valve MV101 Use SCADA access MV101 attacked using SCADAchanged the valve state from Opento Closed

ManualSCADA

10 Pump P101 Use SCADA access LIT301 set point changed ManualSCADA

11 Tank fill levelLIT301

Using SCADA access LIT301 set point altered ManualSCADA

12 Chemical dosingP201

Control MV101 and AIT503 setpoints of LIT301 to ensure flowthis triggered chemical dosing

Dosing pump attack on P201 ManualSCADA

13 HMISCADALIT101

Functional block introduce newconstant tag tie that to output tagcould only do zero

LIT101 set to zero from PLC Studio5000

14 Chemical dosingpump P205

Use SCADA access Manipulation of the chemical dosingpump (P205)

ManualSCADA

15 HMISCADA DoS on HMI using Level 1 net-work

Attack on HMI EttercapPycomm

16 Historian SYN flood ENIP port at PLC1 DoS to PLC by SYN flooding (attackon HMI)

Ettercap

17 Chemical dosingpump P203

HMI-based direct manipulation Attack on P203 while the four dosingpumps are running

ManualHMI

18 HMISCADALIT101

Re-program PLC to fix LIT101value to an arbitrary value

Attack on LIT101 Studio5000

416 are cyber criminal attacks in S3-2016

Control of the chemical dosing system through a Pythonscript (Pycomm) The objective of this attack (attack 15 inTable IV) was to change chemical dosing at the end of the de-chlorination system (Stage 4) First the attackers compromisedVirtual Network Computing (VNC) Then they used a Pythonscript (Pycomm) and Wireshark to gain access to the HMIAfter gaining access to the HMI through the compromisedVNC the cybercriminal attacker used Wireshark to capturethe packets flowing between the HMI and PLC4 The con-troller tags were retrieved by an analysis of the packets Theattackers changed the data associated with these tags to controlthe chemical dosing function using the Pycomm framework

Control of PLC through the Bridged Man-in-the-Middle(MiTM) at Level 0 the objective of this attack was (attack 16in Table IV) to change the commands and values that PLC1receives and sends First the attackers configured a bridgebetween the RIO and PLC1 using Netfilterqueue andScapy The attack was launched at two network levels Ananalysis on the network traffic revealed the packets that theattackers should edit As the target of this attack was thewater level in T101 the attackers set it to a constant valueto hide from PLC1 the rise in water level in T101 Before apacket was forwarded Netfilterqueue rerouted it into aqueue which can be read and modified by the Python script

TABLE IVATTACKS LAUNCHED DURING S3-2017

SNo Target Method Attack Tool

1 HMISCADA LIT401 HMI simulation insider attack Change the value of LIT401 in the HMI Manual HMI

2 Historian ARP and drop Change the value stored at the Historian Ettercap

3 Valve MV201 Reprogram PLC Change the status of the MV201 Studio 5000

4 Tank fill level LIT301 420to 320

Manual Lower the water tank level from 820mm to420mm without raising any alarm LIT301decreased till 320mm

Manual HMI

5 Pump P101 Manual mode of pump Alternate the state [OnOff] of the pump P101 Manual HMI

6 Chemical dosing P205 Manually dosing chemical pump Change the chemical dosage of sodiumhypochlorite (NaOCl) in P2

Manual SCADA

7 PLC Disconnect cable Disrupt sensor values from remote inputoutput(RIO) to the PLC

Manual

8 RIO Display Disconnect IO PIN manual Disrupt the sensor reading send to PLC throughRemote IO (RIO)

Manual

9 Chemical dosing P404 MiTM Python script to control Increase chemical dosage in pre-treatment Python script

10 LIT101 (476mm to 540mm ) Reprogram PLC Falsify water level display at SCADA Studio 5000

11 Pump P101 HMI simulation insider attack Alternate the state [OnOff] of the pump P101 Manual HMI

12 HMISCADA AIT 504 ARP+rewriting Increase AIT504 Ettercap

13 PLC LIT401 Reprogram PLC Falsify water level display at SCADA Studio 5000

14 RIODisplay Disconnect specific IO PIN basedon manual

Disrupt the sensor reading send to PLC throughremote IO (RIO)

Manual

15 Chemical dosing pumpP403 AIT501

Based on captured traffic betweenHMI and PLC4

Change chemical dosing function VNC Python script PycommWireshark

16 PLC LT101 from 742mm to500mm

Level 0 MITM Change the commands and values that the PLCreceives and sends

Aircrack Airodump AireplayNetfilterqueue Scapy

17 Historian LT101 tag Aircrack WiFi ARP spoofingEttercap

Compromise historian data Ettercap Aircrack

18 Pressure sensor DPIT30130MV301-4

SMB to EW get project files runFT

Disrupt valves operation of Ultrafiltration andBackwash (P3)

SMB

19 MV201 LT101 metasploit+vnc Change the water level of the tank LIT101 Metasploit+vnc

20 Pump P501 Rogue AP disassociated Telnetwith default credentials to turnoff original AP Scapy rewrite

Disrupt pump control operation KisMAC Password crackingtool 3vilTwinAttacker TelnetScapy

21 PLC LIT101 Reprogram PLC Change level indicator value Studio 5000

22 Pump P101 LIT301 Using back-door connection Establish back-door connection Mimikatz malicious VBAMacro SOCKS proxy

23 HMISCADA P201 Netfilterqueue Scapy Change the display value of the HMI Netfilterqueue Scapy

24 Historian LIT101 Overwrote specific data stored at the Historian Microsoft PsExec ipconfig

25 RIODisplay Manual Control of the RIO through disconnected Ana-logue InputOutput pin

Manual

26 Valve MV201 Manual Permanently closed the motorised valve re-gardless of commands issued

Manual SCADA

27 RIODisplay AIT202203 Manual Change the pH value shown at HMI Manual HMI

28 PLC MV201 P101 ARP poisoning MiTM Increase the pressure at P1 Ettercap

29 Tank fill level LIT101 Lower Falsify the water level reading of the tankdisplayed at SCADA

Pycomm

30 Chemical dosing PLC2 Use Studio 5000 Change the level of the chemical used fordosing

Studio 5000

31 Pressure MV302 P3012 Using Pycomm script Change the pump state sent to the PLC Pycomm

TABLE VCYBER CRIMINAL ATTACKS IN S3

Cyber Criminal Attacks Insider Attacks

S3-2016 4 16 1 2 3 5 6 7 8 910 11 12 13 14 1517 18

S3-2017 2 9 10 12 13 1516 17 18 19 20 2122 24 28 29 30

1 3 4 5 6 7 8 1114 23 25 26 27 31

To prevent all packets from entering the queue in order notto disrupt other processes iptables was used to identifythe targeted packets entering the queue Using Scapy and acustom dissector the attacker edited the payload of the targetedpacket which was then forwarded to its original destination

Control of Historian through the Aircrack WiFi The objectiveof this attack was (attack 17 in Table IV) to compromise thedata stored in the Historian Attackers performed crack WiFipassword ARP poisoning and MiTM payload manipulationusing Aircrack and Ettercap As PLC1 was operating inthe wireless mode the cybercriminal attacker used Aircrackto obtain the password for connecting to the ICS Access Point(AP) ARP poisoning was executed to reroute traffic betweenPLC1 and the Historian through the attackerrsquos rogue terminalThe attackers then used an Ettercap filter to manipulate thenetwork packets The attackers changed the tag correspondingto LIT101 to an arbitrary value before releasing the packetsto the Historian

Control of pressure through the Server Message Block (SMB)The objective of this attacks was (attack 18 in Table IV) todisrupt the state of four motorized valves in Stage 3 to affectthe differential pressure in UF Vulnerability CVE-2008-21601

in Factory Talk software from Rockwell and in MicrosoftrsquosServer Message Block (SMB) was used by the attackersto obtain files from the HMI As the HMI was runningWindows CE it has a vulnerability that allows an attackerrsquosterminal to execute arbitrary code on the HMI Thus theattackers were able to retrieve the files to create a copy ofthe workstation From the copied workstation the attackersmanually changed the state of the valves in Stage 3 suchthat the differential pressure across the UF unit as measuredby DPIT301 became dangerously high The attackers closedvalves MV301 MV302 and MV303 and opened MV304

Control of water level in the tank through the MetasploitVNC Scanner Objective of this attack was (attack 19 inTable IV) to change the water level in tank T101 The attackersused Metasploit VNC authentication None scanner to ob-tain access to the VNC server without password protection andto check for nodes running a VNC Server Once the scannerdetected the VNC Server running without any authenticationthe attackers penetrated into the server through a VNC Clientconnection As the VNC Server was hosting the HMI which

1httpswwwcvedetailscomcveCVE-2008-2160

controlled the ICS the attackers changed the simulation tagassociated with water level in T101

Control of a pump through a rogue router The objective ofthis attack (attack 20 in Table IV) was to disrupt the controlof pump P501 The attackers used Evil twin (rogue accesspoint) method using KisMAC a password cracking tool3vilTwinAttacker Telnet and Scapy The attackersused KisMAC to scan for wireless networks in the ICS Oncethe targeted wireless network was identified the attackers useddictionary attack to crack the password After the passwordwas cracked the attackers created a rogue wireless routerwith a similar SSID and configuration They then sent a de-authentication packet to disassociate PLC5 and the originalrouter The attackers used Telnet to log into the originalrouter and shut it down Scapy was then used to modify thepackets to turn the pump on

VII RESULTS

Tables VI and VII summarize the response of WD andWDH to the attacks launched during the two S3 events Recallthat both WD and WDH contain exactly the same set ofinvariants In WD the invariants are coded and placed insidethe PLCs whereas in WDH the invariants are coded and placedat the Historian WDH did not exist during S3-2016 and hencethe response of WDH is available only for attacks launchedduring S3-2017

A S3-2016 results

We note from Table VI that 10 out of 18 attacks weredetected immediately while the remaining eight attacks werenot detected Six of the eight undetected attacks did not leadto process anomaly during the observation period and hencedid not violate any invariant This outcome is expected as theinvariants in WD are designed to detect process anomaly

Consider attack 2 ARP spoofing in Table III This is aDoS attack on HMI It leads to defacing the screen on theHMI or displaying incorrect information thereby preventingan operator from knowing the actual plant state Howeverthe attack does not cause process anomaly and hence is notdetected as it does not violate any invariant Similar logic canbe used to explain why the other attacks in Table VI are notdetected

It is important to note that a DoS attack when given enoughtime to evolve and be launched at an appropriate state of theplant may impact physical process behavior In such a caseone or more invariants may detect the attack One such attackis 16 in Table VI This attack prevented the Historian fromreceiving data from PLC1 However if this attack was leftactive for a longer period it would prevent PLC1 from sendingappropriate commands to the actuators eg to MV101 orP101 In turn this would have led to process anomaly Notenough data is available to conclude with certainty whether ornot this attack would be detected by WD if active for sufficienttime

Two single point [2] attacks were not detected by WD Inone attack (attack 6 in Table III) the adversary altered the status

of valve MV301 Under normal circumstances this valve isopened during the backwash process However the attackeropened it when there was no backwash Hence the attackdid not affect the physical process except in changing thevalve status No invariant was violated due to this attackbecause the backwash process ie Stage 6 is not includedin this case study The second single point attack (attack 17 inTable III) was performed on chemical dosing pump P203 whilethe other pump P204 was running Note that under normalcircumstances only one of these two pumps is supposed to berunning while the other remains as a backup Subsequently theattacker shut down pump P204 This attack was not detectedbecause there were no invariants that related to the chemicalproperties of water

Although the overall performance of WD was below 100it did detect all attacks within its scope except two (attacks 6and 17 in Table III) as mentioned earlier

B S3-2017 results

Table VI indicates that 21 out of 31 attacks were detectedby WD while 24 out of 31 attacks were detected by WDHConsidering only the attacks within its scope as mentionedin Section V-A WD detected 21 out of 28 attacks (75)Similarly WDH detected 24 out of 31 attacks (7741) withinits scope mentioned in Section V-B Three attacks on theHistorian are not in the scope of WD All attack targets relatedto RIODisplay (in Table II and in Table IV) are not detectedby both WD and WDH This is because registers inside aPLC save the previous values received from the sensors andthe PLC continues to execute the control code The invariantsalso use the same values stored in the PLC registers and hencedo not raise an alert

In general PLCs send to the Historian via the SCADAworkstation the data received from the sensors When a PLCdoes not have updated values during the attack period it isobvious that the Historian also receives the same stale valuesThis is the reason why WDH also did not detect attacksrelated to RIODisplay Note that the RIODisplay attacks werelaunched and remained active only for a few seconds Duringthis period the PLC did not update the current sensor valuescoming through the RIO If the same attack is performed for alonger duration the PLC would update the data received fromthe sensors Doing so would likely lead to WD and WDHdetecting the RIO attacks

Attacks launched on the Historian were detected by WDHbut not by WD This variance is due to the fact that data inthese attacks is manipulated at the Historian Thus invariantsin a PLC do not have access to the manipulated data andhence the invariants in WD do not raise any alert All attackstargeting a PLC are detected by WD and WDH

WD Detection of physical process attacks All attacks onvalves pressure sensor and level sensors were detected Threeout of four attacks on the chemical dosing process pumpswere detected An example of a detected attack is when theattackers took control of pump P301 (attack 20 in Table IV)

TABLE VIPERFORMANCE OF WD AND WDH

S3-2016 S3-2017

WD WD WDH

Detected 1 5 7 910 11 1213 14 18

3 4 7 9 1011 12 13 1516 18 19 2021 22 23 2628 29 30 31

2 3 4 7 9 1011 12 13 15 1617 18 19 20 2122 23 24 26 2829 30 31

Not detected 2 3 4 68 15 1617

1 2 5 6 8 1417 24 25 27

1 5 6 8 14 2527

through a Python script (Pycomm) to raise the pressure in theUF unit measured by sensor DPIT301 to a dangerous levelWD immediately raised an alarm This invariant ensured thatpump P301 must be OFF when the pressure at DPIT301 wasabove a threshold During the attack the invariant was violatedas the pump was not turned off while DPIT301 indicatedreadings that were above the threshold Consequently an alarmwas raised immediately In certain cases multiple alarmswere raised due to the violation of one or more invariantsFor example when level sensor LIT101 was compromisedthe invariants corresponding to this sensor were violated andraised alarms

WD Detection of sensor data attack WD detected attacks onHMISCADA and PLC values because these attacks directlycompromised the physical processes These attacks eithercompromised chemical dosing water tank levels or pumpstatus through hacking of the HMISCADA or PLC Hencethe robustness of WD in detecting unusual physical processbehavior was found effective in these attacks On the otherhand WD was unable to detect insider attacks that pulled outRIO cables This is because WD triggers an alarm only whenthe invariants are violated Under normal circumstance for aperiod of time a PLC continues to execute its control codeand any invariant code based on the last known state andorvalues Thus the invariants located inside the PLCs are unableto observe this anomalous behavior

WDH Detection of physical process attacks WDH detected14 out of 16 physical process attacks

WDH Detection of sensor data attacks WDH detected theattacks on HMISCADA and PLC values because these attacksdirectly compromised the physical processes albeit with aslightly lower detection rate when compared with the rate ofdetecting physical process attacks As with WD WDH did notdetect any attack launched against the Remote IO by pullingthe cables that connect it to the corresponding PLC WDHfared better in the detection of attacks against the Historian asit was directly accessing data on the Historian server

If the Historian itself or data that is input to the Historian iscompromised WDH takes the decision based on the input itreceives A clever and powerful attacker can attack the physical

TABLE VIIRESULTS FROM S3 2017

Target of Attack Noofattacks

WD WDH

Physical Process Attacks

State of motorised valves 2 100 100

State of water pumps 4 75 75

Pressure in UF 2 100 100

Water tank level 4 100 100

Chemical dosing 4 75 75

Sensor Data Attacks

Data in historian 3 0 100

Data in HMISCADA 3 67 67

Tampering PLC communi-cations

5 100 100

Tampering Remote IO 4 0 0

Total Attacks 31 6774 7741

process and modify values entering the Historian and thusdeceive WDH In general such a situation may arise in allbehavioral intrusion detection systems where the detector takesthe decision based on incorrect input data

Indeed data that appears to be ldquolegitimaterdquo could lead theWDH into believing that there is nothing wrong with thephysical process though there actually is However doingso requires the attacker to continuously manipulate a largenumber of state variables For example consider an attackwhere the attacker turns a pump say P101 ON when it shouldbe OFF and (continually) sends the state of the pump as OFFto the Historian and the corresponding PLC If the pump isOFF then the level of the source and destination tanks must berespectively decreasing and increasing at rates determined bythe pump characteristics Creating ldquolegitimate-lookingrdquo datathus requires an attacker to manipulate several state variablesas explained next (a) Two state variables that correspondto tank levels Two sensors (in SWaT) measure these statevariables (see Figure 2) Thus the attacker must have accessto these level sensors (b) If pump P101 is actually ON whilethe Historian receives its state as OFF then FIT201 must showno flow Thus the attacker will also need to manipulate FIT201to avoid detection This argument can be carried forward tosubsequent stages to show that many sensors will need to bemanipulated by an attacker to ldquohiderdquo a simple attack such asldquochange the state of a pumprdquo In summary yes incorrect dataat the Historian could prevent detection though doing so wouldbe a significant challenge for the attacker due primarily to thedistributed nature of the invariants

VIII DISCUSSION

A Challenges faced

We faced several challenges during S3 For example aftereach teamrsquos performance the operator was required to bringSWaT back to a predefined normal state It was necessary to

keep SWaT in a normal state before another team launched at-tacks Bringing SWaT to its normal state required (a) resettingnetwork communications to ensure that all the communicationchannels are operating as expected (b) the operator to ensurethat all physical processes in SWaT are stable with respectto the control logic (c) the operator to bring back SWaT tothe normal state of that particular device such as a pump or amotorized valve in the case of any physical or manual attacksby the previous team and (d) that the Historian and SCADAservers were reverted to their original state ie the state thatexisted prior to the launch of attacks

B Research questions

RQ1 How do attackers compromise the security of an ICS InSection VI we presented and categorized the attacks based onattacker profiles An attacker can launch physical attacks wheninside the plant such as manually operating a motorized valveor tampering with network cabling Several attacks launchedby the attack teams had not been launched by the authorsin their evaluation of WD [1] and WDH Thus S3 raisedour confidence in the effectiveness of the attack detectionmechanisms based on invariants derived from plant designs

RQ2 How effective is WD in detecting attacks launched byindependent attack teams As mentioned earlier while bothWD and WDH were found to detect a number of attacksthey did fail in several cases Given that the invariants derivedare intended to detect process anomalies it is clear that suchmechanisms must be used in conjunction with other attackdetection tools such as those in [24] [27] [21]

C Assessment by the authors and by independent teamsTable VIII lists the number of attacks launched by the

authors in an experimental evaluation performed prior to S3-2016 [1] Note that the WD detection rate observed by theauthors (89) was higher than the combined rate observedduring the two S3 events (6326) The difference in perfor-mance is due to different attack vectors used in the three setsof experiments WDH detection rate observed during S3 eventis (7741) which is much higher than the WD detection rateSome of these attack vectors are explained in Section VI andthe remaining may be found in [20]

TABLE VIIIPERFORMANCE OF WD AS EVALUATED BY THE AUTHORS AGAINST THOSE

BY PARTICIPANTS IN S3

Experiments by Attacks

Launched Detected (WD) Detected (WDH)

Authors 37 33 (89) NA

S3-2016 18 10 (555) NA

S3-2017 31 21 (677) 24 (774)

NA WDH did not exist at the time of experimentation by the author andduring S3-2016

The data in Table VIII is indicative of the value of orga-nizing S3 events Specifically in the case described in this

paper the two S3 events led to an increased confidence inthe effectiveness of the invariant-based approach in detectingcyber attacks The hackfests also led to the creation of newtypes of attack vectors that were not used earlier to assess theperformance of WD and WDH in detecting cyber attacks

D False alarms

The performance of any attack detection method ought tobe assessed using its detection accuracy ie how many of thelaunched attacks it detects as well as the rate at which falsealarms are raised During S3 each team attempted to launchseveral attacks The attacks listed in Tables III and IV are theones that were successful in realizing the stated attacker intentand were scored by the judges The remaining attacks werenot recorded and hence any alarm generated by such attackswas not considered Some of these unrecorded alarms couldbe false though no specific claims can be made about theirnature

Since S3-2017 the authors have observed no false alarmsfrom WD during normal operation of SWaT WDH has beenin operation since a few weeks prior to S3-2017 Againduring the normal operation of SWaT no alarm has beengenerated by WDH This observation should not be construedto imply that an invariant-based attack detection mechanismwill not generate any false alarmndash in fact it could Howeverif the invariants generated are complete in the sense that theyaccurately capture all aspects of process behavior and theirimplementation is correct and tuned properly the likelihoodof false alarms is low

Even though SWaT is a relatively new plant (2-years sinceits inauguration at the time of writing this paper) we doobserve intermittent failures in a few motorized valves Forexample sometimes MV101 in Stage 1 takes much longer toopen than expected by its controlling PLC1 The PLC itselfdetects such cases In such a case WD or WDH dependingon the time it takes for the valve to finally open will raisean alarm We do not consider this as a false positive simplybecause whether an anomalous behavior is due to a naturalcause or a cyber attack cannot be distinguished by WD orWDH While such distinction is important to make additionalresearch is needed to distinguish process anomalies due tocyber attacks and those arising due to natural componentfailures

E Benefits of S3

S3 exposed the organisers participants and researchers tohow an attacker might design and launch attacks on ICS Bene-fits of S3 include the following 1) An improved understandingof how an ICS operates and the consequent formulation ofnew research directions 2) Opportunity for participants fromindustry and academia to learn from the event and focus onthe limitations of their work 3) An aid to the ICS managementteam to observe the defense teams thus leading to possibleadoption of technology embedded in WD or WDH

F Placement of WD

The placement of WD is another question that ought tobe looked into carefully In this work WD is placed insidePLCs However an exceptionally large number of invariantsmay prevent adding code to the existing control code in a PLCThis may happen due to the computational load requirementson a PLC This aspect led us to create WDH that is placedon the plant network and gets its data from the Historian toevaluate the invariants

G Forensics

One advantage of the invariant-based approach for attackdetection appears while determining the area of impact ofan attack When a single invariant is violated it indicatesclearly the source of process anomaly For example an alertis generated if valve MV101 is closed when the water intank T101 is at or below the L level marker While this alertdoes not indicate how an attacker entered the system or ifthe valve or the level sensor is defective it does assist inlocalising the reason for the alert The analysis becomes abit more complex when multiple invariants raise alerts Thisaspect of an invariant-based detection mechanisms remains tobe analyzed in further detail

H Attacker capabilities

We do not have any validation of the professionalism of theS3 attack teams As mentioned earlier [20] [35] [36] attackteams were from a variety of backgrounds including fromthe industry and academia from Europe and Asia During S3-2017 one team consisting of four membersndashall from outsideof Singaporendash focuses on ethical hacking and cyber-warsinvolving critical infrastructure This team is part of a globalalliance The other teams consist of hackers interested inknowing how vulnerabilities in software can be exploitedand passes this information to others for improving systemssecurity Coverage of attacks launched by the attack teams andattacker profiles is discussed in Section IV and summarizedin Tables I II V and VII

I Attack trees

It is possible to use attack trees [37] [42] to model attackslaunched during the two hackfests reported in this paper Doingso would enable mapping each attack to a specific path inthe attack tree and reveal which attack paths in SWaT weretraversed Such modeling and analysis has not been attemptedin this work and is a possible subject for future research

IX RELATED WORK

S3 is a Capture-The-Flag [15] event on ICS TraditionalCTF events generally attract the attention of both industrialand academic teams and currently enjoy increasing popularityas indicated in [15] The number of such events is graduallyincreasing [13] [16] Such events aid in learning about secu-rity vulnerabilities how these could be exploited nature ofattacks and strength of the deployed [18] [33] [45] defensemechanisms To the best of our knowledge S3 is the first CTF

style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed

The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks

There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc

The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies

Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS

X CONCLUSION

There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense

While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the

actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure

It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously

The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants

The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]

It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question

ACKNOWLEDGMENTS

A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript

REFERENCES

[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016

[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016

[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016

[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016

[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016

[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016

[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017

[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012

[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017

[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and

S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011

[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml

[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010

[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015

[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security

under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler

Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016

[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017

[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016

[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016

[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010

[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for

safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016

[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical

systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013

[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111

[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016

[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013

[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx

[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007

[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016

[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3

[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event

[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008

[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016

[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011

[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf

[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment

of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007

[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016

[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016

[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003

[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014

[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014

[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011

BIOGRAPHY

Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems

Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research

Fig 4 Water level in tank T101 when LIT101 is attacked LIT101readings are observed by PLC1

Fig 5 Level sensor LIT101 is under attack The attackerrsquos intention isto underflow T101 tank and damage P101 The first arrow indicatesthe outflow reducing time second arrow indicates the pump noisestarting time

of water level in T101 When the water level goes down to150 mm tank T101 does not have enough water to send totank T301 Figure 5 shows the change in flow rate duringthe attack as measured by flow meter FIT201 The two arrowsindicate the start of reduction of outflow from T101 At around10 seconds there is no water flowing from P101 even thoughthe pump is ON At this point the pump becomes noisy and theflow rate reduces to zero If not removed this attack may leadto pump damage due to overheating Of course a mechanicalcut off at the pump would avoid such damage

The above example shows how an attacker could potentiallydamage a pump by changing the sensor values and actuatorstates More complex attacks mentioned in Section VI can bedesigned and launched to reduce the chances of being detected

III OVERVIEW OF WDWD is a mechanism to detect process anomalies A process

is considered anomalous when it deviates from its expectedbehavior WD detects such anomalies through the use of in-variants An invariant [4] is a condition among physical andorchemical properties of the process that must hold wheneveran ICS is in a given state At a given time instant sensormeasurements of a suitable set of such properties constitutethe observable state of the physical process as known to theICS

The invariants serve as checkers of the system state Theseare coded and the code placed inside each PLC used for attack

detection The checker code is added to the control code thatalready exists in each PLC The PLC executes the code in acyclic manner In each cycle data from the sensors is obtainedcontrol actions computed and applied when necessary and theinvariants checked against the state variables or otherwise Dis-tributing the attack detection code among various controllersadds to the scalability of the proposed method During S3 theimplementation was located inside the Programmable LogicControllers (PLCs) as well as embedded in the communicationnetwork

Two types of invariants were considered state dependent(SD) and state agnostic (SA) While both types use statesto define relationships that must hold the SA invariants areindependent of any state based guard while SD invariants areAn SD invariant is true when the plant is in a given state anSA invariant is always true

A State-Dependent (SD) invariants

Consider for example the case when the motorized valveMV101 is Open In this case the flow rate indicator FIT101must provide a non-zero reading to the PLC This phys-ical fact leads to the following state-dependent invariantMV101=Open =rArr FIT101lt δ where δ denotes a thresholdindicating flow Note that an SD invariant may include con-ditions from across the various stages of SWaT thus enablingdistributed detection of attacks Derivation of SD invariants isbased on the design of the ICS and is described in [4]

B State-Agnostic (SA) invariants

Under normal system operation an SA invariant mustalways be true regardless of the system state One SA invariantwas derived for each tank in SWaT to detect attacks that affectthe flow of water into and out of a tank These invariants arebased on the flow of water and water level in a tank andhence are identical in terms of the mathematical relationshipthat they capture

As an example of an SA invariant consider the water levelin a tank At time instant k+1 the water level in T101 dependson the level at time k and the inflow and outflow at instant kThis relationship is captured in the following idealized discretetime model of the tank

x(k + 1) = x(k) + α(ui(k)minus uo(k)) (1)

where ui(k) and uo(k) denote the inflow and outflow ratesat time k and α is a proportionality constant that convertsflow rate to change in level using the tank dimensions x(k)is the true state of the water level Let y(k) denote the sensormeasurement of the water level x(k) an estimate of the levelsensor reading and ε a threshold based on experimentationBased on Eqn 1 the statistics obtained experimentally andconverting the true states to their estimates the followinginvariant is derived to test whether or not the tank fillingprocess is anomalous

sumn

i=1|(x(i)minus y(i))|n

gt ε under attack (2)

le ε normal (3)

Fig 6 Invariant to detect anomalous behavior of LIT 101

IV SWAT SECURITY SHOWDOWN (S3)

This section presents details of the two S3 [36] eventsincluding guidelines and selected information on participantsIn S3 the attackers are challenged to realise concrete goals inSWaT Points earned by an attack team are weighted based onthe capabilities needed to launch the attack and the number ofdefence mechanisms successfully bypassed during the attackThe goal was to meet as many pre-defined challenges aspossible within the pre-allocated time

Information disclosed to the attack teams Technical detailson SWaT such as network architecture protocols and devicesused are released to the attackers one month prior to theirarrival for participation in the event Publicly available whitepapers on mechanisms deployed by the defence teams areshared with each attack team

Information disclosed to the defenders S3 organizers workedclosely with the defense teams to integrate their defence mech-anisms into SWaT Information about the normal operation ofSWaT was disclosed to the defenders to enable them to fine-tune their detection systems and reduce false alarms as muchas they could

Attacker profiles Attack teams were asked to select from aset of attacker profiles [34] The following attacker profileswere available cyber-criminal insider or a combination ofboth An attacker profile is intended to restrict availability ofresources and limit the access rights of the attackers as shownin Table I

A S3-2016

Attack teams included three from industry and three fromacademia Similarly there were three defense teams from theindustry and three from academia During the live phase heldat the SWaT testbed all six [35] defence mechanisms weresimultaneously in place Each team was given 12 hours forpassive reconnaissance and team was assigned a 3-hour slotduring which they were able to launch attacks

B S3-2017

Attack teams included one from industry and four fromacademia There were two defense teams from the industryand two from academia Each attack team was given two ses-sions [36] of four hours each to conduct reconnaissance on thetestbeds During these sessions various attacks were preparedand tested with the assistance of the SWaT laboratory engineerDuring the actual event each team was given two hours todemonstrate their attacks that were prepared previously Attack

TABLE IRESOURCES AND ACCESS RIGHTS FOR ATTACKER PROFILES

Profile Constraints

Cyber-criminal Limited number of attempts to realize a goal

Physical access not allowed manual manipulation ofthe sensors and actuators are not allowed

Direct connection to PLCs using any software suchas Allen Bradleyrsquos Studio5000 not allowed

Insider Physical access to SWaT allowed manual manipula-tion of the sensors and actuators are allowed

Allowed to alter the network topology

Direct connection to PLCs using any software suchas Allen Bradleyrsquos Studio5000 allowed

TABLE IITARGETS OF ATTACKS IN S3

Target Description

Physical Process Attacks

Valves Control the motorized valves

Pumps Disrupt pump control operations

Pressure Alter the pressure in pipes

Tank fill level Alter water level in a tank

Chemical dosing Alter chemical dosing

Sensor Data Attacks

Historian Alter data in the Historian

HMISCADA Alter the sensor actuator values at HMI orSCADA DoS Attacks on SCADA HMI

PLC Reprogram PLC DoS attacks on PLCsChange the commands and values in which thePLC receives and sends

RIODisplay Control of the RIO through disconnected ana-logue InputOutput pin

teams were also given a separate network for Internet up-linkand up to three Virtual Machines (VMs) running either Linuxor Windows operating system

C Attack targets

The attack teams were given a list of components andsubsystems in SWaT that could serve as the target of theirattacks Table II lists the targets available to the attack teamsTable II has two kinds of attacks physical process attacks andsensor data attacks In physical process attacks an attackerrsquosobjective is to alter the physical process In the case of sensordata attacks an attackerrsquos objective is to alter the sensor oractuator tags during communication or in the Historian

V PREPARATION FOR S3

To prepare for S3-2016 an earlier version of WD wasextended to all six stages of SWaT This extension requiredthe generation of invariants across all stages coding of theinvariants and placement of the code inside [1] the six PLCsThe modified WD was tested on SWaT by running the plantunder various operating conditions

Based on lessons learned during S3-2016 several newinvariants were generated coded and added to the PLCs ForS3-2017 we decided to use an additional monitoring systemplaced outside the PLCs This system collects data from theHistorian and evaluates the invariants All invariants wereimplemented in a Linux environment using a Piwebclient APIto talk to the Historian This new implementation is referredto as WDH

The invariants in WD are coded using ladder logic andstructured text while those in WDH in Python Both imple-mentations use the same set of invariants the difference is intheir placement The Historian may not get all the data andcommands that flow across the PLCs sensors and actuatorsHowever as WDH gets its data directly from the Historian ithas access to information flowing across SCADA workstationand the Historian This information may be compromised byan attacker and is not available to the PLC

A Scope of WD

WD is designed to detect process anomalies Thus anyabnormal behavior in the water treatment process in SWaTought to be detected by WD However there could be attacksthat do not cause the process to deviate from its normalbehavior but lead to undesirable consequences An exampleof such an attack is one intended to deface the screen on theSCADA workstation or the HMI Such an attack will not bedetected by WD Attacks that may cause process anomaly butonly after an attack has been removed from the system mayalso not be detected by WD Denial of Service is one suchattack

B Scope of WDH

WDH and WD use the same set of invariants Howeverthe placement of WDH could lead to a difference in detectioncapabilities of the two defense mechanisms WDH gets its datafrom Historian while WD directly from the PLC Data that isnot programmed to be logged in the Historian will not beaccessible to WDH Thus any anomaly that requires such datawill likely not be detected by WDH Similarly attacks thatmanipulate data entering the Historian or SCADA may not bevisible to WD Thus while the two invariant-based processanomaly detection mechanisms are identical in the invariantsthey use their placement in SWaT is expected to result indifferent performance in detecting attacks

VI S3 ATTACKS

The attacks launched by teams participating in the two S3

events are described next

A S3-2016 Attacks

All attacks designed and launched during S3-2016 areenumerated in Table III Three attacks selected from Table IIIare described next Details of all attacks are available in [9]Of the 18 attacks in Table III 4 and 16 are cyber criminalattacks and the remaining are insider attacks

DoS attack on SCADA In this attack (attack 4 in Table III)the attackerrsquos intention was to deface the SCADA workstationscreen and hence prevent the operator from observing plantstate The cyber-criminal attacker model was used to designthis attack To realize the intention the attacker launched anARP poisoning Man-in-the-Middle attack in two steps In thefirst step all traffic intended for HMI was redirected to theSCADA workstation In the second step this redirected trafficwas dropped and thus no packets were received at the SCADAworkstation This led to the screen on the workstation becom-ing completely gray and no state information was displayedThis attack was not detected by WD as it did not lead toany process anomaly It is an ARP spoofing attack and not atraditional DoS attack As part of the DoS attack the attackertargeted the PLC and sent millions of packets at a time Thisled to the same effect as would be the case when an ARPspoofing attack is performed on SCADA

Manipulation of the chemical dosing pump Intention of theattacker in this case (attack 14 in Table III) was to manipulatethe pH of water entering Stage 3 of SWaT The insider-attackermodel was used in the design of this attack This attack wasexecuted in two steps In the first step PLC 2 was set to manualmode Note that in manual mode the plant operator can directlycontrol the actuators eg the dosing pumps in this case In thesecond step the attacker altered the chemical dosing processin the Pre-treatment Stage 2 of SWaT by interacting directlywith the HMI interface and overriding the commands sent bythe PLC WD was able to detect this attack because the set-points changed by the attacker were different from those setin WD

DoS to PLC by SYN flooding The intention of the attackerin this case (attack 16 in Table III) was to disable the HMIso that an operator is unable to view or control the plantoperation The insider-attacker model was used in the designof this attack In this way the attacker had an access to theadministrator account and the associated tools The attackerperformed a SYN flooding attack on EthernetIP server ofPLC1

As a result of this DoS attack the HMI was unable toobtain the current state values to display and would insteaddisplay 0 or characters WD was unable to detect this attackphysical process as not affected During the attack period PLCwas controlling the process as expected Such attacks whilenot altering process behavior may impede supervision of theprocess in an operational plant

B S3-2017 Attacks

All attacks designed and launched during S3-2017 areenumerated in Table IV Selected attacks from Table III aredescribed next Details of all attacks are available in [20] Ofthe 31 attacks in Table IV 17 can be classified as cybercriminal attacks and the remaining as insider attacks (Figure I)All attacks launched during S3-2016 and S3-2017 are listedand categorized in Table V

TABLE IIIATTACKS LAUNCHED DURING S3-2016

SNO Target Method Attack Tool

1 Tank fill levelLIT101

Use HMI access Close MV101 and Stop P101 andP102

HMI

2 HMISCADA ARP spoofing Attack HMI DoS attack Ettercap

3 PLC Manual access Removed the cable at the ring atlevel 0

Manual

4 HMISCADA DoS on HMI by droppingall packets between PLC andSCADAHMI

DoS attack on SCADA wide DoSattack took a while to restore SWaTto its normal state

Ettercap

5 Tank fill levelLIT101

Use HMI access Attack on LIT101 ManualHMI

6 Valve MV301 Use SCADA access Attack on MV301 manually openfrom the SCAD workstation

ManualSCADA

7 Pump P101 Use SCADA access Attack pump manually open it fromthe SCADA workstation

ManualSCADA

8 Historian DoS attack using CPPPO andloop

Attack between HMI and PLC CPPPO

9 Valve MV101 Use SCADA access MV101 attacked using SCADAchanged the valve state from Opento Closed

ManualSCADA

10 Pump P101 Use SCADA access LIT301 set point changed ManualSCADA

11 Tank fill levelLIT301

Using SCADA access LIT301 set point altered ManualSCADA

12 Chemical dosingP201

Control MV101 and AIT503 setpoints of LIT301 to ensure flowthis triggered chemical dosing

Dosing pump attack on P201 ManualSCADA

13 HMISCADALIT101

Functional block introduce newconstant tag tie that to output tagcould only do zero

LIT101 set to zero from PLC Studio5000

14 Chemical dosingpump P205

Use SCADA access Manipulation of the chemical dosingpump (P205)

ManualSCADA

15 HMISCADA DoS on HMI using Level 1 net-work

Attack on HMI EttercapPycomm

16 Historian SYN flood ENIP port at PLC1 DoS to PLC by SYN flooding (attackon HMI)

Ettercap

17 Chemical dosingpump P203

HMI-based direct manipulation Attack on P203 while the four dosingpumps are running

ManualHMI

18 HMISCADALIT101

Re-program PLC to fix LIT101value to an arbitrary value

Attack on LIT101 Studio5000

416 are cyber criminal attacks in S3-2016

Control of the chemical dosing system through a Pythonscript (Pycomm) The objective of this attack (attack 15 inTable IV) was to change chemical dosing at the end of the de-chlorination system (Stage 4) First the attackers compromisedVirtual Network Computing (VNC) Then they used a Pythonscript (Pycomm) and Wireshark to gain access to the HMIAfter gaining access to the HMI through the compromisedVNC the cybercriminal attacker used Wireshark to capturethe packets flowing between the HMI and PLC4 The con-troller tags were retrieved by an analysis of the packets Theattackers changed the data associated with these tags to controlthe chemical dosing function using the Pycomm framework

Control of PLC through the Bridged Man-in-the-Middle(MiTM) at Level 0 the objective of this attack was (attack 16in Table IV) to change the commands and values that PLC1receives and sends First the attackers configured a bridgebetween the RIO and PLC1 using Netfilterqueue andScapy The attack was launched at two network levels Ananalysis on the network traffic revealed the packets that theattackers should edit As the target of this attack was thewater level in T101 the attackers set it to a constant valueto hide from PLC1 the rise in water level in T101 Before apacket was forwarded Netfilterqueue rerouted it into aqueue which can be read and modified by the Python script

TABLE IVATTACKS LAUNCHED DURING S3-2017

SNo Target Method Attack Tool

1 HMISCADA LIT401 HMI simulation insider attack Change the value of LIT401 in the HMI Manual HMI

2 Historian ARP and drop Change the value stored at the Historian Ettercap

3 Valve MV201 Reprogram PLC Change the status of the MV201 Studio 5000

4 Tank fill level LIT301 420to 320

Manual Lower the water tank level from 820mm to420mm without raising any alarm LIT301decreased till 320mm

Manual HMI

5 Pump P101 Manual mode of pump Alternate the state [OnOff] of the pump P101 Manual HMI

6 Chemical dosing P205 Manually dosing chemical pump Change the chemical dosage of sodiumhypochlorite (NaOCl) in P2

Manual SCADA

7 PLC Disconnect cable Disrupt sensor values from remote inputoutput(RIO) to the PLC

Manual

8 RIO Display Disconnect IO PIN manual Disrupt the sensor reading send to PLC throughRemote IO (RIO)

Manual

9 Chemical dosing P404 MiTM Python script to control Increase chemical dosage in pre-treatment Python script

10 LIT101 (476mm to 540mm ) Reprogram PLC Falsify water level display at SCADA Studio 5000

11 Pump P101 HMI simulation insider attack Alternate the state [OnOff] of the pump P101 Manual HMI

12 HMISCADA AIT 504 ARP+rewriting Increase AIT504 Ettercap

13 PLC LIT401 Reprogram PLC Falsify water level display at SCADA Studio 5000

14 RIODisplay Disconnect specific IO PIN basedon manual

Disrupt the sensor reading send to PLC throughremote IO (RIO)

Manual

15 Chemical dosing pumpP403 AIT501

Based on captured traffic betweenHMI and PLC4

Change chemical dosing function VNC Python script PycommWireshark

16 PLC LT101 from 742mm to500mm

Level 0 MITM Change the commands and values that the PLCreceives and sends

Aircrack Airodump AireplayNetfilterqueue Scapy

17 Historian LT101 tag Aircrack WiFi ARP spoofingEttercap

Compromise historian data Ettercap Aircrack

18 Pressure sensor DPIT30130MV301-4

SMB to EW get project files runFT

Disrupt valves operation of Ultrafiltration andBackwash (P3)

SMB

19 MV201 LT101 metasploit+vnc Change the water level of the tank LIT101 Metasploit+vnc

20 Pump P501 Rogue AP disassociated Telnetwith default credentials to turnoff original AP Scapy rewrite

Disrupt pump control operation KisMAC Password crackingtool 3vilTwinAttacker TelnetScapy

21 PLC LIT101 Reprogram PLC Change level indicator value Studio 5000

22 Pump P101 LIT301 Using back-door connection Establish back-door connection Mimikatz malicious VBAMacro SOCKS proxy

23 HMISCADA P201 Netfilterqueue Scapy Change the display value of the HMI Netfilterqueue Scapy

24 Historian LIT101 Overwrote specific data stored at the Historian Microsoft PsExec ipconfig

25 RIODisplay Manual Control of the RIO through disconnected Ana-logue InputOutput pin

Manual

26 Valve MV201 Manual Permanently closed the motorised valve re-gardless of commands issued

Manual SCADA

27 RIODisplay AIT202203 Manual Change the pH value shown at HMI Manual HMI

28 PLC MV201 P101 ARP poisoning MiTM Increase the pressure at P1 Ettercap

29 Tank fill level LIT101 Lower Falsify the water level reading of the tankdisplayed at SCADA

Pycomm

30 Chemical dosing PLC2 Use Studio 5000 Change the level of the chemical used fordosing

Studio 5000

31 Pressure MV302 P3012 Using Pycomm script Change the pump state sent to the PLC Pycomm

TABLE VCYBER CRIMINAL ATTACKS IN S3

Cyber Criminal Attacks Insider Attacks

S3-2016 4 16 1 2 3 5 6 7 8 910 11 12 13 14 1517 18

S3-2017 2 9 10 12 13 1516 17 18 19 20 2122 24 28 29 30

1 3 4 5 6 7 8 1114 23 25 26 27 31

To prevent all packets from entering the queue in order notto disrupt other processes iptables was used to identifythe targeted packets entering the queue Using Scapy and acustom dissector the attacker edited the payload of the targetedpacket which was then forwarded to its original destination

Control of Historian through the Aircrack WiFi The objectiveof this attack was (attack 17 in Table IV) to compromise thedata stored in the Historian Attackers performed crack WiFipassword ARP poisoning and MiTM payload manipulationusing Aircrack and Ettercap As PLC1 was operating inthe wireless mode the cybercriminal attacker used Aircrackto obtain the password for connecting to the ICS Access Point(AP) ARP poisoning was executed to reroute traffic betweenPLC1 and the Historian through the attackerrsquos rogue terminalThe attackers then used an Ettercap filter to manipulate thenetwork packets The attackers changed the tag correspondingto LIT101 to an arbitrary value before releasing the packetsto the Historian

Control of pressure through the Server Message Block (SMB)The objective of this attacks was (attack 18 in Table IV) todisrupt the state of four motorized valves in Stage 3 to affectthe differential pressure in UF Vulnerability CVE-2008-21601

in Factory Talk software from Rockwell and in MicrosoftrsquosServer Message Block (SMB) was used by the attackersto obtain files from the HMI As the HMI was runningWindows CE it has a vulnerability that allows an attackerrsquosterminal to execute arbitrary code on the HMI Thus theattackers were able to retrieve the files to create a copy ofthe workstation From the copied workstation the attackersmanually changed the state of the valves in Stage 3 suchthat the differential pressure across the UF unit as measuredby DPIT301 became dangerously high The attackers closedvalves MV301 MV302 and MV303 and opened MV304

Control of water level in the tank through the MetasploitVNC Scanner Objective of this attack was (attack 19 inTable IV) to change the water level in tank T101 The attackersused Metasploit VNC authentication None scanner to ob-tain access to the VNC server without password protection andto check for nodes running a VNC Server Once the scannerdetected the VNC Server running without any authenticationthe attackers penetrated into the server through a VNC Clientconnection As the VNC Server was hosting the HMI which

1httpswwwcvedetailscomcveCVE-2008-2160

controlled the ICS the attackers changed the simulation tagassociated with water level in T101

Control of a pump through a rogue router The objective ofthis attack (attack 20 in Table IV) was to disrupt the controlof pump P501 The attackers used Evil twin (rogue accesspoint) method using KisMAC a password cracking tool3vilTwinAttacker Telnet and Scapy The attackersused KisMAC to scan for wireless networks in the ICS Oncethe targeted wireless network was identified the attackers useddictionary attack to crack the password After the passwordwas cracked the attackers created a rogue wireless routerwith a similar SSID and configuration They then sent a de-authentication packet to disassociate PLC5 and the originalrouter The attackers used Telnet to log into the originalrouter and shut it down Scapy was then used to modify thepackets to turn the pump on

VII RESULTS

Tables VI and VII summarize the response of WD andWDH to the attacks launched during the two S3 events Recallthat both WD and WDH contain exactly the same set ofinvariants In WD the invariants are coded and placed insidethe PLCs whereas in WDH the invariants are coded and placedat the Historian WDH did not exist during S3-2016 and hencethe response of WDH is available only for attacks launchedduring S3-2017

A S3-2016 results

We note from Table VI that 10 out of 18 attacks weredetected immediately while the remaining eight attacks werenot detected Six of the eight undetected attacks did not leadto process anomaly during the observation period and hencedid not violate any invariant This outcome is expected as theinvariants in WD are designed to detect process anomaly

Consider attack 2 ARP spoofing in Table III This is aDoS attack on HMI It leads to defacing the screen on theHMI or displaying incorrect information thereby preventingan operator from knowing the actual plant state Howeverthe attack does not cause process anomaly and hence is notdetected as it does not violate any invariant Similar logic canbe used to explain why the other attacks in Table VI are notdetected

It is important to note that a DoS attack when given enoughtime to evolve and be launched at an appropriate state of theplant may impact physical process behavior In such a caseone or more invariants may detect the attack One such attackis 16 in Table VI This attack prevented the Historian fromreceiving data from PLC1 However if this attack was leftactive for a longer period it would prevent PLC1 from sendingappropriate commands to the actuators eg to MV101 orP101 In turn this would have led to process anomaly Notenough data is available to conclude with certainty whether ornot this attack would be detected by WD if active for sufficienttime

Two single point [2] attacks were not detected by WD Inone attack (attack 6 in Table III) the adversary altered the status

of valve MV301 Under normal circumstances this valve isopened during the backwash process However the attackeropened it when there was no backwash Hence the attackdid not affect the physical process except in changing thevalve status No invariant was violated due to this attackbecause the backwash process ie Stage 6 is not includedin this case study The second single point attack (attack 17 inTable III) was performed on chemical dosing pump P203 whilethe other pump P204 was running Note that under normalcircumstances only one of these two pumps is supposed to berunning while the other remains as a backup Subsequently theattacker shut down pump P204 This attack was not detectedbecause there were no invariants that related to the chemicalproperties of water

Although the overall performance of WD was below 100it did detect all attacks within its scope except two (attacks 6and 17 in Table III) as mentioned earlier

B S3-2017 results

Table VI indicates that 21 out of 31 attacks were detectedby WD while 24 out of 31 attacks were detected by WDHConsidering only the attacks within its scope as mentionedin Section V-A WD detected 21 out of 28 attacks (75)Similarly WDH detected 24 out of 31 attacks (7741) withinits scope mentioned in Section V-B Three attacks on theHistorian are not in the scope of WD All attack targets relatedto RIODisplay (in Table II and in Table IV) are not detectedby both WD and WDH This is because registers inside aPLC save the previous values received from the sensors andthe PLC continues to execute the control code The invariantsalso use the same values stored in the PLC registers and hencedo not raise an alert

In general PLCs send to the Historian via the SCADAworkstation the data received from the sensors When a PLCdoes not have updated values during the attack period it isobvious that the Historian also receives the same stale valuesThis is the reason why WDH also did not detect attacksrelated to RIODisplay Note that the RIODisplay attacks werelaunched and remained active only for a few seconds Duringthis period the PLC did not update the current sensor valuescoming through the RIO If the same attack is performed for alonger duration the PLC would update the data received fromthe sensors Doing so would likely lead to WD and WDHdetecting the RIO attacks

Attacks launched on the Historian were detected by WDHbut not by WD This variance is due to the fact that data inthese attacks is manipulated at the Historian Thus invariantsin a PLC do not have access to the manipulated data andhence the invariants in WD do not raise any alert All attackstargeting a PLC are detected by WD and WDH

WD Detection of physical process attacks All attacks onvalves pressure sensor and level sensors were detected Threeout of four attacks on the chemical dosing process pumpswere detected An example of a detected attack is when theattackers took control of pump P301 (attack 20 in Table IV)

TABLE VIPERFORMANCE OF WD AND WDH

S3-2016 S3-2017

WD WD WDH

Detected 1 5 7 910 11 1213 14 18

3 4 7 9 1011 12 13 1516 18 19 2021 22 23 2628 29 30 31

2 3 4 7 9 1011 12 13 15 1617 18 19 20 2122 23 24 26 2829 30 31

Not detected 2 3 4 68 15 1617

1 2 5 6 8 1417 24 25 27

1 5 6 8 14 2527

through a Python script (Pycomm) to raise the pressure in theUF unit measured by sensor DPIT301 to a dangerous levelWD immediately raised an alarm This invariant ensured thatpump P301 must be OFF when the pressure at DPIT301 wasabove a threshold During the attack the invariant was violatedas the pump was not turned off while DPIT301 indicatedreadings that were above the threshold Consequently an alarmwas raised immediately In certain cases multiple alarmswere raised due to the violation of one or more invariantsFor example when level sensor LIT101 was compromisedthe invariants corresponding to this sensor were violated andraised alarms

WD Detection of sensor data attack WD detected attacks onHMISCADA and PLC values because these attacks directlycompromised the physical processes These attacks eithercompromised chemical dosing water tank levels or pumpstatus through hacking of the HMISCADA or PLC Hencethe robustness of WD in detecting unusual physical processbehavior was found effective in these attacks On the otherhand WD was unable to detect insider attacks that pulled outRIO cables This is because WD triggers an alarm only whenthe invariants are violated Under normal circumstance for aperiod of time a PLC continues to execute its control codeand any invariant code based on the last known state andorvalues Thus the invariants located inside the PLCs are unableto observe this anomalous behavior

WDH Detection of physical process attacks WDH detected14 out of 16 physical process attacks

WDH Detection of sensor data attacks WDH detected theattacks on HMISCADA and PLC values because these attacksdirectly compromised the physical processes albeit with aslightly lower detection rate when compared with the rate ofdetecting physical process attacks As with WD WDH did notdetect any attack launched against the Remote IO by pullingthe cables that connect it to the corresponding PLC WDHfared better in the detection of attacks against the Historian asit was directly accessing data on the Historian server

If the Historian itself or data that is input to the Historian iscompromised WDH takes the decision based on the input itreceives A clever and powerful attacker can attack the physical

TABLE VIIRESULTS FROM S3 2017

Target of Attack Noofattacks

WD WDH

Physical Process Attacks

State of motorised valves 2 100 100

State of water pumps 4 75 75

Pressure in UF 2 100 100

Water tank level 4 100 100

Chemical dosing 4 75 75

Sensor Data Attacks

Data in historian 3 0 100

Data in HMISCADA 3 67 67

Tampering PLC communi-cations

5 100 100

Tampering Remote IO 4 0 0

Total Attacks 31 6774 7741

process and modify values entering the Historian and thusdeceive WDH In general such a situation may arise in allbehavioral intrusion detection systems where the detector takesthe decision based on incorrect input data

Indeed data that appears to be ldquolegitimaterdquo could lead theWDH into believing that there is nothing wrong with thephysical process though there actually is However doingso requires the attacker to continuously manipulate a largenumber of state variables For example consider an attackwhere the attacker turns a pump say P101 ON when it shouldbe OFF and (continually) sends the state of the pump as OFFto the Historian and the corresponding PLC If the pump isOFF then the level of the source and destination tanks must berespectively decreasing and increasing at rates determined bythe pump characteristics Creating ldquolegitimate-lookingrdquo datathus requires an attacker to manipulate several state variablesas explained next (a) Two state variables that correspondto tank levels Two sensors (in SWaT) measure these statevariables (see Figure 2) Thus the attacker must have accessto these level sensors (b) If pump P101 is actually ON whilethe Historian receives its state as OFF then FIT201 must showno flow Thus the attacker will also need to manipulate FIT201to avoid detection This argument can be carried forward tosubsequent stages to show that many sensors will need to bemanipulated by an attacker to ldquohiderdquo a simple attack such asldquochange the state of a pumprdquo In summary yes incorrect dataat the Historian could prevent detection though doing so wouldbe a significant challenge for the attacker due primarily to thedistributed nature of the invariants

VIII DISCUSSION

A Challenges faced

We faced several challenges during S3 For example aftereach teamrsquos performance the operator was required to bringSWaT back to a predefined normal state It was necessary to

keep SWaT in a normal state before another team launched at-tacks Bringing SWaT to its normal state required (a) resettingnetwork communications to ensure that all the communicationchannels are operating as expected (b) the operator to ensurethat all physical processes in SWaT are stable with respectto the control logic (c) the operator to bring back SWaT tothe normal state of that particular device such as a pump or amotorized valve in the case of any physical or manual attacksby the previous team and (d) that the Historian and SCADAservers were reverted to their original state ie the state thatexisted prior to the launch of attacks

B Research questions

RQ1 How do attackers compromise the security of an ICS InSection VI we presented and categorized the attacks based onattacker profiles An attacker can launch physical attacks wheninside the plant such as manually operating a motorized valveor tampering with network cabling Several attacks launchedby the attack teams had not been launched by the authorsin their evaluation of WD [1] and WDH Thus S3 raisedour confidence in the effectiveness of the attack detectionmechanisms based on invariants derived from plant designs

RQ2 How effective is WD in detecting attacks launched byindependent attack teams As mentioned earlier while bothWD and WDH were found to detect a number of attacksthey did fail in several cases Given that the invariants derivedare intended to detect process anomalies it is clear that suchmechanisms must be used in conjunction with other attackdetection tools such as those in [24] [27] [21]

C Assessment by the authors and by independent teamsTable VIII lists the number of attacks launched by the

authors in an experimental evaluation performed prior to S3-2016 [1] Note that the WD detection rate observed by theauthors (89) was higher than the combined rate observedduring the two S3 events (6326) The difference in perfor-mance is due to different attack vectors used in the three setsof experiments WDH detection rate observed during S3 eventis (7741) which is much higher than the WD detection rateSome of these attack vectors are explained in Section VI andthe remaining may be found in [20]

TABLE VIIIPERFORMANCE OF WD AS EVALUATED BY THE AUTHORS AGAINST THOSE

BY PARTICIPANTS IN S3

Experiments by Attacks

Launched Detected (WD) Detected (WDH)

Authors 37 33 (89) NA

S3-2016 18 10 (555) NA

S3-2017 31 21 (677) 24 (774)

NA WDH did not exist at the time of experimentation by the author andduring S3-2016

The data in Table VIII is indicative of the value of orga-nizing S3 events Specifically in the case described in this

paper the two S3 events led to an increased confidence inthe effectiveness of the invariant-based approach in detectingcyber attacks The hackfests also led to the creation of newtypes of attack vectors that were not used earlier to assess theperformance of WD and WDH in detecting cyber attacks

D False alarms

The performance of any attack detection method ought tobe assessed using its detection accuracy ie how many of thelaunched attacks it detects as well as the rate at which falsealarms are raised During S3 each team attempted to launchseveral attacks The attacks listed in Tables III and IV are theones that were successful in realizing the stated attacker intentand were scored by the judges The remaining attacks werenot recorded and hence any alarm generated by such attackswas not considered Some of these unrecorded alarms couldbe false though no specific claims can be made about theirnature

Since S3-2017 the authors have observed no false alarmsfrom WD during normal operation of SWaT WDH has beenin operation since a few weeks prior to S3-2017 Againduring the normal operation of SWaT no alarm has beengenerated by WDH This observation should not be construedto imply that an invariant-based attack detection mechanismwill not generate any false alarmndash in fact it could Howeverif the invariants generated are complete in the sense that theyaccurately capture all aspects of process behavior and theirimplementation is correct and tuned properly the likelihoodof false alarms is low

Even though SWaT is a relatively new plant (2-years sinceits inauguration at the time of writing this paper) we doobserve intermittent failures in a few motorized valves Forexample sometimes MV101 in Stage 1 takes much longer toopen than expected by its controlling PLC1 The PLC itselfdetects such cases In such a case WD or WDH dependingon the time it takes for the valve to finally open will raisean alarm We do not consider this as a false positive simplybecause whether an anomalous behavior is due to a naturalcause or a cyber attack cannot be distinguished by WD orWDH While such distinction is important to make additionalresearch is needed to distinguish process anomalies due tocyber attacks and those arising due to natural componentfailures

E Benefits of S3

S3 exposed the organisers participants and researchers tohow an attacker might design and launch attacks on ICS Bene-fits of S3 include the following 1) An improved understandingof how an ICS operates and the consequent formulation ofnew research directions 2) Opportunity for participants fromindustry and academia to learn from the event and focus onthe limitations of their work 3) An aid to the ICS managementteam to observe the defense teams thus leading to possibleadoption of technology embedded in WD or WDH

F Placement of WD

The placement of WD is another question that ought tobe looked into carefully In this work WD is placed insidePLCs However an exceptionally large number of invariantsmay prevent adding code to the existing control code in a PLCThis may happen due to the computational load requirementson a PLC This aspect led us to create WDH that is placedon the plant network and gets its data from the Historian toevaluate the invariants

G Forensics

One advantage of the invariant-based approach for attackdetection appears while determining the area of impact ofan attack When a single invariant is violated it indicatesclearly the source of process anomaly For example an alertis generated if valve MV101 is closed when the water intank T101 is at or below the L level marker While this alertdoes not indicate how an attacker entered the system or ifthe valve or the level sensor is defective it does assist inlocalising the reason for the alert The analysis becomes abit more complex when multiple invariants raise alerts Thisaspect of an invariant-based detection mechanisms remains tobe analyzed in further detail

H Attacker capabilities

We do not have any validation of the professionalism of theS3 attack teams As mentioned earlier [20] [35] [36] attackteams were from a variety of backgrounds including fromthe industry and academia from Europe and Asia During S3-2017 one team consisting of four membersndashall from outsideof Singaporendash focuses on ethical hacking and cyber-warsinvolving critical infrastructure This team is part of a globalalliance The other teams consist of hackers interested inknowing how vulnerabilities in software can be exploitedand passes this information to others for improving systemssecurity Coverage of attacks launched by the attack teams andattacker profiles is discussed in Section IV and summarizedin Tables I II V and VII

I Attack trees

It is possible to use attack trees [37] [42] to model attackslaunched during the two hackfests reported in this paper Doingso would enable mapping each attack to a specific path inthe attack tree and reveal which attack paths in SWaT weretraversed Such modeling and analysis has not been attemptedin this work and is a possible subject for future research

IX RELATED WORK

S3 is a Capture-The-Flag [15] event on ICS TraditionalCTF events generally attract the attention of both industrialand academic teams and currently enjoy increasing popularityas indicated in [15] The number of such events is graduallyincreasing [13] [16] Such events aid in learning about secu-rity vulnerabilities how these could be exploited nature ofattacks and strength of the deployed [18] [33] [45] defensemechanisms To the best of our knowledge S3 is the first CTF

style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed

The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks

There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc

The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies

Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS

X CONCLUSION

There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense

While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the

actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure

It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously

The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants

The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]

It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question

ACKNOWLEDGMENTS

A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript

REFERENCES

[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016

[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016

[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016

[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016

[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016

[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016

[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017

[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012

[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017

[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and

S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011

[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml

[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010

[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015

[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security

under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler

Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016

[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017

[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016

[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016

[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010

[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for

safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016

[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical

systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013

[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111

[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016

[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013

[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx

[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007

[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016

[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3

[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event

[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008

[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016

[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011

[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf

[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment

of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007

[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016

[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016

[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003

[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014

[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014

[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011

BIOGRAPHY

Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems

Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research

sumn

i=1|(x(i)minus y(i))|n

gt ε under attack (2)

le ε normal (3)

Fig 6 Invariant to detect anomalous behavior of LIT 101

IV SWAT SECURITY SHOWDOWN (S3)

This section presents details of the two S3 [36] eventsincluding guidelines and selected information on participantsIn S3 the attackers are challenged to realise concrete goals inSWaT Points earned by an attack team are weighted based onthe capabilities needed to launch the attack and the number ofdefence mechanisms successfully bypassed during the attackThe goal was to meet as many pre-defined challenges aspossible within the pre-allocated time

Information disclosed to the attack teams Technical detailson SWaT such as network architecture protocols and devicesused are released to the attackers one month prior to theirarrival for participation in the event Publicly available whitepapers on mechanisms deployed by the defence teams areshared with each attack team

Information disclosed to the defenders S3 organizers workedclosely with the defense teams to integrate their defence mech-anisms into SWaT Information about the normal operation ofSWaT was disclosed to the defenders to enable them to fine-tune their detection systems and reduce false alarms as muchas they could

Attacker profiles Attack teams were asked to select from aset of attacker profiles [34] The following attacker profileswere available cyber-criminal insider or a combination ofboth An attacker profile is intended to restrict availability ofresources and limit the access rights of the attackers as shownin Table I

A S3-2016

Attack teams included three from industry and three fromacademia Similarly there were three defense teams from theindustry and three from academia During the live phase heldat the SWaT testbed all six [35] defence mechanisms weresimultaneously in place Each team was given 12 hours forpassive reconnaissance and team was assigned a 3-hour slotduring which they were able to launch attacks

B S3-2017

Attack teams included one from industry and four fromacademia There were two defense teams from the industryand two from academia Each attack team was given two ses-sions [36] of four hours each to conduct reconnaissance on thetestbeds During these sessions various attacks were preparedand tested with the assistance of the SWaT laboratory engineerDuring the actual event each team was given two hours todemonstrate their attacks that were prepared previously Attack

TABLE IRESOURCES AND ACCESS RIGHTS FOR ATTACKER PROFILES

Profile Constraints

Cyber-criminal Limited number of attempts to realize a goal

Physical access not allowed manual manipulation ofthe sensors and actuators are not allowed

Direct connection to PLCs using any software suchas Allen Bradleyrsquos Studio5000 not allowed

Insider Physical access to SWaT allowed manual manipula-tion of the sensors and actuators are allowed

Allowed to alter the network topology

Direct connection to PLCs using any software suchas Allen Bradleyrsquos Studio5000 allowed

TABLE IITARGETS OF ATTACKS IN S3

Target Description

Physical Process Attacks

Valves Control the motorized valves

Pumps Disrupt pump control operations

Pressure Alter the pressure in pipes

Tank fill level Alter water level in a tank

Chemical dosing Alter chemical dosing

Sensor Data Attacks

Historian Alter data in the Historian

HMISCADA Alter the sensor actuator values at HMI orSCADA DoS Attacks on SCADA HMI

PLC Reprogram PLC DoS attacks on PLCsChange the commands and values in which thePLC receives and sends

RIODisplay Control of the RIO through disconnected ana-logue InputOutput pin

teams were also given a separate network for Internet up-linkand up to three Virtual Machines (VMs) running either Linuxor Windows operating system

C Attack targets

The attack teams were given a list of components andsubsystems in SWaT that could serve as the target of theirattacks Table II lists the targets available to the attack teamsTable II has two kinds of attacks physical process attacks andsensor data attacks In physical process attacks an attackerrsquosobjective is to alter the physical process In the case of sensordata attacks an attackerrsquos objective is to alter the sensor oractuator tags during communication or in the Historian

V PREPARATION FOR S3

To prepare for S3-2016 an earlier version of WD wasextended to all six stages of SWaT This extension requiredthe generation of invariants across all stages coding of theinvariants and placement of the code inside [1] the six PLCsThe modified WD was tested on SWaT by running the plantunder various operating conditions

Based on lessons learned during S3-2016 several newinvariants were generated coded and added to the PLCs ForS3-2017 we decided to use an additional monitoring systemplaced outside the PLCs This system collects data from theHistorian and evaluates the invariants All invariants wereimplemented in a Linux environment using a Piwebclient APIto talk to the Historian This new implementation is referredto as WDH

The invariants in WD are coded using ladder logic andstructured text while those in WDH in Python Both imple-mentations use the same set of invariants the difference is intheir placement The Historian may not get all the data andcommands that flow across the PLCs sensors and actuatorsHowever as WDH gets its data directly from the Historian ithas access to information flowing across SCADA workstationand the Historian This information may be compromised byan attacker and is not available to the PLC

A Scope of WD

WD is designed to detect process anomalies Thus anyabnormal behavior in the water treatment process in SWaTought to be detected by WD However there could be attacksthat do not cause the process to deviate from its normalbehavior but lead to undesirable consequences An exampleof such an attack is one intended to deface the screen on theSCADA workstation or the HMI Such an attack will not bedetected by WD Attacks that may cause process anomaly butonly after an attack has been removed from the system mayalso not be detected by WD Denial of Service is one suchattack

B Scope of WDH

WDH and WD use the same set of invariants Howeverthe placement of WDH could lead to a difference in detectioncapabilities of the two defense mechanisms WDH gets its datafrom Historian while WD directly from the PLC Data that isnot programmed to be logged in the Historian will not beaccessible to WDH Thus any anomaly that requires such datawill likely not be detected by WDH Similarly attacks thatmanipulate data entering the Historian or SCADA may not bevisible to WD Thus while the two invariant-based processanomaly detection mechanisms are identical in the invariantsthey use their placement in SWaT is expected to result indifferent performance in detecting attacks

VI S3 ATTACKS

The attacks launched by teams participating in the two S3

events are described next

A S3-2016 Attacks

All attacks designed and launched during S3-2016 areenumerated in Table III Three attacks selected from Table IIIare described next Details of all attacks are available in [9]Of the 18 attacks in Table III 4 and 16 are cyber criminalattacks and the remaining are insider attacks

DoS attack on SCADA In this attack (attack 4 in Table III)the attackerrsquos intention was to deface the SCADA workstationscreen and hence prevent the operator from observing plantstate The cyber-criminal attacker model was used to designthis attack To realize the intention the attacker launched anARP poisoning Man-in-the-Middle attack in two steps In thefirst step all traffic intended for HMI was redirected to theSCADA workstation In the second step this redirected trafficwas dropped and thus no packets were received at the SCADAworkstation This led to the screen on the workstation becom-ing completely gray and no state information was displayedThis attack was not detected by WD as it did not lead toany process anomaly It is an ARP spoofing attack and not atraditional DoS attack As part of the DoS attack the attackertargeted the PLC and sent millions of packets at a time Thisled to the same effect as would be the case when an ARPspoofing attack is performed on SCADA

Manipulation of the chemical dosing pump Intention of theattacker in this case (attack 14 in Table III) was to manipulatethe pH of water entering Stage 3 of SWaT The insider-attackermodel was used in the design of this attack This attack wasexecuted in two steps In the first step PLC 2 was set to manualmode Note that in manual mode the plant operator can directlycontrol the actuators eg the dosing pumps in this case In thesecond step the attacker altered the chemical dosing processin the Pre-treatment Stage 2 of SWaT by interacting directlywith the HMI interface and overriding the commands sent bythe PLC WD was able to detect this attack because the set-points changed by the attacker were different from those setin WD

DoS to PLC by SYN flooding The intention of the attackerin this case (attack 16 in Table III) was to disable the HMIso that an operator is unable to view or control the plantoperation The insider-attacker model was used in the designof this attack In this way the attacker had an access to theadministrator account and the associated tools The attackerperformed a SYN flooding attack on EthernetIP server ofPLC1

As a result of this DoS attack the HMI was unable toobtain the current state values to display and would insteaddisplay 0 or characters WD was unable to detect this attackphysical process as not affected During the attack period PLCwas controlling the process as expected Such attacks whilenot altering process behavior may impede supervision of theprocess in an operational plant

B S3-2017 Attacks

All attacks designed and launched during S3-2017 areenumerated in Table IV Selected attacks from Table III aredescribed next Details of all attacks are available in [20] Ofthe 31 attacks in Table IV 17 can be classified as cybercriminal attacks and the remaining as insider attacks (Figure I)All attacks launched during S3-2016 and S3-2017 are listedand categorized in Table V

TABLE IIIATTACKS LAUNCHED DURING S3-2016

SNO Target Method Attack Tool

1 Tank fill levelLIT101

Use HMI access Close MV101 and Stop P101 andP102

HMI

2 HMISCADA ARP spoofing Attack HMI DoS attack Ettercap

3 PLC Manual access Removed the cable at the ring atlevel 0

Manual

4 HMISCADA DoS on HMI by droppingall packets between PLC andSCADAHMI

DoS attack on SCADA wide DoSattack took a while to restore SWaTto its normal state

Ettercap

5 Tank fill levelLIT101

Use HMI access Attack on LIT101 ManualHMI

6 Valve MV301 Use SCADA access Attack on MV301 manually openfrom the SCAD workstation

ManualSCADA

7 Pump P101 Use SCADA access Attack pump manually open it fromthe SCADA workstation

ManualSCADA

8 Historian DoS attack using CPPPO andloop

Attack between HMI and PLC CPPPO

9 Valve MV101 Use SCADA access MV101 attacked using SCADAchanged the valve state from Opento Closed

ManualSCADA

10 Pump P101 Use SCADA access LIT301 set point changed ManualSCADA

11 Tank fill levelLIT301

Using SCADA access LIT301 set point altered ManualSCADA

12 Chemical dosingP201

Control MV101 and AIT503 setpoints of LIT301 to ensure flowthis triggered chemical dosing

Dosing pump attack on P201 ManualSCADA

13 HMISCADALIT101

Functional block introduce newconstant tag tie that to output tagcould only do zero

LIT101 set to zero from PLC Studio5000

14 Chemical dosingpump P205

Use SCADA access Manipulation of the chemical dosingpump (P205)

ManualSCADA

15 HMISCADA DoS on HMI using Level 1 net-work

Attack on HMI EttercapPycomm

16 Historian SYN flood ENIP port at PLC1 DoS to PLC by SYN flooding (attackon HMI)

Ettercap

17 Chemical dosingpump P203

HMI-based direct manipulation Attack on P203 while the four dosingpumps are running

ManualHMI

18 HMISCADALIT101

Re-program PLC to fix LIT101value to an arbitrary value

Attack on LIT101 Studio5000

416 are cyber criminal attacks in S3-2016

Control of the chemical dosing system through a Pythonscript (Pycomm) The objective of this attack (attack 15 inTable IV) was to change chemical dosing at the end of the de-chlorination system (Stage 4) First the attackers compromisedVirtual Network Computing (VNC) Then they used a Pythonscript (Pycomm) and Wireshark to gain access to the HMIAfter gaining access to the HMI through the compromisedVNC the cybercriminal attacker used Wireshark to capturethe packets flowing between the HMI and PLC4 The con-troller tags were retrieved by an analysis of the packets Theattackers changed the data associated with these tags to controlthe chemical dosing function using the Pycomm framework

Control of PLC through the Bridged Man-in-the-Middle(MiTM) at Level 0 the objective of this attack was (attack 16in Table IV) to change the commands and values that PLC1receives and sends First the attackers configured a bridgebetween the RIO and PLC1 using Netfilterqueue andScapy The attack was launched at two network levels Ananalysis on the network traffic revealed the packets that theattackers should edit As the target of this attack was thewater level in T101 the attackers set it to a constant valueto hide from PLC1 the rise in water level in T101 Before apacket was forwarded Netfilterqueue rerouted it into aqueue which can be read and modified by the Python script

TABLE IVATTACKS LAUNCHED DURING S3-2017

SNo Target Method Attack Tool

1 HMISCADA LIT401 HMI simulation insider attack Change the value of LIT401 in the HMI Manual HMI

2 Historian ARP and drop Change the value stored at the Historian Ettercap

3 Valve MV201 Reprogram PLC Change the status of the MV201 Studio 5000

4 Tank fill level LIT301 420to 320

Manual Lower the water tank level from 820mm to420mm without raising any alarm LIT301decreased till 320mm

Manual HMI

5 Pump P101 Manual mode of pump Alternate the state [OnOff] of the pump P101 Manual HMI

6 Chemical dosing P205 Manually dosing chemical pump Change the chemical dosage of sodiumhypochlorite (NaOCl) in P2

Manual SCADA

7 PLC Disconnect cable Disrupt sensor values from remote inputoutput(RIO) to the PLC

Manual

8 RIO Display Disconnect IO PIN manual Disrupt the sensor reading send to PLC throughRemote IO (RIO)

Manual

9 Chemical dosing P404 MiTM Python script to control Increase chemical dosage in pre-treatment Python script

10 LIT101 (476mm to 540mm ) Reprogram PLC Falsify water level display at SCADA Studio 5000

11 Pump P101 HMI simulation insider attack Alternate the state [OnOff] of the pump P101 Manual HMI

12 HMISCADA AIT 504 ARP+rewriting Increase AIT504 Ettercap

13 PLC LIT401 Reprogram PLC Falsify water level display at SCADA Studio 5000

14 RIODisplay Disconnect specific IO PIN basedon manual

Disrupt the sensor reading send to PLC throughremote IO (RIO)

Manual

15 Chemical dosing pumpP403 AIT501

Based on captured traffic betweenHMI and PLC4

Change chemical dosing function VNC Python script PycommWireshark

16 PLC LT101 from 742mm to500mm

Level 0 MITM Change the commands and values that the PLCreceives and sends

Aircrack Airodump AireplayNetfilterqueue Scapy

17 Historian LT101 tag Aircrack WiFi ARP spoofingEttercap

Compromise historian data Ettercap Aircrack

18 Pressure sensor DPIT30130MV301-4

SMB to EW get project files runFT

Disrupt valves operation of Ultrafiltration andBackwash (P3)

SMB

19 MV201 LT101 metasploit+vnc Change the water level of the tank LIT101 Metasploit+vnc

20 Pump P501 Rogue AP disassociated Telnetwith default credentials to turnoff original AP Scapy rewrite

Disrupt pump control operation KisMAC Password crackingtool 3vilTwinAttacker TelnetScapy

21 PLC LIT101 Reprogram PLC Change level indicator value Studio 5000

22 Pump P101 LIT301 Using back-door connection Establish back-door connection Mimikatz malicious VBAMacro SOCKS proxy

23 HMISCADA P201 Netfilterqueue Scapy Change the display value of the HMI Netfilterqueue Scapy

24 Historian LIT101 Overwrote specific data stored at the Historian Microsoft PsExec ipconfig

25 RIODisplay Manual Control of the RIO through disconnected Ana-logue InputOutput pin

Manual

26 Valve MV201 Manual Permanently closed the motorised valve re-gardless of commands issued

Manual SCADA

27 RIODisplay AIT202203 Manual Change the pH value shown at HMI Manual HMI

28 PLC MV201 P101 ARP poisoning MiTM Increase the pressure at P1 Ettercap

29 Tank fill level LIT101 Lower Falsify the water level reading of the tankdisplayed at SCADA

Pycomm

30 Chemical dosing PLC2 Use Studio 5000 Change the level of the chemical used fordosing

Studio 5000

31 Pressure MV302 P3012 Using Pycomm script Change the pump state sent to the PLC Pycomm

TABLE VCYBER CRIMINAL ATTACKS IN S3

Cyber Criminal Attacks Insider Attacks

S3-2016 4 16 1 2 3 5 6 7 8 910 11 12 13 14 1517 18

S3-2017 2 9 10 12 13 1516 17 18 19 20 2122 24 28 29 30

1 3 4 5 6 7 8 1114 23 25 26 27 31

To prevent all packets from entering the queue in order notto disrupt other processes iptables was used to identifythe targeted packets entering the queue Using Scapy and acustom dissector the attacker edited the payload of the targetedpacket which was then forwarded to its original destination

Control of Historian through the Aircrack WiFi The objectiveof this attack was (attack 17 in Table IV) to compromise thedata stored in the Historian Attackers performed crack WiFipassword ARP poisoning and MiTM payload manipulationusing Aircrack and Ettercap As PLC1 was operating inthe wireless mode the cybercriminal attacker used Aircrackto obtain the password for connecting to the ICS Access Point(AP) ARP poisoning was executed to reroute traffic betweenPLC1 and the Historian through the attackerrsquos rogue terminalThe attackers then used an Ettercap filter to manipulate thenetwork packets The attackers changed the tag correspondingto LIT101 to an arbitrary value before releasing the packetsto the Historian

Control of pressure through the Server Message Block (SMB)The objective of this attacks was (attack 18 in Table IV) todisrupt the state of four motorized valves in Stage 3 to affectthe differential pressure in UF Vulnerability CVE-2008-21601

in Factory Talk software from Rockwell and in MicrosoftrsquosServer Message Block (SMB) was used by the attackersto obtain files from the HMI As the HMI was runningWindows CE it has a vulnerability that allows an attackerrsquosterminal to execute arbitrary code on the HMI Thus theattackers were able to retrieve the files to create a copy ofthe workstation From the copied workstation the attackersmanually changed the state of the valves in Stage 3 suchthat the differential pressure across the UF unit as measuredby DPIT301 became dangerously high The attackers closedvalves MV301 MV302 and MV303 and opened MV304

Control of water level in the tank through the MetasploitVNC Scanner Objective of this attack was (attack 19 inTable IV) to change the water level in tank T101 The attackersused Metasploit VNC authentication None scanner to ob-tain access to the VNC server without password protection andto check for nodes running a VNC Server Once the scannerdetected the VNC Server running without any authenticationthe attackers penetrated into the server through a VNC Clientconnection As the VNC Server was hosting the HMI which

1httpswwwcvedetailscomcveCVE-2008-2160

controlled the ICS the attackers changed the simulation tagassociated with water level in T101

Control of a pump through a rogue router The objective ofthis attack (attack 20 in Table IV) was to disrupt the controlof pump P501 The attackers used Evil twin (rogue accesspoint) method using KisMAC a password cracking tool3vilTwinAttacker Telnet and Scapy The attackersused KisMAC to scan for wireless networks in the ICS Oncethe targeted wireless network was identified the attackers useddictionary attack to crack the password After the passwordwas cracked the attackers created a rogue wireless routerwith a similar SSID and configuration They then sent a de-authentication packet to disassociate PLC5 and the originalrouter The attackers used Telnet to log into the originalrouter and shut it down Scapy was then used to modify thepackets to turn the pump on

VII RESULTS

Tables VI and VII summarize the response of WD andWDH to the attacks launched during the two S3 events Recallthat both WD and WDH contain exactly the same set ofinvariants In WD the invariants are coded and placed insidethe PLCs whereas in WDH the invariants are coded and placedat the Historian WDH did not exist during S3-2016 and hencethe response of WDH is available only for attacks launchedduring S3-2017

A S3-2016 results

We note from Table VI that 10 out of 18 attacks weredetected immediately while the remaining eight attacks werenot detected Six of the eight undetected attacks did not leadto process anomaly during the observation period and hencedid not violate any invariant This outcome is expected as theinvariants in WD are designed to detect process anomaly

Consider attack 2 ARP spoofing in Table III This is aDoS attack on HMI It leads to defacing the screen on theHMI or displaying incorrect information thereby preventingan operator from knowing the actual plant state Howeverthe attack does not cause process anomaly and hence is notdetected as it does not violate any invariant Similar logic canbe used to explain why the other attacks in Table VI are notdetected

It is important to note that a DoS attack when given enoughtime to evolve and be launched at an appropriate state of theplant may impact physical process behavior In such a caseone or more invariants may detect the attack One such attackis 16 in Table VI This attack prevented the Historian fromreceiving data from PLC1 However if this attack was leftactive for a longer period it would prevent PLC1 from sendingappropriate commands to the actuators eg to MV101 orP101 In turn this would have led to process anomaly Notenough data is available to conclude with certainty whether ornot this attack would be detected by WD if active for sufficienttime

Two single point [2] attacks were not detected by WD Inone attack (attack 6 in Table III) the adversary altered the status

of valve MV301 Under normal circumstances this valve isopened during the backwash process However the attackeropened it when there was no backwash Hence the attackdid not affect the physical process except in changing thevalve status No invariant was violated due to this attackbecause the backwash process ie Stage 6 is not includedin this case study The second single point attack (attack 17 inTable III) was performed on chemical dosing pump P203 whilethe other pump P204 was running Note that under normalcircumstances only one of these two pumps is supposed to berunning while the other remains as a backup Subsequently theattacker shut down pump P204 This attack was not detectedbecause there were no invariants that related to the chemicalproperties of water

Although the overall performance of WD was below 100it did detect all attacks within its scope except two (attacks 6and 17 in Table III) as mentioned earlier

B S3-2017 results

Table VI indicates that 21 out of 31 attacks were detectedby WD while 24 out of 31 attacks were detected by WDHConsidering only the attacks within its scope as mentionedin Section V-A WD detected 21 out of 28 attacks (75)Similarly WDH detected 24 out of 31 attacks (7741) withinits scope mentioned in Section V-B Three attacks on theHistorian are not in the scope of WD All attack targets relatedto RIODisplay (in Table II and in Table IV) are not detectedby both WD and WDH This is because registers inside aPLC save the previous values received from the sensors andthe PLC continues to execute the control code The invariantsalso use the same values stored in the PLC registers and hencedo not raise an alert

In general PLCs send to the Historian via the SCADAworkstation the data received from the sensors When a PLCdoes not have updated values during the attack period it isobvious that the Historian also receives the same stale valuesThis is the reason why WDH also did not detect attacksrelated to RIODisplay Note that the RIODisplay attacks werelaunched and remained active only for a few seconds Duringthis period the PLC did not update the current sensor valuescoming through the RIO If the same attack is performed for alonger duration the PLC would update the data received fromthe sensors Doing so would likely lead to WD and WDHdetecting the RIO attacks

Attacks launched on the Historian were detected by WDHbut not by WD This variance is due to the fact that data inthese attacks is manipulated at the Historian Thus invariantsin a PLC do not have access to the manipulated data andhence the invariants in WD do not raise any alert All attackstargeting a PLC are detected by WD and WDH

WD Detection of physical process attacks All attacks onvalves pressure sensor and level sensors were detected Threeout of four attacks on the chemical dosing process pumpswere detected An example of a detected attack is when theattackers took control of pump P301 (attack 20 in Table IV)

TABLE VIPERFORMANCE OF WD AND WDH

S3-2016 S3-2017

WD WD WDH

Detected 1 5 7 910 11 1213 14 18

3 4 7 9 1011 12 13 1516 18 19 2021 22 23 2628 29 30 31

2 3 4 7 9 1011 12 13 15 1617 18 19 20 2122 23 24 26 2829 30 31

Not detected 2 3 4 68 15 1617

1 2 5 6 8 1417 24 25 27

1 5 6 8 14 2527

through a Python script (Pycomm) to raise the pressure in theUF unit measured by sensor DPIT301 to a dangerous levelWD immediately raised an alarm This invariant ensured thatpump P301 must be OFF when the pressure at DPIT301 wasabove a threshold During the attack the invariant was violatedas the pump was not turned off while DPIT301 indicatedreadings that were above the threshold Consequently an alarmwas raised immediately In certain cases multiple alarmswere raised due to the violation of one or more invariantsFor example when level sensor LIT101 was compromisedthe invariants corresponding to this sensor were violated andraised alarms

WD Detection of sensor data attack WD detected attacks onHMISCADA and PLC values because these attacks directlycompromised the physical processes These attacks eithercompromised chemical dosing water tank levels or pumpstatus through hacking of the HMISCADA or PLC Hencethe robustness of WD in detecting unusual physical processbehavior was found effective in these attacks On the otherhand WD was unable to detect insider attacks that pulled outRIO cables This is because WD triggers an alarm only whenthe invariants are violated Under normal circumstance for aperiod of time a PLC continues to execute its control codeand any invariant code based on the last known state andorvalues Thus the invariants located inside the PLCs are unableto observe this anomalous behavior

WDH Detection of physical process attacks WDH detected14 out of 16 physical process attacks

WDH Detection of sensor data attacks WDH detected theattacks on HMISCADA and PLC values because these attacksdirectly compromised the physical processes albeit with aslightly lower detection rate when compared with the rate ofdetecting physical process attacks As with WD WDH did notdetect any attack launched against the Remote IO by pullingthe cables that connect it to the corresponding PLC WDHfared better in the detection of attacks against the Historian asit was directly accessing data on the Historian server

If the Historian itself or data that is input to the Historian iscompromised WDH takes the decision based on the input itreceives A clever and powerful attacker can attack the physical

TABLE VIIRESULTS FROM S3 2017

Target of Attack Noofattacks

WD WDH

Physical Process Attacks

State of motorised valves 2 100 100

State of water pumps 4 75 75

Pressure in UF 2 100 100

Water tank level 4 100 100

Chemical dosing 4 75 75

Sensor Data Attacks

Data in historian 3 0 100

Data in HMISCADA 3 67 67

Tampering PLC communi-cations

5 100 100

Tampering Remote IO 4 0 0

Total Attacks 31 6774 7741

process and modify values entering the Historian and thusdeceive WDH In general such a situation may arise in allbehavioral intrusion detection systems where the detector takesthe decision based on incorrect input data

Indeed data that appears to be ldquolegitimaterdquo could lead theWDH into believing that there is nothing wrong with thephysical process though there actually is However doingso requires the attacker to continuously manipulate a largenumber of state variables For example consider an attackwhere the attacker turns a pump say P101 ON when it shouldbe OFF and (continually) sends the state of the pump as OFFto the Historian and the corresponding PLC If the pump isOFF then the level of the source and destination tanks must berespectively decreasing and increasing at rates determined bythe pump characteristics Creating ldquolegitimate-lookingrdquo datathus requires an attacker to manipulate several state variablesas explained next (a) Two state variables that correspondto tank levels Two sensors (in SWaT) measure these statevariables (see Figure 2) Thus the attacker must have accessto these level sensors (b) If pump P101 is actually ON whilethe Historian receives its state as OFF then FIT201 must showno flow Thus the attacker will also need to manipulate FIT201to avoid detection This argument can be carried forward tosubsequent stages to show that many sensors will need to bemanipulated by an attacker to ldquohiderdquo a simple attack such asldquochange the state of a pumprdquo In summary yes incorrect dataat the Historian could prevent detection though doing so wouldbe a significant challenge for the attacker due primarily to thedistributed nature of the invariants

VIII DISCUSSION

A Challenges faced

We faced several challenges during S3 For example aftereach teamrsquos performance the operator was required to bringSWaT back to a predefined normal state It was necessary to

keep SWaT in a normal state before another team launched at-tacks Bringing SWaT to its normal state required (a) resettingnetwork communications to ensure that all the communicationchannels are operating as expected (b) the operator to ensurethat all physical processes in SWaT are stable with respectto the control logic (c) the operator to bring back SWaT tothe normal state of that particular device such as a pump or amotorized valve in the case of any physical or manual attacksby the previous team and (d) that the Historian and SCADAservers were reverted to their original state ie the state thatexisted prior to the launch of attacks

B Research questions

RQ1 How do attackers compromise the security of an ICS InSection VI we presented and categorized the attacks based onattacker profiles An attacker can launch physical attacks wheninside the plant such as manually operating a motorized valveor tampering with network cabling Several attacks launchedby the attack teams had not been launched by the authorsin their evaluation of WD [1] and WDH Thus S3 raisedour confidence in the effectiveness of the attack detectionmechanisms based on invariants derived from plant designs

RQ2 How effective is WD in detecting attacks launched byindependent attack teams As mentioned earlier while bothWD and WDH were found to detect a number of attacksthey did fail in several cases Given that the invariants derivedare intended to detect process anomalies it is clear that suchmechanisms must be used in conjunction with other attackdetection tools such as those in [24] [27] [21]

C Assessment by the authors and by independent teamsTable VIII lists the number of attacks launched by the

authors in an experimental evaluation performed prior to S3-2016 [1] Note that the WD detection rate observed by theauthors (89) was higher than the combined rate observedduring the two S3 events (6326) The difference in perfor-mance is due to different attack vectors used in the three setsof experiments WDH detection rate observed during S3 eventis (7741) which is much higher than the WD detection rateSome of these attack vectors are explained in Section VI andthe remaining may be found in [20]

TABLE VIIIPERFORMANCE OF WD AS EVALUATED BY THE AUTHORS AGAINST THOSE

BY PARTICIPANTS IN S3

Experiments by Attacks

Launched Detected (WD) Detected (WDH)

Authors 37 33 (89) NA

S3-2016 18 10 (555) NA

S3-2017 31 21 (677) 24 (774)

NA WDH did not exist at the time of experimentation by the author andduring S3-2016

The data in Table VIII is indicative of the value of orga-nizing S3 events Specifically in the case described in this

paper the two S3 events led to an increased confidence inthe effectiveness of the invariant-based approach in detectingcyber attacks The hackfests also led to the creation of newtypes of attack vectors that were not used earlier to assess theperformance of WD and WDH in detecting cyber attacks

D False alarms

The performance of any attack detection method ought tobe assessed using its detection accuracy ie how many of thelaunched attacks it detects as well as the rate at which falsealarms are raised During S3 each team attempted to launchseveral attacks The attacks listed in Tables III and IV are theones that were successful in realizing the stated attacker intentand were scored by the judges The remaining attacks werenot recorded and hence any alarm generated by such attackswas not considered Some of these unrecorded alarms couldbe false though no specific claims can be made about theirnature

Since S3-2017 the authors have observed no false alarmsfrom WD during normal operation of SWaT WDH has beenin operation since a few weeks prior to S3-2017 Againduring the normal operation of SWaT no alarm has beengenerated by WDH This observation should not be construedto imply that an invariant-based attack detection mechanismwill not generate any false alarmndash in fact it could Howeverif the invariants generated are complete in the sense that theyaccurately capture all aspects of process behavior and theirimplementation is correct and tuned properly the likelihoodof false alarms is low

Even though SWaT is a relatively new plant (2-years sinceits inauguration at the time of writing this paper) we doobserve intermittent failures in a few motorized valves Forexample sometimes MV101 in Stage 1 takes much longer toopen than expected by its controlling PLC1 The PLC itselfdetects such cases In such a case WD or WDH dependingon the time it takes for the valve to finally open will raisean alarm We do not consider this as a false positive simplybecause whether an anomalous behavior is due to a naturalcause or a cyber attack cannot be distinguished by WD orWDH While such distinction is important to make additionalresearch is needed to distinguish process anomalies due tocyber attacks and those arising due to natural componentfailures

E Benefits of S3

S3 exposed the organisers participants and researchers tohow an attacker might design and launch attacks on ICS Bene-fits of S3 include the following 1) An improved understandingof how an ICS operates and the consequent formulation ofnew research directions 2) Opportunity for participants fromindustry and academia to learn from the event and focus onthe limitations of their work 3) An aid to the ICS managementteam to observe the defense teams thus leading to possibleadoption of technology embedded in WD or WDH

F Placement of WD

The placement of WD is another question that ought tobe looked into carefully In this work WD is placed insidePLCs However an exceptionally large number of invariantsmay prevent adding code to the existing control code in a PLCThis may happen due to the computational load requirementson a PLC This aspect led us to create WDH that is placedon the plant network and gets its data from the Historian toevaluate the invariants

G Forensics

One advantage of the invariant-based approach for attackdetection appears while determining the area of impact ofan attack When a single invariant is violated it indicatesclearly the source of process anomaly For example an alertis generated if valve MV101 is closed when the water intank T101 is at or below the L level marker While this alertdoes not indicate how an attacker entered the system or ifthe valve or the level sensor is defective it does assist inlocalising the reason for the alert The analysis becomes abit more complex when multiple invariants raise alerts Thisaspect of an invariant-based detection mechanisms remains tobe analyzed in further detail

H Attacker capabilities

We do not have any validation of the professionalism of theS3 attack teams As mentioned earlier [20] [35] [36] attackteams were from a variety of backgrounds including fromthe industry and academia from Europe and Asia During S3-2017 one team consisting of four membersndashall from outsideof Singaporendash focuses on ethical hacking and cyber-warsinvolving critical infrastructure This team is part of a globalalliance The other teams consist of hackers interested inknowing how vulnerabilities in software can be exploitedand passes this information to others for improving systemssecurity Coverage of attacks launched by the attack teams andattacker profiles is discussed in Section IV and summarizedin Tables I II V and VII

I Attack trees

It is possible to use attack trees [37] [42] to model attackslaunched during the two hackfests reported in this paper Doingso would enable mapping each attack to a specific path inthe attack tree and reveal which attack paths in SWaT weretraversed Such modeling and analysis has not been attemptedin this work and is a possible subject for future research

IX RELATED WORK

S3 is a Capture-The-Flag [15] event on ICS TraditionalCTF events generally attract the attention of both industrialand academic teams and currently enjoy increasing popularityas indicated in [15] The number of such events is graduallyincreasing [13] [16] Such events aid in learning about secu-rity vulnerabilities how these could be exploited nature ofattacks and strength of the deployed [18] [33] [45] defensemechanisms To the best of our knowledge S3 is the first CTF

style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed

The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks

There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc

The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies

Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS

X CONCLUSION

There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense

While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the

actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure

It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously

The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants

The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]

It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question

ACKNOWLEDGMENTS

A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript

REFERENCES

[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016

[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016

[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016

[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016

[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016

[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016

[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017

[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012

[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017

[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and

S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011

[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml

[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010

[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015

[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security

under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler

Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016

[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017

[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016

[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016

[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010

[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for

safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016

[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical

systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013

[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111

[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016

[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013

[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx

[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007

[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016

[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3

[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event

[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008

[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016

[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011

[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf

[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment

of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007

[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016

[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016

[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003

[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014

[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014

[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011

BIOGRAPHY

Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems

Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research

Based on lessons learned during S3-2016 several newinvariants were generated coded and added to the PLCs ForS3-2017 we decided to use an additional monitoring systemplaced outside the PLCs This system collects data from theHistorian and evaluates the invariants All invariants wereimplemented in a Linux environment using a Piwebclient APIto talk to the Historian This new implementation is referredto as WDH

The invariants in WD are coded using ladder logic andstructured text while those in WDH in Python Both imple-mentations use the same set of invariants the difference is intheir placement The Historian may not get all the data andcommands that flow across the PLCs sensors and actuatorsHowever as WDH gets its data directly from the Historian ithas access to information flowing across SCADA workstationand the Historian This information may be compromised byan attacker and is not available to the PLC

A Scope of WD

WD is designed to detect process anomalies Thus anyabnormal behavior in the water treatment process in SWaTought to be detected by WD However there could be attacksthat do not cause the process to deviate from its normalbehavior but lead to undesirable consequences An exampleof such an attack is one intended to deface the screen on theSCADA workstation or the HMI Such an attack will not bedetected by WD Attacks that may cause process anomaly butonly after an attack has been removed from the system mayalso not be detected by WD Denial of Service is one suchattack

B Scope of WDH

WDH and WD use the same set of invariants Howeverthe placement of WDH could lead to a difference in detectioncapabilities of the two defense mechanisms WDH gets its datafrom Historian while WD directly from the PLC Data that isnot programmed to be logged in the Historian will not beaccessible to WDH Thus any anomaly that requires such datawill likely not be detected by WDH Similarly attacks thatmanipulate data entering the Historian or SCADA may not bevisible to WD Thus while the two invariant-based processanomaly detection mechanisms are identical in the invariantsthey use their placement in SWaT is expected to result indifferent performance in detecting attacks

VI S3 ATTACKS

The attacks launched by teams participating in the two S3

events are described next

A S3-2016 Attacks

All attacks designed and launched during S3-2016 areenumerated in Table III Three attacks selected from Table IIIare described next Details of all attacks are available in [9]Of the 18 attacks in Table III 4 and 16 are cyber criminalattacks and the remaining are insider attacks

DoS attack on SCADA In this attack (attack 4 in Table III)the attackerrsquos intention was to deface the SCADA workstationscreen and hence prevent the operator from observing plantstate The cyber-criminal attacker model was used to designthis attack To realize the intention the attacker launched anARP poisoning Man-in-the-Middle attack in two steps In thefirst step all traffic intended for HMI was redirected to theSCADA workstation In the second step this redirected trafficwas dropped and thus no packets were received at the SCADAworkstation This led to the screen on the workstation becom-ing completely gray and no state information was displayedThis attack was not detected by WD as it did not lead toany process anomaly It is an ARP spoofing attack and not atraditional DoS attack As part of the DoS attack the attackertargeted the PLC and sent millions of packets at a time Thisled to the same effect as would be the case when an ARPspoofing attack is performed on SCADA

Manipulation of the chemical dosing pump Intention of theattacker in this case (attack 14 in Table III) was to manipulatethe pH of water entering Stage 3 of SWaT The insider-attackermodel was used in the design of this attack This attack wasexecuted in two steps In the first step PLC 2 was set to manualmode Note that in manual mode the plant operator can directlycontrol the actuators eg the dosing pumps in this case In thesecond step the attacker altered the chemical dosing processin the Pre-treatment Stage 2 of SWaT by interacting directlywith the HMI interface and overriding the commands sent bythe PLC WD was able to detect this attack because the set-points changed by the attacker were different from those setin WD

DoS to PLC by SYN flooding The intention of the attackerin this case (attack 16 in Table III) was to disable the HMIso that an operator is unable to view or control the plantoperation The insider-attacker model was used in the designof this attack In this way the attacker had an access to theadministrator account and the associated tools The attackerperformed a SYN flooding attack on EthernetIP server ofPLC1

As a result of this DoS attack the HMI was unable toobtain the current state values to display and would insteaddisplay 0 or characters WD was unable to detect this attackphysical process as not affected During the attack period PLCwas controlling the process as expected Such attacks whilenot altering process behavior may impede supervision of theprocess in an operational plant

B S3-2017 Attacks

All attacks designed and launched during S3-2017 areenumerated in Table IV Selected attacks from Table III aredescribed next Details of all attacks are available in [20] Ofthe 31 attacks in Table IV 17 can be classified as cybercriminal attacks and the remaining as insider attacks (Figure I)All attacks launched during S3-2016 and S3-2017 are listedand categorized in Table V

TABLE IIIATTACKS LAUNCHED DURING S3-2016

SNO Target Method Attack Tool

1 Tank fill levelLIT101

Use HMI access Close MV101 and Stop P101 andP102

HMI

2 HMISCADA ARP spoofing Attack HMI DoS attack Ettercap

3 PLC Manual access Removed the cable at the ring atlevel 0

Manual

4 HMISCADA DoS on HMI by droppingall packets between PLC andSCADAHMI

DoS attack on SCADA wide DoSattack took a while to restore SWaTto its normal state

Ettercap

5 Tank fill levelLIT101

Use HMI access Attack on LIT101 ManualHMI

6 Valve MV301 Use SCADA access Attack on MV301 manually openfrom the SCAD workstation

ManualSCADA

7 Pump P101 Use SCADA access Attack pump manually open it fromthe SCADA workstation

ManualSCADA

8 Historian DoS attack using CPPPO andloop

Attack between HMI and PLC CPPPO

9 Valve MV101 Use SCADA access MV101 attacked using SCADAchanged the valve state from Opento Closed

ManualSCADA

10 Pump P101 Use SCADA access LIT301 set point changed ManualSCADA

11 Tank fill levelLIT301

Using SCADA access LIT301 set point altered ManualSCADA

12 Chemical dosingP201

Control MV101 and AIT503 setpoints of LIT301 to ensure flowthis triggered chemical dosing

Dosing pump attack on P201 ManualSCADA

13 HMISCADALIT101

Functional block introduce newconstant tag tie that to output tagcould only do zero

LIT101 set to zero from PLC Studio5000

14 Chemical dosingpump P205

Use SCADA access Manipulation of the chemical dosingpump (P205)

ManualSCADA

15 HMISCADA DoS on HMI using Level 1 net-work

Attack on HMI EttercapPycomm

16 Historian SYN flood ENIP port at PLC1 DoS to PLC by SYN flooding (attackon HMI)

Ettercap

17 Chemical dosingpump P203

HMI-based direct manipulation Attack on P203 while the four dosingpumps are running

ManualHMI

18 HMISCADALIT101

Re-program PLC to fix LIT101value to an arbitrary value

Attack on LIT101 Studio5000

416 are cyber criminal attacks in S3-2016

Control of the chemical dosing system through a Pythonscript (Pycomm) The objective of this attack (attack 15 inTable IV) was to change chemical dosing at the end of the de-chlorination system (Stage 4) First the attackers compromisedVirtual Network Computing (VNC) Then they used a Pythonscript (Pycomm) and Wireshark to gain access to the HMIAfter gaining access to the HMI through the compromisedVNC the cybercriminal attacker used Wireshark to capturethe packets flowing between the HMI and PLC4 The con-troller tags were retrieved by an analysis of the packets Theattackers changed the data associated with these tags to controlthe chemical dosing function using the Pycomm framework

Control of PLC through the Bridged Man-in-the-Middle(MiTM) at Level 0 the objective of this attack was (attack 16in Table IV) to change the commands and values that PLC1receives and sends First the attackers configured a bridgebetween the RIO and PLC1 using Netfilterqueue andScapy The attack was launched at two network levels Ananalysis on the network traffic revealed the packets that theattackers should edit As the target of this attack was thewater level in T101 the attackers set it to a constant valueto hide from PLC1 the rise in water level in T101 Before apacket was forwarded Netfilterqueue rerouted it into aqueue which can be read and modified by the Python script

TABLE IVATTACKS LAUNCHED DURING S3-2017

SNo Target Method Attack Tool

1 HMISCADA LIT401 HMI simulation insider attack Change the value of LIT401 in the HMI Manual HMI

2 Historian ARP and drop Change the value stored at the Historian Ettercap

3 Valve MV201 Reprogram PLC Change the status of the MV201 Studio 5000

4 Tank fill level LIT301 420to 320

Manual Lower the water tank level from 820mm to420mm without raising any alarm LIT301decreased till 320mm

Manual HMI

5 Pump P101 Manual mode of pump Alternate the state [OnOff] of the pump P101 Manual HMI

6 Chemical dosing P205 Manually dosing chemical pump Change the chemical dosage of sodiumhypochlorite (NaOCl) in P2

Manual SCADA

7 PLC Disconnect cable Disrupt sensor values from remote inputoutput(RIO) to the PLC

Manual

8 RIO Display Disconnect IO PIN manual Disrupt the sensor reading send to PLC throughRemote IO (RIO)

Manual

9 Chemical dosing P404 MiTM Python script to control Increase chemical dosage in pre-treatment Python script

10 LIT101 (476mm to 540mm ) Reprogram PLC Falsify water level display at SCADA Studio 5000

11 Pump P101 HMI simulation insider attack Alternate the state [OnOff] of the pump P101 Manual HMI

12 HMISCADA AIT 504 ARP+rewriting Increase AIT504 Ettercap

13 PLC LIT401 Reprogram PLC Falsify water level display at SCADA Studio 5000

14 RIODisplay Disconnect specific IO PIN basedon manual

Disrupt the sensor reading send to PLC throughremote IO (RIO)

Manual

15 Chemical dosing pumpP403 AIT501

Based on captured traffic betweenHMI and PLC4

Change chemical dosing function VNC Python script PycommWireshark

16 PLC LT101 from 742mm to500mm

Level 0 MITM Change the commands and values that the PLCreceives and sends

Aircrack Airodump AireplayNetfilterqueue Scapy

17 Historian LT101 tag Aircrack WiFi ARP spoofingEttercap

Compromise historian data Ettercap Aircrack

18 Pressure sensor DPIT30130MV301-4

SMB to EW get project files runFT

Disrupt valves operation of Ultrafiltration andBackwash (P3)

SMB

19 MV201 LT101 metasploit+vnc Change the water level of the tank LIT101 Metasploit+vnc

20 Pump P501 Rogue AP disassociated Telnetwith default credentials to turnoff original AP Scapy rewrite

Disrupt pump control operation KisMAC Password crackingtool 3vilTwinAttacker TelnetScapy

21 PLC LIT101 Reprogram PLC Change level indicator value Studio 5000

22 Pump P101 LIT301 Using back-door connection Establish back-door connection Mimikatz malicious VBAMacro SOCKS proxy

23 HMISCADA P201 Netfilterqueue Scapy Change the display value of the HMI Netfilterqueue Scapy

24 Historian LIT101 Overwrote specific data stored at the Historian Microsoft PsExec ipconfig

25 RIODisplay Manual Control of the RIO through disconnected Ana-logue InputOutput pin

Manual

26 Valve MV201 Manual Permanently closed the motorised valve re-gardless of commands issued

Manual SCADA

27 RIODisplay AIT202203 Manual Change the pH value shown at HMI Manual HMI

28 PLC MV201 P101 ARP poisoning MiTM Increase the pressure at P1 Ettercap

29 Tank fill level LIT101 Lower Falsify the water level reading of the tankdisplayed at SCADA

Pycomm

30 Chemical dosing PLC2 Use Studio 5000 Change the level of the chemical used fordosing

Studio 5000

31 Pressure MV302 P3012 Using Pycomm script Change the pump state sent to the PLC Pycomm

TABLE VCYBER CRIMINAL ATTACKS IN S3

Cyber Criminal Attacks Insider Attacks

S3-2016 4 16 1 2 3 5 6 7 8 910 11 12 13 14 1517 18

S3-2017 2 9 10 12 13 1516 17 18 19 20 2122 24 28 29 30

1 3 4 5 6 7 8 1114 23 25 26 27 31

To prevent all packets from entering the queue in order notto disrupt other processes iptables was used to identifythe targeted packets entering the queue Using Scapy and acustom dissector the attacker edited the payload of the targetedpacket which was then forwarded to its original destination

Control of Historian through the Aircrack WiFi The objectiveof this attack was (attack 17 in Table IV) to compromise thedata stored in the Historian Attackers performed crack WiFipassword ARP poisoning and MiTM payload manipulationusing Aircrack and Ettercap As PLC1 was operating inthe wireless mode the cybercriminal attacker used Aircrackto obtain the password for connecting to the ICS Access Point(AP) ARP poisoning was executed to reroute traffic betweenPLC1 and the Historian through the attackerrsquos rogue terminalThe attackers then used an Ettercap filter to manipulate thenetwork packets The attackers changed the tag correspondingto LIT101 to an arbitrary value before releasing the packetsto the Historian

Control of pressure through the Server Message Block (SMB)The objective of this attacks was (attack 18 in Table IV) todisrupt the state of four motorized valves in Stage 3 to affectthe differential pressure in UF Vulnerability CVE-2008-21601

in Factory Talk software from Rockwell and in MicrosoftrsquosServer Message Block (SMB) was used by the attackersto obtain files from the HMI As the HMI was runningWindows CE it has a vulnerability that allows an attackerrsquosterminal to execute arbitrary code on the HMI Thus theattackers were able to retrieve the files to create a copy ofthe workstation From the copied workstation the attackersmanually changed the state of the valves in Stage 3 suchthat the differential pressure across the UF unit as measuredby DPIT301 became dangerously high The attackers closedvalves MV301 MV302 and MV303 and opened MV304

Control of water level in the tank through the MetasploitVNC Scanner Objective of this attack was (attack 19 inTable IV) to change the water level in tank T101 The attackersused Metasploit VNC authentication None scanner to ob-tain access to the VNC server without password protection andto check for nodes running a VNC Server Once the scannerdetected the VNC Server running without any authenticationthe attackers penetrated into the server through a VNC Clientconnection As the VNC Server was hosting the HMI which

1httpswwwcvedetailscomcveCVE-2008-2160

controlled the ICS the attackers changed the simulation tagassociated with water level in T101

Control of a pump through a rogue router The objective ofthis attack (attack 20 in Table IV) was to disrupt the controlof pump P501 The attackers used Evil twin (rogue accesspoint) method using KisMAC a password cracking tool3vilTwinAttacker Telnet and Scapy The attackersused KisMAC to scan for wireless networks in the ICS Oncethe targeted wireless network was identified the attackers useddictionary attack to crack the password After the passwordwas cracked the attackers created a rogue wireless routerwith a similar SSID and configuration They then sent a de-authentication packet to disassociate PLC5 and the originalrouter The attackers used Telnet to log into the originalrouter and shut it down Scapy was then used to modify thepackets to turn the pump on

VII RESULTS

Tables VI and VII summarize the response of WD andWDH to the attacks launched during the two S3 events Recallthat both WD and WDH contain exactly the same set ofinvariants In WD the invariants are coded and placed insidethe PLCs whereas in WDH the invariants are coded and placedat the Historian WDH did not exist during S3-2016 and hencethe response of WDH is available only for attacks launchedduring S3-2017

A S3-2016 results

We note from Table VI that 10 out of 18 attacks weredetected immediately while the remaining eight attacks werenot detected Six of the eight undetected attacks did not leadto process anomaly during the observation period and hencedid not violate any invariant This outcome is expected as theinvariants in WD are designed to detect process anomaly

Consider attack 2 ARP spoofing in Table III This is aDoS attack on HMI It leads to defacing the screen on theHMI or displaying incorrect information thereby preventingan operator from knowing the actual plant state Howeverthe attack does not cause process anomaly and hence is notdetected as it does not violate any invariant Similar logic canbe used to explain why the other attacks in Table VI are notdetected

It is important to note that a DoS attack when given enoughtime to evolve and be launched at an appropriate state of theplant may impact physical process behavior In such a caseone or more invariants may detect the attack One such attackis 16 in Table VI This attack prevented the Historian fromreceiving data from PLC1 However if this attack was leftactive for a longer period it would prevent PLC1 from sendingappropriate commands to the actuators eg to MV101 orP101 In turn this would have led to process anomaly Notenough data is available to conclude with certainty whether ornot this attack would be detected by WD if active for sufficienttime

Two single point [2] attacks were not detected by WD Inone attack (attack 6 in Table III) the adversary altered the status

of valve MV301 Under normal circumstances this valve isopened during the backwash process However the attackeropened it when there was no backwash Hence the attackdid not affect the physical process except in changing thevalve status No invariant was violated due to this attackbecause the backwash process ie Stage 6 is not includedin this case study The second single point attack (attack 17 inTable III) was performed on chemical dosing pump P203 whilethe other pump P204 was running Note that under normalcircumstances only one of these two pumps is supposed to berunning while the other remains as a backup Subsequently theattacker shut down pump P204 This attack was not detectedbecause there were no invariants that related to the chemicalproperties of water

Although the overall performance of WD was below 100it did detect all attacks within its scope except two (attacks 6and 17 in Table III) as mentioned earlier

B S3-2017 results

Table VI indicates that 21 out of 31 attacks were detectedby WD while 24 out of 31 attacks were detected by WDHConsidering only the attacks within its scope as mentionedin Section V-A WD detected 21 out of 28 attacks (75)Similarly WDH detected 24 out of 31 attacks (7741) withinits scope mentioned in Section V-B Three attacks on theHistorian are not in the scope of WD All attack targets relatedto RIODisplay (in Table II and in Table IV) are not detectedby both WD and WDH This is because registers inside aPLC save the previous values received from the sensors andthe PLC continues to execute the control code The invariantsalso use the same values stored in the PLC registers and hencedo not raise an alert

In general PLCs send to the Historian via the SCADAworkstation the data received from the sensors When a PLCdoes not have updated values during the attack period it isobvious that the Historian also receives the same stale valuesThis is the reason why WDH also did not detect attacksrelated to RIODisplay Note that the RIODisplay attacks werelaunched and remained active only for a few seconds Duringthis period the PLC did not update the current sensor valuescoming through the RIO If the same attack is performed for alonger duration the PLC would update the data received fromthe sensors Doing so would likely lead to WD and WDHdetecting the RIO attacks

Attacks launched on the Historian were detected by WDHbut not by WD This variance is due to the fact that data inthese attacks is manipulated at the Historian Thus invariantsin a PLC do not have access to the manipulated data andhence the invariants in WD do not raise any alert All attackstargeting a PLC are detected by WD and WDH

WD Detection of physical process attacks All attacks onvalves pressure sensor and level sensors were detected Threeout of four attacks on the chemical dosing process pumpswere detected An example of a detected attack is when theattackers took control of pump P301 (attack 20 in Table IV)

TABLE VIPERFORMANCE OF WD AND WDH

S3-2016 S3-2017

WD WD WDH

Detected 1 5 7 910 11 1213 14 18

3 4 7 9 1011 12 13 1516 18 19 2021 22 23 2628 29 30 31

2 3 4 7 9 1011 12 13 15 1617 18 19 20 2122 23 24 26 2829 30 31

Not detected 2 3 4 68 15 1617

1 2 5 6 8 1417 24 25 27

1 5 6 8 14 2527

through a Python script (Pycomm) to raise the pressure in theUF unit measured by sensor DPIT301 to a dangerous levelWD immediately raised an alarm This invariant ensured thatpump P301 must be OFF when the pressure at DPIT301 wasabove a threshold During the attack the invariant was violatedas the pump was not turned off while DPIT301 indicatedreadings that were above the threshold Consequently an alarmwas raised immediately In certain cases multiple alarmswere raised due to the violation of one or more invariantsFor example when level sensor LIT101 was compromisedthe invariants corresponding to this sensor were violated andraised alarms

WD Detection of sensor data attack WD detected attacks onHMISCADA and PLC values because these attacks directlycompromised the physical processes These attacks eithercompromised chemical dosing water tank levels or pumpstatus through hacking of the HMISCADA or PLC Hencethe robustness of WD in detecting unusual physical processbehavior was found effective in these attacks On the otherhand WD was unable to detect insider attacks that pulled outRIO cables This is because WD triggers an alarm only whenthe invariants are violated Under normal circumstance for aperiod of time a PLC continues to execute its control codeand any invariant code based on the last known state andorvalues Thus the invariants located inside the PLCs are unableto observe this anomalous behavior

WDH Detection of physical process attacks WDH detected14 out of 16 physical process attacks

WDH Detection of sensor data attacks WDH detected theattacks on HMISCADA and PLC values because these attacksdirectly compromised the physical processes albeit with aslightly lower detection rate when compared with the rate ofdetecting physical process attacks As with WD WDH did notdetect any attack launched against the Remote IO by pullingthe cables that connect it to the corresponding PLC WDHfared better in the detection of attacks against the Historian asit was directly accessing data on the Historian server

If the Historian itself or data that is input to the Historian iscompromised WDH takes the decision based on the input itreceives A clever and powerful attacker can attack the physical

TABLE VIIRESULTS FROM S3 2017

Target of Attack Noofattacks

WD WDH

Physical Process Attacks

State of motorised valves 2 100 100

State of water pumps 4 75 75

Pressure in UF 2 100 100

Water tank level 4 100 100

Chemical dosing 4 75 75

Sensor Data Attacks

Data in historian 3 0 100

Data in HMISCADA 3 67 67

Tampering PLC communi-cations

5 100 100

Tampering Remote IO 4 0 0

Total Attacks 31 6774 7741

process and modify values entering the Historian and thusdeceive WDH In general such a situation may arise in allbehavioral intrusion detection systems where the detector takesthe decision based on incorrect input data

Indeed data that appears to be ldquolegitimaterdquo could lead theWDH into believing that there is nothing wrong with thephysical process though there actually is However doingso requires the attacker to continuously manipulate a largenumber of state variables For example consider an attackwhere the attacker turns a pump say P101 ON when it shouldbe OFF and (continually) sends the state of the pump as OFFto the Historian and the corresponding PLC If the pump isOFF then the level of the source and destination tanks must berespectively decreasing and increasing at rates determined bythe pump characteristics Creating ldquolegitimate-lookingrdquo datathus requires an attacker to manipulate several state variablesas explained next (a) Two state variables that correspondto tank levels Two sensors (in SWaT) measure these statevariables (see Figure 2) Thus the attacker must have accessto these level sensors (b) If pump P101 is actually ON whilethe Historian receives its state as OFF then FIT201 must showno flow Thus the attacker will also need to manipulate FIT201to avoid detection This argument can be carried forward tosubsequent stages to show that many sensors will need to bemanipulated by an attacker to ldquohiderdquo a simple attack such asldquochange the state of a pumprdquo In summary yes incorrect dataat the Historian could prevent detection though doing so wouldbe a significant challenge for the attacker due primarily to thedistributed nature of the invariants

VIII DISCUSSION

A Challenges faced

We faced several challenges during S3 For example aftereach teamrsquos performance the operator was required to bringSWaT back to a predefined normal state It was necessary to

keep SWaT in a normal state before another team launched at-tacks Bringing SWaT to its normal state required (a) resettingnetwork communications to ensure that all the communicationchannels are operating as expected (b) the operator to ensurethat all physical processes in SWaT are stable with respectto the control logic (c) the operator to bring back SWaT tothe normal state of that particular device such as a pump or amotorized valve in the case of any physical or manual attacksby the previous team and (d) that the Historian and SCADAservers were reverted to their original state ie the state thatexisted prior to the launch of attacks

B Research questions

RQ1 How do attackers compromise the security of an ICS InSection VI we presented and categorized the attacks based onattacker profiles An attacker can launch physical attacks wheninside the plant such as manually operating a motorized valveor tampering with network cabling Several attacks launchedby the attack teams had not been launched by the authorsin their evaluation of WD [1] and WDH Thus S3 raisedour confidence in the effectiveness of the attack detectionmechanisms based on invariants derived from plant designs

RQ2 How effective is WD in detecting attacks launched byindependent attack teams As mentioned earlier while bothWD and WDH were found to detect a number of attacksthey did fail in several cases Given that the invariants derivedare intended to detect process anomalies it is clear that suchmechanisms must be used in conjunction with other attackdetection tools such as those in [24] [27] [21]

C Assessment by the authors and by independent teamsTable VIII lists the number of attacks launched by the

authors in an experimental evaluation performed prior to S3-2016 [1] Note that the WD detection rate observed by theauthors (89) was higher than the combined rate observedduring the two S3 events (6326) The difference in perfor-mance is due to different attack vectors used in the three setsof experiments WDH detection rate observed during S3 eventis (7741) which is much higher than the WD detection rateSome of these attack vectors are explained in Section VI andthe remaining may be found in [20]

TABLE VIIIPERFORMANCE OF WD AS EVALUATED BY THE AUTHORS AGAINST THOSE

BY PARTICIPANTS IN S3

Experiments by Attacks

Launched Detected (WD) Detected (WDH)

Authors 37 33 (89) NA

S3-2016 18 10 (555) NA

S3-2017 31 21 (677) 24 (774)

NA WDH did not exist at the time of experimentation by the author andduring S3-2016

The data in Table VIII is indicative of the value of orga-nizing S3 events Specifically in the case described in this

paper the two S3 events led to an increased confidence inthe effectiveness of the invariant-based approach in detectingcyber attacks The hackfests also led to the creation of newtypes of attack vectors that were not used earlier to assess theperformance of WD and WDH in detecting cyber attacks

D False alarms

The performance of any attack detection method ought tobe assessed using its detection accuracy ie how many of thelaunched attacks it detects as well as the rate at which falsealarms are raised During S3 each team attempted to launchseveral attacks The attacks listed in Tables III and IV are theones that were successful in realizing the stated attacker intentand were scored by the judges The remaining attacks werenot recorded and hence any alarm generated by such attackswas not considered Some of these unrecorded alarms couldbe false though no specific claims can be made about theirnature

Since S3-2017 the authors have observed no false alarmsfrom WD during normal operation of SWaT WDH has beenin operation since a few weeks prior to S3-2017 Againduring the normal operation of SWaT no alarm has beengenerated by WDH This observation should not be construedto imply that an invariant-based attack detection mechanismwill not generate any false alarmndash in fact it could Howeverif the invariants generated are complete in the sense that theyaccurately capture all aspects of process behavior and theirimplementation is correct and tuned properly the likelihoodof false alarms is low

Even though SWaT is a relatively new plant (2-years sinceits inauguration at the time of writing this paper) we doobserve intermittent failures in a few motorized valves Forexample sometimes MV101 in Stage 1 takes much longer toopen than expected by its controlling PLC1 The PLC itselfdetects such cases In such a case WD or WDH dependingon the time it takes for the valve to finally open will raisean alarm We do not consider this as a false positive simplybecause whether an anomalous behavior is due to a naturalcause or a cyber attack cannot be distinguished by WD orWDH While such distinction is important to make additionalresearch is needed to distinguish process anomalies due tocyber attacks and those arising due to natural componentfailures

E Benefits of S3

S3 exposed the organisers participants and researchers tohow an attacker might design and launch attacks on ICS Bene-fits of S3 include the following 1) An improved understandingof how an ICS operates and the consequent formulation ofnew research directions 2) Opportunity for participants fromindustry and academia to learn from the event and focus onthe limitations of their work 3) An aid to the ICS managementteam to observe the defense teams thus leading to possibleadoption of technology embedded in WD or WDH

F Placement of WD

The placement of WD is another question that ought tobe looked into carefully In this work WD is placed insidePLCs However an exceptionally large number of invariantsmay prevent adding code to the existing control code in a PLCThis may happen due to the computational load requirementson a PLC This aspect led us to create WDH that is placedon the plant network and gets its data from the Historian toevaluate the invariants

G Forensics

One advantage of the invariant-based approach for attackdetection appears while determining the area of impact ofan attack When a single invariant is violated it indicatesclearly the source of process anomaly For example an alertis generated if valve MV101 is closed when the water intank T101 is at or below the L level marker While this alertdoes not indicate how an attacker entered the system or ifthe valve or the level sensor is defective it does assist inlocalising the reason for the alert The analysis becomes abit more complex when multiple invariants raise alerts Thisaspect of an invariant-based detection mechanisms remains tobe analyzed in further detail

H Attacker capabilities

We do not have any validation of the professionalism of theS3 attack teams As mentioned earlier [20] [35] [36] attackteams were from a variety of backgrounds including fromthe industry and academia from Europe and Asia During S3-2017 one team consisting of four membersndashall from outsideof Singaporendash focuses on ethical hacking and cyber-warsinvolving critical infrastructure This team is part of a globalalliance The other teams consist of hackers interested inknowing how vulnerabilities in software can be exploitedand passes this information to others for improving systemssecurity Coverage of attacks launched by the attack teams andattacker profiles is discussed in Section IV and summarizedin Tables I II V and VII

I Attack trees

It is possible to use attack trees [37] [42] to model attackslaunched during the two hackfests reported in this paper Doingso would enable mapping each attack to a specific path inthe attack tree and reveal which attack paths in SWaT weretraversed Such modeling and analysis has not been attemptedin this work and is a possible subject for future research

IX RELATED WORK

S3 is a Capture-The-Flag [15] event on ICS TraditionalCTF events generally attract the attention of both industrialand academic teams and currently enjoy increasing popularityas indicated in [15] The number of such events is graduallyincreasing [13] [16] Such events aid in learning about secu-rity vulnerabilities how these could be exploited nature ofattacks and strength of the deployed [18] [33] [45] defensemechanisms To the best of our knowledge S3 is the first CTF

style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed

The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks

There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc

The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies

Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS

X CONCLUSION

There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense

While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the

actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure

It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously

The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants

The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]

It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question

ACKNOWLEDGMENTS

A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript

REFERENCES

[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016

[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016

[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016

[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016

[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016

[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016

[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017

[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012

[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017

[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and

S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011

[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml

[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010

[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015

[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security

under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler

Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016

[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017

[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016

[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016

[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010

[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for

safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016

[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical

systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013

[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111

[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016

[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013

[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx

[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007

[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016

[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3

[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event

[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008

[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016

[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011

[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf

[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment

of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007

[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016

[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016

[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003

[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014

[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014

[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011

BIOGRAPHY

Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems

Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research

TABLE IIIATTACKS LAUNCHED DURING S3-2016

SNO Target Method Attack Tool

1 Tank fill levelLIT101

Use HMI access Close MV101 and Stop P101 andP102

HMI

2 HMISCADA ARP spoofing Attack HMI DoS attack Ettercap

3 PLC Manual access Removed the cable at the ring atlevel 0

Manual

4 HMISCADA DoS on HMI by droppingall packets between PLC andSCADAHMI

DoS attack on SCADA wide DoSattack took a while to restore SWaTto its normal state

Ettercap

5 Tank fill levelLIT101

Use HMI access Attack on LIT101 ManualHMI

6 Valve MV301 Use SCADA access Attack on MV301 manually openfrom the SCAD workstation

ManualSCADA

7 Pump P101 Use SCADA access Attack pump manually open it fromthe SCADA workstation

ManualSCADA

8 Historian DoS attack using CPPPO andloop

Attack between HMI and PLC CPPPO

9 Valve MV101 Use SCADA access MV101 attacked using SCADAchanged the valve state from Opento Closed

ManualSCADA

10 Pump P101 Use SCADA access LIT301 set point changed ManualSCADA

11 Tank fill levelLIT301

Using SCADA access LIT301 set point altered ManualSCADA

12 Chemical dosingP201

Control MV101 and AIT503 setpoints of LIT301 to ensure flowthis triggered chemical dosing

Dosing pump attack on P201 ManualSCADA

13 HMISCADALIT101

Functional block introduce newconstant tag tie that to output tagcould only do zero

LIT101 set to zero from PLC Studio5000

14 Chemical dosingpump P205

Use SCADA access Manipulation of the chemical dosingpump (P205)

ManualSCADA

15 HMISCADA DoS on HMI using Level 1 net-work

Attack on HMI EttercapPycomm

16 Historian SYN flood ENIP port at PLC1 DoS to PLC by SYN flooding (attackon HMI)

Ettercap

17 Chemical dosingpump P203

HMI-based direct manipulation Attack on P203 while the four dosingpumps are running

ManualHMI

18 HMISCADALIT101

Re-program PLC to fix LIT101value to an arbitrary value

Attack on LIT101 Studio5000

416 are cyber criminal attacks in S3-2016

Control of the chemical dosing system through a Pythonscript (Pycomm) The objective of this attack (attack 15 inTable IV) was to change chemical dosing at the end of the de-chlorination system (Stage 4) First the attackers compromisedVirtual Network Computing (VNC) Then they used a Pythonscript (Pycomm) and Wireshark to gain access to the HMIAfter gaining access to the HMI through the compromisedVNC the cybercriminal attacker used Wireshark to capturethe packets flowing between the HMI and PLC4 The con-troller tags were retrieved by an analysis of the packets Theattackers changed the data associated with these tags to controlthe chemical dosing function using the Pycomm framework

Control of PLC through the Bridged Man-in-the-Middle(MiTM) at Level 0 the objective of this attack was (attack 16in Table IV) to change the commands and values that PLC1receives and sends First the attackers configured a bridgebetween the RIO and PLC1 using Netfilterqueue andScapy The attack was launched at two network levels Ananalysis on the network traffic revealed the packets that theattackers should edit As the target of this attack was thewater level in T101 the attackers set it to a constant valueto hide from PLC1 the rise in water level in T101 Before apacket was forwarded Netfilterqueue rerouted it into aqueue which can be read and modified by the Python script

TABLE IVATTACKS LAUNCHED DURING S3-2017

SNo Target Method Attack Tool

1 HMISCADA LIT401 HMI simulation insider attack Change the value of LIT401 in the HMI Manual HMI

2 Historian ARP and drop Change the value stored at the Historian Ettercap

3 Valve MV201 Reprogram PLC Change the status of the MV201 Studio 5000

4 Tank fill level LIT301 420to 320

Manual Lower the water tank level from 820mm to420mm without raising any alarm LIT301decreased till 320mm

Manual HMI

5 Pump P101 Manual mode of pump Alternate the state [OnOff] of the pump P101 Manual HMI

6 Chemical dosing P205 Manually dosing chemical pump Change the chemical dosage of sodiumhypochlorite (NaOCl) in P2

Manual SCADA

7 PLC Disconnect cable Disrupt sensor values from remote inputoutput(RIO) to the PLC

Manual

8 RIO Display Disconnect IO PIN manual Disrupt the sensor reading send to PLC throughRemote IO (RIO)

Manual

9 Chemical dosing P404 MiTM Python script to control Increase chemical dosage in pre-treatment Python script

10 LIT101 (476mm to 540mm ) Reprogram PLC Falsify water level display at SCADA Studio 5000

11 Pump P101 HMI simulation insider attack Alternate the state [OnOff] of the pump P101 Manual HMI

12 HMISCADA AIT 504 ARP+rewriting Increase AIT504 Ettercap

13 PLC LIT401 Reprogram PLC Falsify water level display at SCADA Studio 5000

14 RIODisplay Disconnect specific IO PIN basedon manual

Disrupt the sensor reading send to PLC throughremote IO (RIO)

Manual

15 Chemical dosing pumpP403 AIT501

Based on captured traffic betweenHMI and PLC4

Change chemical dosing function VNC Python script PycommWireshark

16 PLC LT101 from 742mm to500mm

Level 0 MITM Change the commands and values that the PLCreceives and sends

Aircrack Airodump AireplayNetfilterqueue Scapy

17 Historian LT101 tag Aircrack WiFi ARP spoofingEttercap

Compromise historian data Ettercap Aircrack

18 Pressure sensor DPIT30130MV301-4

SMB to EW get project files runFT

Disrupt valves operation of Ultrafiltration andBackwash (P3)

SMB

19 MV201 LT101 metasploit+vnc Change the water level of the tank LIT101 Metasploit+vnc

20 Pump P501 Rogue AP disassociated Telnetwith default credentials to turnoff original AP Scapy rewrite

Disrupt pump control operation KisMAC Password crackingtool 3vilTwinAttacker TelnetScapy

21 PLC LIT101 Reprogram PLC Change level indicator value Studio 5000

22 Pump P101 LIT301 Using back-door connection Establish back-door connection Mimikatz malicious VBAMacro SOCKS proxy

23 HMISCADA P201 Netfilterqueue Scapy Change the display value of the HMI Netfilterqueue Scapy

24 Historian LIT101 Overwrote specific data stored at the Historian Microsoft PsExec ipconfig

25 RIODisplay Manual Control of the RIO through disconnected Ana-logue InputOutput pin

Manual

26 Valve MV201 Manual Permanently closed the motorised valve re-gardless of commands issued

Manual SCADA

27 RIODisplay AIT202203 Manual Change the pH value shown at HMI Manual HMI

28 PLC MV201 P101 ARP poisoning MiTM Increase the pressure at P1 Ettercap

29 Tank fill level LIT101 Lower Falsify the water level reading of the tankdisplayed at SCADA

Pycomm

30 Chemical dosing PLC2 Use Studio 5000 Change the level of the chemical used fordosing

Studio 5000

31 Pressure MV302 P3012 Using Pycomm script Change the pump state sent to the PLC Pycomm

TABLE VCYBER CRIMINAL ATTACKS IN S3

Cyber Criminal Attacks Insider Attacks

S3-2016 4 16 1 2 3 5 6 7 8 910 11 12 13 14 1517 18

S3-2017 2 9 10 12 13 1516 17 18 19 20 2122 24 28 29 30

1 3 4 5 6 7 8 1114 23 25 26 27 31

To prevent all packets from entering the queue in order notto disrupt other processes iptables was used to identifythe targeted packets entering the queue Using Scapy and acustom dissector the attacker edited the payload of the targetedpacket which was then forwarded to its original destination

Control of Historian through the Aircrack WiFi The objectiveof this attack was (attack 17 in Table IV) to compromise thedata stored in the Historian Attackers performed crack WiFipassword ARP poisoning and MiTM payload manipulationusing Aircrack and Ettercap As PLC1 was operating inthe wireless mode the cybercriminal attacker used Aircrackto obtain the password for connecting to the ICS Access Point(AP) ARP poisoning was executed to reroute traffic betweenPLC1 and the Historian through the attackerrsquos rogue terminalThe attackers then used an Ettercap filter to manipulate thenetwork packets The attackers changed the tag correspondingto LIT101 to an arbitrary value before releasing the packetsto the Historian

Control of pressure through the Server Message Block (SMB)The objective of this attacks was (attack 18 in Table IV) todisrupt the state of four motorized valves in Stage 3 to affectthe differential pressure in UF Vulnerability CVE-2008-21601

in Factory Talk software from Rockwell and in MicrosoftrsquosServer Message Block (SMB) was used by the attackersto obtain files from the HMI As the HMI was runningWindows CE it has a vulnerability that allows an attackerrsquosterminal to execute arbitrary code on the HMI Thus theattackers were able to retrieve the files to create a copy ofthe workstation From the copied workstation the attackersmanually changed the state of the valves in Stage 3 suchthat the differential pressure across the UF unit as measuredby DPIT301 became dangerously high The attackers closedvalves MV301 MV302 and MV303 and opened MV304

Control of water level in the tank through the MetasploitVNC Scanner Objective of this attack was (attack 19 inTable IV) to change the water level in tank T101 The attackersused Metasploit VNC authentication None scanner to ob-tain access to the VNC server without password protection andto check for nodes running a VNC Server Once the scannerdetected the VNC Server running without any authenticationthe attackers penetrated into the server through a VNC Clientconnection As the VNC Server was hosting the HMI which

1httpswwwcvedetailscomcveCVE-2008-2160

controlled the ICS the attackers changed the simulation tagassociated with water level in T101

Control of a pump through a rogue router The objective ofthis attack (attack 20 in Table IV) was to disrupt the controlof pump P501 The attackers used Evil twin (rogue accesspoint) method using KisMAC a password cracking tool3vilTwinAttacker Telnet and Scapy The attackersused KisMAC to scan for wireless networks in the ICS Oncethe targeted wireless network was identified the attackers useddictionary attack to crack the password After the passwordwas cracked the attackers created a rogue wireless routerwith a similar SSID and configuration They then sent a de-authentication packet to disassociate PLC5 and the originalrouter The attackers used Telnet to log into the originalrouter and shut it down Scapy was then used to modify thepackets to turn the pump on

VII RESULTS

Tables VI and VII summarize the response of WD andWDH to the attacks launched during the two S3 events Recallthat both WD and WDH contain exactly the same set ofinvariants In WD the invariants are coded and placed insidethe PLCs whereas in WDH the invariants are coded and placedat the Historian WDH did not exist during S3-2016 and hencethe response of WDH is available only for attacks launchedduring S3-2017

A S3-2016 results

We note from Table VI that 10 out of 18 attacks weredetected immediately while the remaining eight attacks werenot detected Six of the eight undetected attacks did not leadto process anomaly during the observation period and hencedid not violate any invariant This outcome is expected as theinvariants in WD are designed to detect process anomaly

Consider attack 2 ARP spoofing in Table III This is aDoS attack on HMI It leads to defacing the screen on theHMI or displaying incorrect information thereby preventingan operator from knowing the actual plant state Howeverthe attack does not cause process anomaly and hence is notdetected as it does not violate any invariant Similar logic canbe used to explain why the other attacks in Table VI are notdetected

It is important to note that a DoS attack when given enoughtime to evolve and be launched at an appropriate state of theplant may impact physical process behavior In such a caseone or more invariants may detect the attack One such attackis 16 in Table VI This attack prevented the Historian fromreceiving data from PLC1 However if this attack was leftactive for a longer period it would prevent PLC1 from sendingappropriate commands to the actuators eg to MV101 orP101 In turn this would have led to process anomaly Notenough data is available to conclude with certainty whether ornot this attack would be detected by WD if active for sufficienttime

Two single point [2] attacks were not detected by WD Inone attack (attack 6 in Table III) the adversary altered the status

of valve MV301 Under normal circumstances this valve isopened during the backwash process However the attackeropened it when there was no backwash Hence the attackdid not affect the physical process except in changing thevalve status No invariant was violated due to this attackbecause the backwash process ie Stage 6 is not includedin this case study The second single point attack (attack 17 inTable III) was performed on chemical dosing pump P203 whilethe other pump P204 was running Note that under normalcircumstances only one of these two pumps is supposed to berunning while the other remains as a backup Subsequently theattacker shut down pump P204 This attack was not detectedbecause there were no invariants that related to the chemicalproperties of water

Although the overall performance of WD was below 100it did detect all attacks within its scope except two (attacks 6and 17 in Table III) as mentioned earlier

B S3-2017 results

Table VI indicates that 21 out of 31 attacks were detectedby WD while 24 out of 31 attacks were detected by WDHConsidering only the attacks within its scope as mentionedin Section V-A WD detected 21 out of 28 attacks (75)Similarly WDH detected 24 out of 31 attacks (7741) withinits scope mentioned in Section V-B Three attacks on theHistorian are not in the scope of WD All attack targets relatedto RIODisplay (in Table II and in Table IV) are not detectedby both WD and WDH This is because registers inside aPLC save the previous values received from the sensors andthe PLC continues to execute the control code The invariantsalso use the same values stored in the PLC registers and hencedo not raise an alert

In general PLCs send to the Historian via the SCADAworkstation the data received from the sensors When a PLCdoes not have updated values during the attack period it isobvious that the Historian also receives the same stale valuesThis is the reason why WDH also did not detect attacksrelated to RIODisplay Note that the RIODisplay attacks werelaunched and remained active only for a few seconds Duringthis period the PLC did not update the current sensor valuescoming through the RIO If the same attack is performed for alonger duration the PLC would update the data received fromthe sensors Doing so would likely lead to WD and WDHdetecting the RIO attacks

Attacks launched on the Historian were detected by WDHbut not by WD This variance is due to the fact that data inthese attacks is manipulated at the Historian Thus invariantsin a PLC do not have access to the manipulated data andhence the invariants in WD do not raise any alert All attackstargeting a PLC are detected by WD and WDH

WD Detection of physical process attacks All attacks onvalves pressure sensor and level sensors were detected Threeout of four attacks on the chemical dosing process pumpswere detected An example of a detected attack is when theattackers took control of pump P301 (attack 20 in Table IV)

TABLE VIPERFORMANCE OF WD AND WDH

S3-2016 S3-2017

WD WD WDH

Detected 1 5 7 910 11 1213 14 18

3 4 7 9 1011 12 13 1516 18 19 2021 22 23 2628 29 30 31

2 3 4 7 9 1011 12 13 15 1617 18 19 20 2122 23 24 26 2829 30 31

Not detected 2 3 4 68 15 1617

1 2 5 6 8 1417 24 25 27

1 5 6 8 14 2527

through a Python script (Pycomm) to raise the pressure in theUF unit measured by sensor DPIT301 to a dangerous levelWD immediately raised an alarm This invariant ensured thatpump P301 must be OFF when the pressure at DPIT301 wasabove a threshold During the attack the invariant was violatedas the pump was not turned off while DPIT301 indicatedreadings that were above the threshold Consequently an alarmwas raised immediately In certain cases multiple alarmswere raised due to the violation of one or more invariantsFor example when level sensor LIT101 was compromisedthe invariants corresponding to this sensor were violated andraised alarms

WD Detection of sensor data attack WD detected attacks onHMISCADA and PLC values because these attacks directlycompromised the physical processes These attacks eithercompromised chemical dosing water tank levels or pumpstatus through hacking of the HMISCADA or PLC Hencethe robustness of WD in detecting unusual physical processbehavior was found effective in these attacks On the otherhand WD was unable to detect insider attacks that pulled outRIO cables This is because WD triggers an alarm only whenthe invariants are violated Under normal circumstance for aperiod of time a PLC continues to execute its control codeand any invariant code based on the last known state andorvalues Thus the invariants located inside the PLCs are unableto observe this anomalous behavior

WDH Detection of physical process attacks WDH detected14 out of 16 physical process attacks

WDH Detection of sensor data attacks WDH detected theattacks on HMISCADA and PLC values because these attacksdirectly compromised the physical processes albeit with aslightly lower detection rate when compared with the rate ofdetecting physical process attacks As with WD WDH did notdetect any attack launched against the Remote IO by pullingthe cables that connect it to the corresponding PLC WDHfared better in the detection of attacks against the Historian asit was directly accessing data on the Historian server

If the Historian itself or data that is input to the Historian iscompromised WDH takes the decision based on the input itreceives A clever and powerful attacker can attack the physical

TABLE VIIRESULTS FROM S3 2017

Target of Attack Noofattacks

WD WDH

Physical Process Attacks

State of motorised valves 2 100 100

State of water pumps 4 75 75

Pressure in UF 2 100 100

Water tank level 4 100 100

Chemical dosing 4 75 75

Sensor Data Attacks

Data in historian 3 0 100

Data in HMISCADA 3 67 67

Tampering PLC communi-cations

5 100 100

Tampering Remote IO 4 0 0

Total Attacks 31 6774 7741

process and modify values entering the Historian and thusdeceive WDH In general such a situation may arise in allbehavioral intrusion detection systems where the detector takesthe decision based on incorrect input data

Indeed data that appears to be ldquolegitimaterdquo could lead theWDH into believing that there is nothing wrong with thephysical process though there actually is However doingso requires the attacker to continuously manipulate a largenumber of state variables For example consider an attackwhere the attacker turns a pump say P101 ON when it shouldbe OFF and (continually) sends the state of the pump as OFFto the Historian and the corresponding PLC If the pump isOFF then the level of the source and destination tanks must berespectively decreasing and increasing at rates determined bythe pump characteristics Creating ldquolegitimate-lookingrdquo datathus requires an attacker to manipulate several state variablesas explained next (a) Two state variables that correspondto tank levels Two sensors (in SWaT) measure these statevariables (see Figure 2) Thus the attacker must have accessto these level sensors (b) If pump P101 is actually ON whilethe Historian receives its state as OFF then FIT201 must showno flow Thus the attacker will also need to manipulate FIT201to avoid detection This argument can be carried forward tosubsequent stages to show that many sensors will need to bemanipulated by an attacker to ldquohiderdquo a simple attack such asldquochange the state of a pumprdquo In summary yes incorrect dataat the Historian could prevent detection though doing so wouldbe a significant challenge for the attacker due primarily to thedistributed nature of the invariants

VIII DISCUSSION

A Challenges faced

We faced several challenges during S3 For example aftereach teamrsquos performance the operator was required to bringSWaT back to a predefined normal state It was necessary to

keep SWaT in a normal state before another team launched at-tacks Bringing SWaT to its normal state required (a) resettingnetwork communications to ensure that all the communicationchannels are operating as expected (b) the operator to ensurethat all physical processes in SWaT are stable with respectto the control logic (c) the operator to bring back SWaT tothe normal state of that particular device such as a pump or amotorized valve in the case of any physical or manual attacksby the previous team and (d) that the Historian and SCADAservers were reverted to their original state ie the state thatexisted prior to the launch of attacks

B Research questions

RQ1 How do attackers compromise the security of an ICS InSection VI we presented and categorized the attacks based onattacker profiles An attacker can launch physical attacks wheninside the plant such as manually operating a motorized valveor tampering with network cabling Several attacks launchedby the attack teams had not been launched by the authorsin their evaluation of WD [1] and WDH Thus S3 raisedour confidence in the effectiveness of the attack detectionmechanisms based on invariants derived from plant designs

RQ2 How effective is WD in detecting attacks launched byindependent attack teams As mentioned earlier while bothWD and WDH were found to detect a number of attacksthey did fail in several cases Given that the invariants derivedare intended to detect process anomalies it is clear that suchmechanisms must be used in conjunction with other attackdetection tools such as those in [24] [27] [21]

C Assessment by the authors and by independent teamsTable VIII lists the number of attacks launched by the

authors in an experimental evaluation performed prior to S3-2016 [1] Note that the WD detection rate observed by theauthors (89) was higher than the combined rate observedduring the two S3 events (6326) The difference in perfor-mance is due to different attack vectors used in the three setsof experiments WDH detection rate observed during S3 eventis (7741) which is much higher than the WD detection rateSome of these attack vectors are explained in Section VI andthe remaining may be found in [20]

TABLE VIIIPERFORMANCE OF WD AS EVALUATED BY THE AUTHORS AGAINST THOSE

BY PARTICIPANTS IN S3

Experiments by Attacks

Launched Detected (WD) Detected (WDH)

Authors 37 33 (89) NA

S3-2016 18 10 (555) NA

S3-2017 31 21 (677) 24 (774)

NA WDH did not exist at the time of experimentation by the author andduring S3-2016

The data in Table VIII is indicative of the value of orga-nizing S3 events Specifically in the case described in this

paper the two S3 events led to an increased confidence inthe effectiveness of the invariant-based approach in detectingcyber attacks The hackfests also led to the creation of newtypes of attack vectors that were not used earlier to assess theperformance of WD and WDH in detecting cyber attacks

D False alarms

The performance of any attack detection method ought tobe assessed using its detection accuracy ie how many of thelaunched attacks it detects as well as the rate at which falsealarms are raised During S3 each team attempted to launchseveral attacks The attacks listed in Tables III and IV are theones that were successful in realizing the stated attacker intentand were scored by the judges The remaining attacks werenot recorded and hence any alarm generated by such attackswas not considered Some of these unrecorded alarms couldbe false though no specific claims can be made about theirnature

Since S3-2017 the authors have observed no false alarmsfrom WD during normal operation of SWaT WDH has beenin operation since a few weeks prior to S3-2017 Againduring the normal operation of SWaT no alarm has beengenerated by WDH This observation should not be construedto imply that an invariant-based attack detection mechanismwill not generate any false alarmndash in fact it could Howeverif the invariants generated are complete in the sense that theyaccurately capture all aspects of process behavior and theirimplementation is correct and tuned properly the likelihoodof false alarms is low

Even though SWaT is a relatively new plant (2-years sinceits inauguration at the time of writing this paper) we doobserve intermittent failures in a few motorized valves Forexample sometimes MV101 in Stage 1 takes much longer toopen than expected by its controlling PLC1 The PLC itselfdetects such cases In such a case WD or WDH dependingon the time it takes for the valve to finally open will raisean alarm We do not consider this as a false positive simplybecause whether an anomalous behavior is due to a naturalcause or a cyber attack cannot be distinguished by WD orWDH While such distinction is important to make additionalresearch is needed to distinguish process anomalies due tocyber attacks and those arising due to natural componentfailures

E Benefits of S3

S3 exposed the organisers participants and researchers tohow an attacker might design and launch attacks on ICS Bene-fits of S3 include the following 1) An improved understandingof how an ICS operates and the consequent formulation ofnew research directions 2) Opportunity for participants fromindustry and academia to learn from the event and focus onthe limitations of their work 3) An aid to the ICS managementteam to observe the defense teams thus leading to possibleadoption of technology embedded in WD or WDH

F Placement of WD

The placement of WD is another question that ought tobe looked into carefully In this work WD is placed insidePLCs However an exceptionally large number of invariantsmay prevent adding code to the existing control code in a PLCThis may happen due to the computational load requirementson a PLC This aspect led us to create WDH that is placedon the plant network and gets its data from the Historian toevaluate the invariants

G Forensics

One advantage of the invariant-based approach for attackdetection appears while determining the area of impact ofan attack When a single invariant is violated it indicatesclearly the source of process anomaly For example an alertis generated if valve MV101 is closed when the water intank T101 is at or below the L level marker While this alertdoes not indicate how an attacker entered the system or ifthe valve or the level sensor is defective it does assist inlocalising the reason for the alert The analysis becomes abit more complex when multiple invariants raise alerts Thisaspect of an invariant-based detection mechanisms remains tobe analyzed in further detail

H Attacker capabilities

We do not have any validation of the professionalism of theS3 attack teams As mentioned earlier [20] [35] [36] attackteams were from a variety of backgrounds including fromthe industry and academia from Europe and Asia During S3-2017 one team consisting of four membersndashall from outsideof Singaporendash focuses on ethical hacking and cyber-warsinvolving critical infrastructure This team is part of a globalalliance The other teams consist of hackers interested inknowing how vulnerabilities in software can be exploitedand passes this information to others for improving systemssecurity Coverage of attacks launched by the attack teams andattacker profiles is discussed in Section IV and summarizedin Tables I II V and VII

I Attack trees

It is possible to use attack trees [37] [42] to model attackslaunched during the two hackfests reported in this paper Doingso would enable mapping each attack to a specific path inthe attack tree and reveal which attack paths in SWaT weretraversed Such modeling and analysis has not been attemptedin this work and is a possible subject for future research

IX RELATED WORK

S3 is a Capture-The-Flag [15] event on ICS TraditionalCTF events generally attract the attention of both industrialand academic teams and currently enjoy increasing popularityas indicated in [15] The number of such events is graduallyincreasing [13] [16] Such events aid in learning about secu-rity vulnerabilities how these could be exploited nature ofattacks and strength of the deployed [18] [33] [45] defensemechanisms To the best of our knowledge S3 is the first CTF

style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed

The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks

There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc

The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies

Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS

X CONCLUSION

There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense

While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the

actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure

It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously

The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants

The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]

It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question

ACKNOWLEDGMENTS

A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript

REFERENCES

[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016

[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016

[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016

[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016

[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016

[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016

[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017

[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012

[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017

[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and

S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011

[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml

[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010

[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015

[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security

under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler

Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016

[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017

[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016

[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016

[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010

[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for

safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016

[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical

systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013

[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111

[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016

[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013

[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx

[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007

[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016

[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3

[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event

[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008

[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016

[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011

[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf

[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment

of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007

[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016

[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016

[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003

[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014

[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014

[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011

BIOGRAPHY

Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems

Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research

TABLE IVATTACKS LAUNCHED DURING S3-2017

SNo Target Method Attack Tool

1 HMISCADA LIT401 HMI simulation insider attack Change the value of LIT401 in the HMI Manual HMI

2 Historian ARP and drop Change the value stored at the Historian Ettercap

3 Valve MV201 Reprogram PLC Change the status of the MV201 Studio 5000

4 Tank fill level LIT301 420to 320

Manual Lower the water tank level from 820mm to420mm without raising any alarm LIT301decreased till 320mm

Manual HMI

5 Pump P101 Manual mode of pump Alternate the state [OnOff] of the pump P101 Manual HMI

6 Chemical dosing P205 Manually dosing chemical pump Change the chemical dosage of sodiumhypochlorite (NaOCl) in P2

Manual SCADA

7 PLC Disconnect cable Disrupt sensor values from remote inputoutput(RIO) to the PLC

Manual

8 RIO Display Disconnect IO PIN manual Disrupt the sensor reading send to PLC throughRemote IO (RIO)

Manual

9 Chemical dosing P404 MiTM Python script to control Increase chemical dosage in pre-treatment Python script

10 LIT101 (476mm to 540mm ) Reprogram PLC Falsify water level display at SCADA Studio 5000

11 Pump P101 HMI simulation insider attack Alternate the state [OnOff] of the pump P101 Manual HMI

12 HMISCADA AIT 504 ARP+rewriting Increase AIT504 Ettercap

13 PLC LIT401 Reprogram PLC Falsify water level display at SCADA Studio 5000

14 RIODisplay Disconnect specific IO PIN basedon manual

Disrupt the sensor reading send to PLC throughremote IO (RIO)

Manual

15 Chemical dosing pumpP403 AIT501

Based on captured traffic betweenHMI and PLC4

Change chemical dosing function VNC Python script PycommWireshark

16 PLC LT101 from 742mm to500mm

Level 0 MITM Change the commands and values that the PLCreceives and sends

Aircrack Airodump AireplayNetfilterqueue Scapy

17 Historian LT101 tag Aircrack WiFi ARP spoofingEttercap

Compromise historian data Ettercap Aircrack

18 Pressure sensor DPIT30130MV301-4

SMB to EW get project files runFT

Disrupt valves operation of Ultrafiltration andBackwash (P3)

SMB

19 MV201 LT101 metasploit+vnc Change the water level of the tank LIT101 Metasploit+vnc

20 Pump P501 Rogue AP disassociated Telnetwith default credentials to turnoff original AP Scapy rewrite

Disrupt pump control operation KisMAC Password crackingtool 3vilTwinAttacker TelnetScapy

21 PLC LIT101 Reprogram PLC Change level indicator value Studio 5000

22 Pump P101 LIT301 Using back-door connection Establish back-door connection Mimikatz malicious VBAMacro SOCKS proxy

23 HMISCADA P201 Netfilterqueue Scapy Change the display value of the HMI Netfilterqueue Scapy

24 Historian LIT101 Overwrote specific data stored at the Historian Microsoft PsExec ipconfig

25 RIODisplay Manual Control of the RIO through disconnected Ana-logue InputOutput pin

Manual

26 Valve MV201 Manual Permanently closed the motorised valve re-gardless of commands issued

Manual SCADA

27 RIODisplay AIT202203 Manual Change the pH value shown at HMI Manual HMI

28 PLC MV201 P101 ARP poisoning MiTM Increase the pressure at P1 Ettercap

29 Tank fill level LIT101 Lower Falsify the water level reading of the tankdisplayed at SCADA

Pycomm

30 Chemical dosing PLC2 Use Studio 5000 Change the level of the chemical used fordosing

Studio 5000

31 Pressure MV302 P3012 Using Pycomm script Change the pump state sent to the PLC Pycomm

TABLE VCYBER CRIMINAL ATTACKS IN S3

Cyber Criminal Attacks Insider Attacks

S3-2016 4 16 1 2 3 5 6 7 8 910 11 12 13 14 1517 18

S3-2017 2 9 10 12 13 1516 17 18 19 20 2122 24 28 29 30

1 3 4 5 6 7 8 1114 23 25 26 27 31

To prevent all packets from entering the queue in order notto disrupt other processes iptables was used to identifythe targeted packets entering the queue Using Scapy and acustom dissector the attacker edited the payload of the targetedpacket which was then forwarded to its original destination

Control of Historian through the Aircrack WiFi The objectiveof this attack was (attack 17 in Table IV) to compromise thedata stored in the Historian Attackers performed crack WiFipassword ARP poisoning and MiTM payload manipulationusing Aircrack and Ettercap As PLC1 was operating inthe wireless mode the cybercriminal attacker used Aircrackto obtain the password for connecting to the ICS Access Point(AP) ARP poisoning was executed to reroute traffic betweenPLC1 and the Historian through the attackerrsquos rogue terminalThe attackers then used an Ettercap filter to manipulate thenetwork packets The attackers changed the tag correspondingto LIT101 to an arbitrary value before releasing the packetsto the Historian

Control of pressure through the Server Message Block (SMB)The objective of this attacks was (attack 18 in Table IV) todisrupt the state of four motorized valves in Stage 3 to affectthe differential pressure in UF Vulnerability CVE-2008-21601

in Factory Talk software from Rockwell and in MicrosoftrsquosServer Message Block (SMB) was used by the attackersto obtain files from the HMI As the HMI was runningWindows CE it has a vulnerability that allows an attackerrsquosterminal to execute arbitrary code on the HMI Thus theattackers were able to retrieve the files to create a copy ofthe workstation From the copied workstation the attackersmanually changed the state of the valves in Stage 3 suchthat the differential pressure across the UF unit as measuredby DPIT301 became dangerously high The attackers closedvalves MV301 MV302 and MV303 and opened MV304

Control of water level in the tank through the MetasploitVNC Scanner Objective of this attack was (attack 19 inTable IV) to change the water level in tank T101 The attackersused Metasploit VNC authentication None scanner to ob-tain access to the VNC server without password protection andto check for nodes running a VNC Server Once the scannerdetected the VNC Server running without any authenticationthe attackers penetrated into the server through a VNC Clientconnection As the VNC Server was hosting the HMI which

1httpswwwcvedetailscomcveCVE-2008-2160

controlled the ICS the attackers changed the simulation tagassociated with water level in T101

Control of a pump through a rogue router The objective ofthis attack (attack 20 in Table IV) was to disrupt the controlof pump P501 The attackers used Evil twin (rogue accesspoint) method using KisMAC a password cracking tool3vilTwinAttacker Telnet and Scapy The attackersused KisMAC to scan for wireless networks in the ICS Oncethe targeted wireless network was identified the attackers useddictionary attack to crack the password After the passwordwas cracked the attackers created a rogue wireless routerwith a similar SSID and configuration They then sent a de-authentication packet to disassociate PLC5 and the originalrouter The attackers used Telnet to log into the originalrouter and shut it down Scapy was then used to modify thepackets to turn the pump on

VII RESULTS

Tables VI and VII summarize the response of WD andWDH to the attacks launched during the two S3 events Recallthat both WD and WDH contain exactly the same set ofinvariants In WD the invariants are coded and placed insidethe PLCs whereas in WDH the invariants are coded and placedat the Historian WDH did not exist during S3-2016 and hencethe response of WDH is available only for attacks launchedduring S3-2017

A S3-2016 results

We note from Table VI that 10 out of 18 attacks weredetected immediately while the remaining eight attacks werenot detected Six of the eight undetected attacks did not leadto process anomaly during the observation period and hencedid not violate any invariant This outcome is expected as theinvariants in WD are designed to detect process anomaly

Consider attack 2 ARP spoofing in Table III This is aDoS attack on HMI It leads to defacing the screen on theHMI or displaying incorrect information thereby preventingan operator from knowing the actual plant state Howeverthe attack does not cause process anomaly and hence is notdetected as it does not violate any invariant Similar logic canbe used to explain why the other attacks in Table VI are notdetected

It is important to note that a DoS attack when given enoughtime to evolve and be launched at an appropriate state of theplant may impact physical process behavior In such a caseone or more invariants may detect the attack One such attackis 16 in Table VI This attack prevented the Historian fromreceiving data from PLC1 However if this attack was leftactive for a longer period it would prevent PLC1 from sendingappropriate commands to the actuators eg to MV101 orP101 In turn this would have led to process anomaly Notenough data is available to conclude with certainty whether ornot this attack would be detected by WD if active for sufficienttime

Two single point [2] attacks were not detected by WD Inone attack (attack 6 in Table III) the adversary altered the status

of valve MV301 Under normal circumstances this valve isopened during the backwash process However the attackeropened it when there was no backwash Hence the attackdid not affect the physical process except in changing thevalve status No invariant was violated due to this attackbecause the backwash process ie Stage 6 is not includedin this case study The second single point attack (attack 17 inTable III) was performed on chemical dosing pump P203 whilethe other pump P204 was running Note that under normalcircumstances only one of these two pumps is supposed to berunning while the other remains as a backup Subsequently theattacker shut down pump P204 This attack was not detectedbecause there were no invariants that related to the chemicalproperties of water

Although the overall performance of WD was below 100it did detect all attacks within its scope except two (attacks 6and 17 in Table III) as mentioned earlier

B S3-2017 results

Table VI indicates that 21 out of 31 attacks were detectedby WD while 24 out of 31 attacks were detected by WDHConsidering only the attacks within its scope as mentionedin Section V-A WD detected 21 out of 28 attacks (75)Similarly WDH detected 24 out of 31 attacks (7741) withinits scope mentioned in Section V-B Three attacks on theHistorian are not in the scope of WD All attack targets relatedto RIODisplay (in Table II and in Table IV) are not detectedby both WD and WDH This is because registers inside aPLC save the previous values received from the sensors andthe PLC continues to execute the control code The invariantsalso use the same values stored in the PLC registers and hencedo not raise an alert

In general PLCs send to the Historian via the SCADAworkstation the data received from the sensors When a PLCdoes not have updated values during the attack period it isobvious that the Historian also receives the same stale valuesThis is the reason why WDH also did not detect attacksrelated to RIODisplay Note that the RIODisplay attacks werelaunched and remained active only for a few seconds Duringthis period the PLC did not update the current sensor valuescoming through the RIO If the same attack is performed for alonger duration the PLC would update the data received fromthe sensors Doing so would likely lead to WD and WDHdetecting the RIO attacks

Attacks launched on the Historian were detected by WDHbut not by WD This variance is due to the fact that data inthese attacks is manipulated at the Historian Thus invariantsin a PLC do not have access to the manipulated data andhence the invariants in WD do not raise any alert All attackstargeting a PLC are detected by WD and WDH

WD Detection of physical process attacks All attacks onvalves pressure sensor and level sensors were detected Threeout of four attacks on the chemical dosing process pumpswere detected An example of a detected attack is when theattackers took control of pump P301 (attack 20 in Table IV)

TABLE VIPERFORMANCE OF WD AND WDH

S3-2016 S3-2017

WD WD WDH

Detected 1 5 7 910 11 1213 14 18

3 4 7 9 1011 12 13 1516 18 19 2021 22 23 2628 29 30 31

2 3 4 7 9 1011 12 13 15 1617 18 19 20 2122 23 24 26 2829 30 31

Not detected 2 3 4 68 15 1617

1 2 5 6 8 1417 24 25 27

1 5 6 8 14 2527

through a Python script (Pycomm) to raise the pressure in theUF unit measured by sensor DPIT301 to a dangerous levelWD immediately raised an alarm This invariant ensured thatpump P301 must be OFF when the pressure at DPIT301 wasabove a threshold During the attack the invariant was violatedas the pump was not turned off while DPIT301 indicatedreadings that were above the threshold Consequently an alarmwas raised immediately In certain cases multiple alarmswere raised due to the violation of one or more invariantsFor example when level sensor LIT101 was compromisedthe invariants corresponding to this sensor were violated andraised alarms

WD Detection of sensor data attack WD detected attacks onHMISCADA and PLC values because these attacks directlycompromised the physical processes These attacks eithercompromised chemical dosing water tank levels or pumpstatus through hacking of the HMISCADA or PLC Hencethe robustness of WD in detecting unusual physical processbehavior was found effective in these attacks On the otherhand WD was unable to detect insider attacks that pulled outRIO cables This is because WD triggers an alarm only whenthe invariants are violated Under normal circumstance for aperiod of time a PLC continues to execute its control codeand any invariant code based on the last known state andorvalues Thus the invariants located inside the PLCs are unableto observe this anomalous behavior

WDH Detection of physical process attacks WDH detected14 out of 16 physical process attacks

WDH Detection of sensor data attacks WDH detected theattacks on HMISCADA and PLC values because these attacksdirectly compromised the physical processes albeit with aslightly lower detection rate when compared with the rate ofdetecting physical process attacks As with WD WDH did notdetect any attack launched against the Remote IO by pullingthe cables that connect it to the corresponding PLC WDHfared better in the detection of attacks against the Historian asit was directly accessing data on the Historian server

If the Historian itself or data that is input to the Historian iscompromised WDH takes the decision based on the input itreceives A clever and powerful attacker can attack the physical

TABLE VIIRESULTS FROM S3 2017

Target of Attack Noofattacks

WD WDH

Physical Process Attacks

State of motorised valves 2 100 100

State of water pumps 4 75 75

Pressure in UF 2 100 100

Water tank level 4 100 100

Chemical dosing 4 75 75

Sensor Data Attacks

Data in historian 3 0 100

Data in HMISCADA 3 67 67

Tampering PLC communi-cations

5 100 100

Tampering Remote IO 4 0 0

Total Attacks 31 6774 7741

process and modify values entering the Historian and thusdeceive WDH In general such a situation may arise in allbehavioral intrusion detection systems where the detector takesthe decision based on incorrect input data

Indeed data that appears to be ldquolegitimaterdquo could lead theWDH into believing that there is nothing wrong with thephysical process though there actually is However doingso requires the attacker to continuously manipulate a largenumber of state variables For example consider an attackwhere the attacker turns a pump say P101 ON when it shouldbe OFF and (continually) sends the state of the pump as OFFto the Historian and the corresponding PLC If the pump isOFF then the level of the source and destination tanks must berespectively decreasing and increasing at rates determined bythe pump characteristics Creating ldquolegitimate-lookingrdquo datathus requires an attacker to manipulate several state variablesas explained next (a) Two state variables that correspondto tank levels Two sensors (in SWaT) measure these statevariables (see Figure 2) Thus the attacker must have accessto these level sensors (b) If pump P101 is actually ON whilethe Historian receives its state as OFF then FIT201 must showno flow Thus the attacker will also need to manipulate FIT201to avoid detection This argument can be carried forward tosubsequent stages to show that many sensors will need to bemanipulated by an attacker to ldquohiderdquo a simple attack such asldquochange the state of a pumprdquo In summary yes incorrect dataat the Historian could prevent detection though doing so wouldbe a significant challenge for the attacker due primarily to thedistributed nature of the invariants

VIII DISCUSSION

A Challenges faced

We faced several challenges during S3 For example aftereach teamrsquos performance the operator was required to bringSWaT back to a predefined normal state It was necessary to

keep SWaT in a normal state before another team launched at-tacks Bringing SWaT to its normal state required (a) resettingnetwork communications to ensure that all the communicationchannels are operating as expected (b) the operator to ensurethat all physical processes in SWaT are stable with respectto the control logic (c) the operator to bring back SWaT tothe normal state of that particular device such as a pump or amotorized valve in the case of any physical or manual attacksby the previous team and (d) that the Historian and SCADAservers were reverted to their original state ie the state thatexisted prior to the launch of attacks

B Research questions

RQ1 How do attackers compromise the security of an ICS InSection VI we presented and categorized the attacks based onattacker profiles An attacker can launch physical attacks wheninside the plant such as manually operating a motorized valveor tampering with network cabling Several attacks launchedby the attack teams had not been launched by the authorsin their evaluation of WD [1] and WDH Thus S3 raisedour confidence in the effectiveness of the attack detectionmechanisms based on invariants derived from plant designs

RQ2 How effective is WD in detecting attacks launched byindependent attack teams As mentioned earlier while bothWD and WDH were found to detect a number of attacksthey did fail in several cases Given that the invariants derivedare intended to detect process anomalies it is clear that suchmechanisms must be used in conjunction with other attackdetection tools such as those in [24] [27] [21]

C Assessment by the authors and by independent teamsTable VIII lists the number of attacks launched by the

authors in an experimental evaluation performed prior to S3-2016 [1] Note that the WD detection rate observed by theauthors (89) was higher than the combined rate observedduring the two S3 events (6326) The difference in perfor-mance is due to different attack vectors used in the three setsof experiments WDH detection rate observed during S3 eventis (7741) which is much higher than the WD detection rateSome of these attack vectors are explained in Section VI andthe remaining may be found in [20]

TABLE VIIIPERFORMANCE OF WD AS EVALUATED BY THE AUTHORS AGAINST THOSE

BY PARTICIPANTS IN S3

Experiments by Attacks

Launched Detected (WD) Detected (WDH)

Authors 37 33 (89) NA

S3-2016 18 10 (555) NA

S3-2017 31 21 (677) 24 (774)

NA WDH did not exist at the time of experimentation by the author andduring S3-2016

The data in Table VIII is indicative of the value of orga-nizing S3 events Specifically in the case described in this

paper the two S3 events led to an increased confidence inthe effectiveness of the invariant-based approach in detectingcyber attacks The hackfests also led to the creation of newtypes of attack vectors that were not used earlier to assess theperformance of WD and WDH in detecting cyber attacks

D False alarms

The performance of any attack detection method ought tobe assessed using its detection accuracy ie how many of thelaunched attacks it detects as well as the rate at which falsealarms are raised During S3 each team attempted to launchseveral attacks The attacks listed in Tables III and IV are theones that were successful in realizing the stated attacker intentand were scored by the judges The remaining attacks werenot recorded and hence any alarm generated by such attackswas not considered Some of these unrecorded alarms couldbe false though no specific claims can be made about theirnature

Since S3-2017 the authors have observed no false alarmsfrom WD during normal operation of SWaT WDH has beenin operation since a few weeks prior to S3-2017 Againduring the normal operation of SWaT no alarm has beengenerated by WDH This observation should not be construedto imply that an invariant-based attack detection mechanismwill not generate any false alarmndash in fact it could Howeverif the invariants generated are complete in the sense that theyaccurately capture all aspects of process behavior and theirimplementation is correct and tuned properly the likelihoodof false alarms is low

Even though SWaT is a relatively new plant (2-years sinceits inauguration at the time of writing this paper) we doobserve intermittent failures in a few motorized valves Forexample sometimes MV101 in Stage 1 takes much longer toopen than expected by its controlling PLC1 The PLC itselfdetects such cases In such a case WD or WDH dependingon the time it takes for the valve to finally open will raisean alarm We do not consider this as a false positive simplybecause whether an anomalous behavior is due to a naturalcause or a cyber attack cannot be distinguished by WD orWDH While such distinction is important to make additionalresearch is needed to distinguish process anomalies due tocyber attacks and those arising due to natural componentfailures

E Benefits of S3

S3 exposed the organisers participants and researchers tohow an attacker might design and launch attacks on ICS Bene-fits of S3 include the following 1) An improved understandingof how an ICS operates and the consequent formulation ofnew research directions 2) Opportunity for participants fromindustry and academia to learn from the event and focus onthe limitations of their work 3) An aid to the ICS managementteam to observe the defense teams thus leading to possibleadoption of technology embedded in WD or WDH

F Placement of WD

The placement of WD is another question that ought tobe looked into carefully In this work WD is placed insidePLCs However an exceptionally large number of invariantsmay prevent adding code to the existing control code in a PLCThis may happen due to the computational load requirementson a PLC This aspect led us to create WDH that is placedon the plant network and gets its data from the Historian toevaluate the invariants

G Forensics

One advantage of the invariant-based approach for attackdetection appears while determining the area of impact ofan attack When a single invariant is violated it indicatesclearly the source of process anomaly For example an alertis generated if valve MV101 is closed when the water intank T101 is at or below the L level marker While this alertdoes not indicate how an attacker entered the system or ifthe valve or the level sensor is defective it does assist inlocalising the reason for the alert The analysis becomes abit more complex when multiple invariants raise alerts Thisaspect of an invariant-based detection mechanisms remains tobe analyzed in further detail

H Attacker capabilities

We do not have any validation of the professionalism of theS3 attack teams As mentioned earlier [20] [35] [36] attackteams were from a variety of backgrounds including fromthe industry and academia from Europe and Asia During S3-2017 one team consisting of four membersndashall from outsideof Singaporendash focuses on ethical hacking and cyber-warsinvolving critical infrastructure This team is part of a globalalliance The other teams consist of hackers interested inknowing how vulnerabilities in software can be exploitedand passes this information to others for improving systemssecurity Coverage of attacks launched by the attack teams andattacker profiles is discussed in Section IV and summarizedin Tables I II V and VII

I Attack trees

It is possible to use attack trees [37] [42] to model attackslaunched during the two hackfests reported in this paper Doingso would enable mapping each attack to a specific path inthe attack tree and reveal which attack paths in SWaT weretraversed Such modeling and analysis has not been attemptedin this work and is a possible subject for future research

IX RELATED WORK

S3 is a Capture-The-Flag [15] event on ICS TraditionalCTF events generally attract the attention of both industrialand academic teams and currently enjoy increasing popularityas indicated in [15] The number of such events is graduallyincreasing [13] [16] Such events aid in learning about secu-rity vulnerabilities how these could be exploited nature ofattacks and strength of the deployed [18] [33] [45] defensemechanisms To the best of our knowledge S3 is the first CTF

style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed

The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks

There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc

The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies

Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS

X CONCLUSION

There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense

While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the

actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure

It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously

The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants

The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]

It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question

ACKNOWLEDGMENTS

A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript

REFERENCES

[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016

[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016

[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016

[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016

[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016

[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016

[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017

[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012

[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017

[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and

S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011

[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml

[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010

[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015

[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security

under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler

Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016

[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017

[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016

[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016

[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010

[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for

safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016

[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical

systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013

[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111

[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016

[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013

[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx

[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007

[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016

[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3

[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event

[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008

[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016

[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011

[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf

[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment

of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007

[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016

[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016

[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003

[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014

[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014

[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011

BIOGRAPHY

Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems

Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research

TABLE VCYBER CRIMINAL ATTACKS IN S3

Cyber Criminal Attacks Insider Attacks

S3-2016 4 16 1 2 3 5 6 7 8 910 11 12 13 14 1517 18

S3-2017 2 9 10 12 13 1516 17 18 19 20 2122 24 28 29 30

1 3 4 5 6 7 8 1114 23 25 26 27 31

To prevent all packets from entering the queue in order notto disrupt other processes iptables was used to identifythe targeted packets entering the queue Using Scapy and acustom dissector the attacker edited the payload of the targetedpacket which was then forwarded to its original destination

Control of Historian through the Aircrack WiFi The objectiveof this attack was (attack 17 in Table IV) to compromise thedata stored in the Historian Attackers performed crack WiFipassword ARP poisoning and MiTM payload manipulationusing Aircrack and Ettercap As PLC1 was operating inthe wireless mode the cybercriminal attacker used Aircrackto obtain the password for connecting to the ICS Access Point(AP) ARP poisoning was executed to reroute traffic betweenPLC1 and the Historian through the attackerrsquos rogue terminalThe attackers then used an Ettercap filter to manipulate thenetwork packets The attackers changed the tag correspondingto LIT101 to an arbitrary value before releasing the packetsto the Historian

Control of pressure through the Server Message Block (SMB)The objective of this attacks was (attack 18 in Table IV) todisrupt the state of four motorized valves in Stage 3 to affectthe differential pressure in UF Vulnerability CVE-2008-21601

in Factory Talk software from Rockwell and in MicrosoftrsquosServer Message Block (SMB) was used by the attackersto obtain files from the HMI As the HMI was runningWindows CE it has a vulnerability that allows an attackerrsquosterminal to execute arbitrary code on the HMI Thus theattackers were able to retrieve the files to create a copy ofthe workstation From the copied workstation the attackersmanually changed the state of the valves in Stage 3 suchthat the differential pressure across the UF unit as measuredby DPIT301 became dangerously high The attackers closedvalves MV301 MV302 and MV303 and opened MV304

Control of water level in the tank through the MetasploitVNC Scanner Objective of this attack was (attack 19 inTable IV) to change the water level in tank T101 The attackersused Metasploit VNC authentication None scanner to ob-tain access to the VNC server without password protection andto check for nodes running a VNC Server Once the scannerdetected the VNC Server running without any authenticationthe attackers penetrated into the server through a VNC Clientconnection As the VNC Server was hosting the HMI which

1httpswwwcvedetailscomcveCVE-2008-2160

controlled the ICS the attackers changed the simulation tagassociated with water level in T101

Control of a pump through a rogue router The objective ofthis attack (attack 20 in Table IV) was to disrupt the controlof pump P501 The attackers used Evil twin (rogue accesspoint) method using KisMAC a password cracking tool3vilTwinAttacker Telnet and Scapy The attackersused KisMAC to scan for wireless networks in the ICS Oncethe targeted wireless network was identified the attackers useddictionary attack to crack the password After the passwordwas cracked the attackers created a rogue wireless routerwith a similar SSID and configuration They then sent a de-authentication packet to disassociate PLC5 and the originalrouter The attackers used Telnet to log into the originalrouter and shut it down Scapy was then used to modify thepackets to turn the pump on

VII RESULTS

Tables VI and VII summarize the response of WD andWDH to the attacks launched during the two S3 events Recallthat both WD and WDH contain exactly the same set ofinvariants In WD the invariants are coded and placed insidethe PLCs whereas in WDH the invariants are coded and placedat the Historian WDH did not exist during S3-2016 and hencethe response of WDH is available only for attacks launchedduring S3-2017

A S3-2016 results

We note from Table VI that 10 out of 18 attacks weredetected immediately while the remaining eight attacks werenot detected Six of the eight undetected attacks did not leadto process anomaly during the observation period and hencedid not violate any invariant This outcome is expected as theinvariants in WD are designed to detect process anomaly

Consider attack 2 ARP spoofing in Table III This is aDoS attack on HMI It leads to defacing the screen on theHMI or displaying incorrect information thereby preventingan operator from knowing the actual plant state Howeverthe attack does not cause process anomaly and hence is notdetected as it does not violate any invariant Similar logic canbe used to explain why the other attacks in Table VI are notdetected

It is important to note that a DoS attack when given enoughtime to evolve and be launched at an appropriate state of theplant may impact physical process behavior In such a caseone or more invariants may detect the attack One such attackis 16 in Table VI This attack prevented the Historian fromreceiving data from PLC1 However if this attack was leftactive for a longer period it would prevent PLC1 from sendingappropriate commands to the actuators eg to MV101 orP101 In turn this would have led to process anomaly Notenough data is available to conclude with certainty whether ornot this attack would be detected by WD if active for sufficienttime

Two single point [2] attacks were not detected by WD Inone attack (attack 6 in Table III) the adversary altered the status

of valve MV301 Under normal circumstances this valve isopened during the backwash process However the attackeropened it when there was no backwash Hence the attackdid not affect the physical process except in changing thevalve status No invariant was violated due to this attackbecause the backwash process ie Stage 6 is not includedin this case study The second single point attack (attack 17 inTable III) was performed on chemical dosing pump P203 whilethe other pump P204 was running Note that under normalcircumstances only one of these two pumps is supposed to berunning while the other remains as a backup Subsequently theattacker shut down pump P204 This attack was not detectedbecause there were no invariants that related to the chemicalproperties of water

Although the overall performance of WD was below 100it did detect all attacks within its scope except two (attacks 6and 17 in Table III) as mentioned earlier

B S3-2017 results

Table VI indicates that 21 out of 31 attacks were detectedby WD while 24 out of 31 attacks were detected by WDHConsidering only the attacks within its scope as mentionedin Section V-A WD detected 21 out of 28 attacks (75)Similarly WDH detected 24 out of 31 attacks (7741) withinits scope mentioned in Section V-B Three attacks on theHistorian are not in the scope of WD All attack targets relatedto RIODisplay (in Table II and in Table IV) are not detectedby both WD and WDH This is because registers inside aPLC save the previous values received from the sensors andthe PLC continues to execute the control code The invariantsalso use the same values stored in the PLC registers and hencedo not raise an alert

In general PLCs send to the Historian via the SCADAworkstation the data received from the sensors When a PLCdoes not have updated values during the attack period it isobvious that the Historian also receives the same stale valuesThis is the reason why WDH also did not detect attacksrelated to RIODisplay Note that the RIODisplay attacks werelaunched and remained active only for a few seconds Duringthis period the PLC did not update the current sensor valuescoming through the RIO If the same attack is performed for alonger duration the PLC would update the data received fromthe sensors Doing so would likely lead to WD and WDHdetecting the RIO attacks

Attacks launched on the Historian were detected by WDHbut not by WD This variance is due to the fact that data inthese attacks is manipulated at the Historian Thus invariantsin a PLC do not have access to the manipulated data andhence the invariants in WD do not raise any alert All attackstargeting a PLC are detected by WD and WDH

WD Detection of physical process attacks All attacks onvalves pressure sensor and level sensors were detected Threeout of four attacks on the chemical dosing process pumpswere detected An example of a detected attack is when theattackers took control of pump P301 (attack 20 in Table IV)

TABLE VIPERFORMANCE OF WD AND WDH

S3-2016 S3-2017

WD WD WDH

Detected 1 5 7 910 11 1213 14 18

3 4 7 9 1011 12 13 1516 18 19 2021 22 23 2628 29 30 31

2 3 4 7 9 1011 12 13 15 1617 18 19 20 2122 23 24 26 2829 30 31

Not detected 2 3 4 68 15 1617

1 2 5 6 8 1417 24 25 27

1 5 6 8 14 2527

through a Python script (Pycomm) to raise the pressure in theUF unit measured by sensor DPIT301 to a dangerous levelWD immediately raised an alarm This invariant ensured thatpump P301 must be OFF when the pressure at DPIT301 wasabove a threshold During the attack the invariant was violatedas the pump was not turned off while DPIT301 indicatedreadings that were above the threshold Consequently an alarmwas raised immediately In certain cases multiple alarmswere raised due to the violation of one or more invariantsFor example when level sensor LIT101 was compromisedthe invariants corresponding to this sensor were violated andraised alarms

WD Detection of sensor data attack WD detected attacks onHMISCADA and PLC values because these attacks directlycompromised the physical processes These attacks eithercompromised chemical dosing water tank levels or pumpstatus through hacking of the HMISCADA or PLC Hencethe robustness of WD in detecting unusual physical processbehavior was found effective in these attacks On the otherhand WD was unable to detect insider attacks that pulled outRIO cables This is because WD triggers an alarm only whenthe invariants are violated Under normal circumstance for aperiod of time a PLC continues to execute its control codeand any invariant code based on the last known state andorvalues Thus the invariants located inside the PLCs are unableto observe this anomalous behavior

WDH Detection of physical process attacks WDH detected14 out of 16 physical process attacks

WDH Detection of sensor data attacks WDH detected theattacks on HMISCADA and PLC values because these attacksdirectly compromised the physical processes albeit with aslightly lower detection rate when compared with the rate ofdetecting physical process attacks As with WD WDH did notdetect any attack launched against the Remote IO by pullingthe cables that connect it to the corresponding PLC WDHfared better in the detection of attacks against the Historian asit was directly accessing data on the Historian server

If the Historian itself or data that is input to the Historian iscompromised WDH takes the decision based on the input itreceives A clever and powerful attacker can attack the physical

TABLE VIIRESULTS FROM S3 2017

Target of Attack Noofattacks

WD WDH

Physical Process Attacks

State of motorised valves 2 100 100

State of water pumps 4 75 75

Pressure in UF 2 100 100

Water tank level 4 100 100

Chemical dosing 4 75 75

Sensor Data Attacks

Data in historian 3 0 100

Data in HMISCADA 3 67 67

Tampering PLC communi-cations

5 100 100

Tampering Remote IO 4 0 0

Total Attacks 31 6774 7741

process and modify values entering the Historian and thusdeceive WDH In general such a situation may arise in allbehavioral intrusion detection systems where the detector takesthe decision based on incorrect input data

Indeed data that appears to be ldquolegitimaterdquo could lead theWDH into believing that there is nothing wrong with thephysical process though there actually is However doingso requires the attacker to continuously manipulate a largenumber of state variables For example consider an attackwhere the attacker turns a pump say P101 ON when it shouldbe OFF and (continually) sends the state of the pump as OFFto the Historian and the corresponding PLC If the pump isOFF then the level of the source and destination tanks must berespectively decreasing and increasing at rates determined bythe pump characteristics Creating ldquolegitimate-lookingrdquo datathus requires an attacker to manipulate several state variablesas explained next (a) Two state variables that correspondto tank levels Two sensors (in SWaT) measure these statevariables (see Figure 2) Thus the attacker must have accessto these level sensors (b) If pump P101 is actually ON whilethe Historian receives its state as OFF then FIT201 must showno flow Thus the attacker will also need to manipulate FIT201to avoid detection This argument can be carried forward tosubsequent stages to show that many sensors will need to bemanipulated by an attacker to ldquohiderdquo a simple attack such asldquochange the state of a pumprdquo In summary yes incorrect dataat the Historian could prevent detection though doing so wouldbe a significant challenge for the attacker due primarily to thedistributed nature of the invariants

VIII DISCUSSION

A Challenges faced

We faced several challenges during S3 For example aftereach teamrsquos performance the operator was required to bringSWaT back to a predefined normal state It was necessary to

keep SWaT in a normal state before another team launched at-tacks Bringing SWaT to its normal state required (a) resettingnetwork communications to ensure that all the communicationchannels are operating as expected (b) the operator to ensurethat all physical processes in SWaT are stable with respectto the control logic (c) the operator to bring back SWaT tothe normal state of that particular device such as a pump or amotorized valve in the case of any physical or manual attacksby the previous team and (d) that the Historian and SCADAservers were reverted to their original state ie the state thatexisted prior to the launch of attacks

B Research questions

RQ1 How do attackers compromise the security of an ICS InSection VI we presented and categorized the attacks based onattacker profiles An attacker can launch physical attacks wheninside the plant such as manually operating a motorized valveor tampering with network cabling Several attacks launchedby the attack teams had not been launched by the authorsin their evaluation of WD [1] and WDH Thus S3 raisedour confidence in the effectiveness of the attack detectionmechanisms based on invariants derived from plant designs

RQ2 How effective is WD in detecting attacks launched byindependent attack teams As mentioned earlier while bothWD and WDH were found to detect a number of attacksthey did fail in several cases Given that the invariants derivedare intended to detect process anomalies it is clear that suchmechanisms must be used in conjunction with other attackdetection tools such as those in [24] [27] [21]

C Assessment by the authors and by independent teamsTable VIII lists the number of attacks launched by the

authors in an experimental evaluation performed prior to S3-2016 [1] Note that the WD detection rate observed by theauthors (89) was higher than the combined rate observedduring the two S3 events (6326) The difference in perfor-mance is due to different attack vectors used in the three setsof experiments WDH detection rate observed during S3 eventis (7741) which is much higher than the WD detection rateSome of these attack vectors are explained in Section VI andthe remaining may be found in [20]

TABLE VIIIPERFORMANCE OF WD AS EVALUATED BY THE AUTHORS AGAINST THOSE

BY PARTICIPANTS IN S3

Experiments by Attacks

Launched Detected (WD) Detected (WDH)

Authors 37 33 (89) NA

S3-2016 18 10 (555) NA

S3-2017 31 21 (677) 24 (774)

NA WDH did not exist at the time of experimentation by the author andduring S3-2016

The data in Table VIII is indicative of the value of orga-nizing S3 events Specifically in the case described in this

paper the two S3 events led to an increased confidence inthe effectiveness of the invariant-based approach in detectingcyber attacks The hackfests also led to the creation of newtypes of attack vectors that were not used earlier to assess theperformance of WD and WDH in detecting cyber attacks

D False alarms

The performance of any attack detection method ought tobe assessed using its detection accuracy ie how many of thelaunched attacks it detects as well as the rate at which falsealarms are raised During S3 each team attempted to launchseveral attacks The attacks listed in Tables III and IV are theones that were successful in realizing the stated attacker intentand were scored by the judges The remaining attacks werenot recorded and hence any alarm generated by such attackswas not considered Some of these unrecorded alarms couldbe false though no specific claims can be made about theirnature

Since S3-2017 the authors have observed no false alarmsfrom WD during normal operation of SWaT WDH has beenin operation since a few weeks prior to S3-2017 Againduring the normal operation of SWaT no alarm has beengenerated by WDH This observation should not be construedto imply that an invariant-based attack detection mechanismwill not generate any false alarmndash in fact it could Howeverif the invariants generated are complete in the sense that theyaccurately capture all aspects of process behavior and theirimplementation is correct and tuned properly the likelihoodof false alarms is low

Even though SWaT is a relatively new plant (2-years sinceits inauguration at the time of writing this paper) we doobserve intermittent failures in a few motorized valves Forexample sometimes MV101 in Stage 1 takes much longer toopen than expected by its controlling PLC1 The PLC itselfdetects such cases In such a case WD or WDH dependingon the time it takes for the valve to finally open will raisean alarm We do not consider this as a false positive simplybecause whether an anomalous behavior is due to a naturalcause or a cyber attack cannot be distinguished by WD orWDH While such distinction is important to make additionalresearch is needed to distinguish process anomalies due tocyber attacks and those arising due to natural componentfailures

E Benefits of S3

S3 exposed the organisers participants and researchers tohow an attacker might design and launch attacks on ICS Bene-fits of S3 include the following 1) An improved understandingof how an ICS operates and the consequent formulation ofnew research directions 2) Opportunity for participants fromindustry and academia to learn from the event and focus onthe limitations of their work 3) An aid to the ICS managementteam to observe the defense teams thus leading to possibleadoption of technology embedded in WD or WDH

F Placement of WD

The placement of WD is another question that ought tobe looked into carefully In this work WD is placed insidePLCs However an exceptionally large number of invariantsmay prevent adding code to the existing control code in a PLCThis may happen due to the computational load requirementson a PLC This aspect led us to create WDH that is placedon the plant network and gets its data from the Historian toevaluate the invariants

G Forensics

One advantage of the invariant-based approach for attackdetection appears while determining the area of impact ofan attack When a single invariant is violated it indicatesclearly the source of process anomaly For example an alertis generated if valve MV101 is closed when the water intank T101 is at or below the L level marker While this alertdoes not indicate how an attacker entered the system or ifthe valve or the level sensor is defective it does assist inlocalising the reason for the alert The analysis becomes abit more complex when multiple invariants raise alerts Thisaspect of an invariant-based detection mechanisms remains tobe analyzed in further detail

H Attacker capabilities

We do not have any validation of the professionalism of theS3 attack teams As mentioned earlier [20] [35] [36] attackteams were from a variety of backgrounds including fromthe industry and academia from Europe and Asia During S3-2017 one team consisting of four membersndashall from outsideof Singaporendash focuses on ethical hacking and cyber-warsinvolving critical infrastructure This team is part of a globalalliance The other teams consist of hackers interested inknowing how vulnerabilities in software can be exploitedand passes this information to others for improving systemssecurity Coverage of attacks launched by the attack teams andattacker profiles is discussed in Section IV and summarizedin Tables I II V and VII

I Attack trees

It is possible to use attack trees [37] [42] to model attackslaunched during the two hackfests reported in this paper Doingso would enable mapping each attack to a specific path inthe attack tree and reveal which attack paths in SWaT weretraversed Such modeling and analysis has not been attemptedin this work and is a possible subject for future research

IX RELATED WORK

S3 is a Capture-The-Flag [15] event on ICS TraditionalCTF events generally attract the attention of both industrialand academic teams and currently enjoy increasing popularityas indicated in [15] The number of such events is graduallyincreasing [13] [16] Such events aid in learning about secu-rity vulnerabilities how these could be exploited nature ofattacks and strength of the deployed [18] [33] [45] defensemechanisms To the best of our knowledge S3 is the first CTF

style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed

The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks

There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc

The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies

Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS

X CONCLUSION

There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense

While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the

actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure

It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously

The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants

The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]

It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question

ACKNOWLEDGMENTS

A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript

REFERENCES

[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016

[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016

[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016

[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016

[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016

[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016

[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017

[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012

[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017

[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and

S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011

[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml

[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010

[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015

[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security

under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler

Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016

[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017

[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016

[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016

[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010

[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for

safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016

[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical

systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013

[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111

[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016

[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013

[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx

[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007

[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016

[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3

[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event

[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008

[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016

[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011

[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf

[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment

of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007

[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016

[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016

[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003

[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014

[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014

[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011

BIOGRAPHY

Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems

Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research

of valve MV301 Under normal circumstances this valve isopened during the backwash process However the attackeropened it when there was no backwash Hence the attackdid not affect the physical process except in changing thevalve status No invariant was violated due to this attackbecause the backwash process ie Stage 6 is not includedin this case study The second single point attack (attack 17 inTable III) was performed on chemical dosing pump P203 whilethe other pump P204 was running Note that under normalcircumstances only one of these two pumps is supposed to berunning while the other remains as a backup Subsequently theattacker shut down pump P204 This attack was not detectedbecause there were no invariants that related to the chemicalproperties of water

Although the overall performance of WD was below 100it did detect all attacks within its scope except two (attacks 6and 17 in Table III) as mentioned earlier

B S3-2017 results

Table VI indicates that 21 out of 31 attacks were detectedby WD while 24 out of 31 attacks were detected by WDHConsidering only the attacks within its scope as mentionedin Section V-A WD detected 21 out of 28 attacks (75)Similarly WDH detected 24 out of 31 attacks (7741) withinits scope mentioned in Section V-B Three attacks on theHistorian are not in the scope of WD All attack targets relatedto RIODisplay (in Table II and in Table IV) are not detectedby both WD and WDH This is because registers inside aPLC save the previous values received from the sensors andthe PLC continues to execute the control code The invariantsalso use the same values stored in the PLC registers and hencedo not raise an alert

In general PLCs send to the Historian via the SCADAworkstation the data received from the sensors When a PLCdoes not have updated values during the attack period it isobvious that the Historian also receives the same stale valuesThis is the reason why WDH also did not detect attacksrelated to RIODisplay Note that the RIODisplay attacks werelaunched and remained active only for a few seconds Duringthis period the PLC did not update the current sensor valuescoming through the RIO If the same attack is performed for alonger duration the PLC would update the data received fromthe sensors Doing so would likely lead to WD and WDHdetecting the RIO attacks

Attacks launched on the Historian were detected by WDHbut not by WD This variance is due to the fact that data inthese attacks is manipulated at the Historian Thus invariantsin a PLC do not have access to the manipulated data andhence the invariants in WD do not raise any alert All attackstargeting a PLC are detected by WD and WDH

WD Detection of physical process attacks All attacks onvalves pressure sensor and level sensors were detected Threeout of four attacks on the chemical dosing process pumpswere detected An example of a detected attack is when theattackers took control of pump P301 (attack 20 in Table IV)

TABLE VIPERFORMANCE OF WD AND WDH

S3-2016 S3-2017

WD WD WDH

Detected 1 5 7 910 11 1213 14 18

3 4 7 9 1011 12 13 1516 18 19 2021 22 23 2628 29 30 31

2 3 4 7 9 1011 12 13 15 1617 18 19 20 2122 23 24 26 2829 30 31

Not detected 2 3 4 68 15 1617

1 2 5 6 8 1417 24 25 27

1 5 6 8 14 2527

through a Python script (Pycomm) to raise the pressure in theUF unit measured by sensor DPIT301 to a dangerous levelWD immediately raised an alarm This invariant ensured thatpump P301 must be OFF when the pressure at DPIT301 wasabove a threshold During the attack the invariant was violatedas the pump was not turned off while DPIT301 indicatedreadings that were above the threshold Consequently an alarmwas raised immediately In certain cases multiple alarmswere raised due to the violation of one or more invariantsFor example when level sensor LIT101 was compromisedthe invariants corresponding to this sensor were violated andraised alarms

WD Detection of sensor data attack WD detected attacks onHMISCADA and PLC values because these attacks directlycompromised the physical processes These attacks eithercompromised chemical dosing water tank levels or pumpstatus through hacking of the HMISCADA or PLC Hencethe robustness of WD in detecting unusual physical processbehavior was found effective in these attacks On the otherhand WD was unable to detect insider attacks that pulled outRIO cables This is because WD triggers an alarm only whenthe invariants are violated Under normal circumstance for aperiod of time a PLC continues to execute its control codeand any invariant code based on the last known state andorvalues Thus the invariants located inside the PLCs are unableto observe this anomalous behavior

WDH Detection of physical process attacks WDH detected14 out of 16 physical process attacks

WDH Detection of sensor data attacks WDH detected theattacks on HMISCADA and PLC values because these attacksdirectly compromised the physical processes albeit with aslightly lower detection rate when compared with the rate ofdetecting physical process attacks As with WD WDH did notdetect any attack launched against the Remote IO by pullingthe cables that connect it to the corresponding PLC WDHfared better in the detection of attacks against the Historian asit was directly accessing data on the Historian server

If the Historian itself or data that is input to the Historian iscompromised WDH takes the decision based on the input itreceives A clever and powerful attacker can attack the physical

TABLE VIIRESULTS FROM S3 2017

Target of Attack Noofattacks

WD WDH

Physical Process Attacks

State of motorised valves 2 100 100

State of water pumps 4 75 75

Pressure in UF 2 100 100

Water tank level 4 100 100

Chemical dosing 4 75 75

Sensor Data Attacks

Data in historian 3 0 100

Data in HMISCADA 3 67 67

Tampering PLC communi-cations

5 100 100

Tampering Remote IO 4 0 0

Total Attacks 31 6774 7741

process and modify values entering the Historian and thusdeceive WDH In general such a situation may arise in allbehavioral intrusion detection systems where the detector takesthe decision based on incorrect input data

Indeed data that appears to be ldquolegitimaterdquo could lead theWDH into believing that there is nothing wrong with thephysical process though there actually is However doingso requires the attacker to continuously manipulate a largenumber of state variables For example consider an attackwhere the attacker turns a pump say P101 ON when it shouldbe OFF and (continually) sends the state of the pump as OFFto the Historian and the corresponding PLC If the pump isOFF then the level of the source and destination tanks must berespectively decreasing and increasing at rates determined bythe pump characteristics Creating ldquolegitimate-lookingrdquo datathus requires an attacker to manipulate several state variablesas explained next (a) Two state variables that correspondto tank levels Two sensors (in SWaT) measure these statevariables (see Figure 2) Thus the attacker must have accessto these level sensors (b) If pump P101 is actually ON whilethe Historian receives its state as OFF then FIT201 must showno flow Thus the attacker will also need to manipulate FIT201to avoid detection This argument can be carried forward tosubsequent stages to show that many sensors will need to bemanipulated by an attacker to ldquohiderdquo a simple attack such asldquochange the state of a pumprdquo In summary yes incorrect dataat the Historian could prevent detection though doing so wouldbe a significant challenge for the attacker due primarily to thedistributed nature of the invariants

VIII DISCUSSION

A Challenges faced

We faced several challenges during S3 For example aftereach teamrsquos performance the operator was required to bringSWaT back to a predefined normal state It was necessary to

keep SWaT in a normal state before another team launched at-tacks Bringing SWaT to its normal state required (a) resettingnetwork communications to ensure that all the communicationchannels are operating as expected (b) the operator to ensurethat all physical processes in SWaT are stable with respectto the control logic (c) the operator to bring back SWaT tothe normal state of that particular device such as a pump or amotorized valve in the case of any physical or manual attacksby the previous team and (d) that the Historian and SCADAservers were reverted to their original state ie the state thatexisted prior to the launch of attacks

B Research questions

RQ1 How do attackers compromise the security of an ICS InSection VI we presented and categorized the attacks based onattacker profiles An attacker can launch physical attacks wheninside the plant such as manually operating a motorized valveor tampering with network cabling Several attacks launchedby the attack teams had not been launched by the authorsin their evaluation of WD [1] and WDH Thus S3 raisedour confidence in the effectiveness of the attack detectionmechanisms based on invariants derived from plant designs

RQ2 How effective is WD in detecting attacks launched byindependent attack teams As mentioned earlier while bothWD and WDH were found to detect a number of attacksthey did fail in several cases Given that the invariants derivedare intended to detect process anomalies it is clear that suchmechanisms must be used in conjunction with other attackdetection tools such as those in [24] [27] [21]

C Assessment by the authors and by independent teamsTable VIII lists the number of attacks launched by the

authors in an experimental evaluation performed prior to S3-2016 [1] Note that the WD detection rate observed by theauthors (89) was higher than the combined rate observedduring the two S3 events (6326) The difference in perfor-mance is due to different attack vectors used in the three setsof experiments WDH detection rate observed during S3 eventis (7741) which is much higher than the WD detection rateSome of these attack vectors are explained in Section VI andthe remaining may be found in [20]

TABLE VIIIPERFORMANCE OF WD AS EVALUATED BY THE AUTHORS AGAINST THOSE

BY PARTICIPANTS IN S3

Experiments by Attacks

Launched Detected (WD) Detected (WDH)

Authors 37 33 (89) NA

S3-2016 18 10 (555) NA

S3-2017 31 21 (677) 24 (774)

NA WDH did not exist at the time of experimentation by the author andduring S3-2016

The data in Table VIII is indicative of the value of orga-nizing S3 events Specifically in the case described in this

paper the two S3 events led to an increased confidence inthe effectiveness of the invariant-based approach in detectingcyber attacks The hackfests also led to the creation of newtypes of attack vectors that were not used earlier to assess theperformance of WD and WDH in detecting cyber attacks

D False alarms

The performance of any attack detection method ought tobe assessed using its detection accuracy ie how many of thelaunched attacks it detects as well as the rate at which falsealarms are raised During S3 each team attempted to launchseveral attacks The attacks listed in Tables III and IV are theones that were successful in realizing the stated attacker intentand were scored by the judges The remaining attacks werenot recorded and hence any alarm generated by such attackswas not considered Some of these unrecorded alarms couldbe false though no specific claims can be made about theirnature

Since S3-2017 the authors have observed no false alarmsfrom WD during normal operation of SWaT WDH has beenin operation since a few weeks prior to S3-2017 Againduring the normal operation of SWaT no alarm has beengenerated by WDH This observation should not be construedto imply that an invariant-based attack detection mechanismwill not generate any false alarmndash in fact it could Howeverif the invariants generated are complete in the sense that theyaccurately capture all aspects of process behavior and theirimplementation is correct and tuned properly the likelihoodof false alarms is low

Even though SWaT is a relatively new plant (2-years sinceits inauguration at the time of writing this paper) we doobserve intermittent failures in a few motorized valves Forexample sometimes MV101 in Stage 1 takes much longer toopen than expected by its controlling PLC1 The PLC itselfdetects such cases In such a case WD or WDH dependingon the time it takes for the valve to finally open will raisean alarm We do not consider this as a false positive simplybecause whether an anomalous behavior is due to a naturalcause or a cyber attack cannot be distinguished by WD orWDH While such distinction is important to make additionalresearch is needed to distinguish process anomalies due tocyber attacks and those arising due to natural componentfailures

E Benefits of S3

S3 exposed the organisers participants and researchers tohow an attacker might design and launch attacks on ICS Bene-fits of S3 include the following 1) An improved understandingof how an ICS operates and the consequent formulation ofnew research directions 2) Opportunity for participants fromindustry and academia to learn from the event and focus onthe limitations of their work 3) An aid to the ICS managementteam to observe the defense teams thus leading to possibleadoption of technology embedded in WD or WDH

F Placement of WD

The placement of WD is another question that ought tobe looked into carefully In this work WD is placed insidePLCs However an exceptionally large number of invariantsmay prevent adding code to the existing control code in a PLCThis may happen due to the computational load requirementson a PLC This aspect led us to create WDH that is placedon the plant network and gets its data from the Historian toevaluate the invariants

G Forensics

One advantage of the invariant-based approach for attackdetection appears while determining the area of impact ofan attack When a single invariant is violated it indicatesclearly the source of process anomaly For example an alertis generated if valve MV101 is closed when the water intank T101 is at or below the L level marker While this alertdoes not indicate how an attacker entered the system or ifthe valve or the level sensor is defective it does assist inlocalising the reason for the alert The analysis becomes abit more complex when multiple invariants raise alerts Thisaspect of an invariant-based detection mechanisms remains tobe analyzed in further detail

H Attacker capabilities

We do not have any validation of the professionalism of theS3 attack teams As mentioned earlier [20] [35] [36] attackteams were from a variety of backgrounds including fromthe industry and academia from Europe and Asia During S3-2017 one team consisting of four membersndashall from outsideof Singaporendash focuses on ethical hacking and cyber-warsinvolving critical infrastructure This team is part of a globalalliance The other teams consist of hackers interested inknowing how vulnerabilities in software can be exploitedand passes this information to others for improving systemssecurity Coverage of attacks launched by the attack teams andattacker profiles is discussed in Section IV and summarizedin Tables I II V and VII

I Attack trees

It is possible to use attack trees [37] [42] to model attackslaunched during the two hackfests reported in this paper Doingso would enable mapping each attack to a specific path inthe attack tree and reveal which attack paths in SWaT weretraversed Such modeling and analysis has not been attemptedin this work and is a possible subject for future research

IX RELATED WORK

S3 is a Capture-The-Flag [15] event on ICS TraditionalCTF events generally attract the attention of both industrialand academic teams and currently enjoy increasing popularityas indicated in [15] The number of such events is graduallyincreasing [13] [16] Such events aid in learning about secu-rity vulnerabilities how these could be exploited nature ofattacks and strength of the deployed [18] [33] [45] defensemechanisms To the best of our knowledge S3 is the first CTF

style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed

The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks

There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc

The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies

Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS

X CONCLUSION

There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense

While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the

actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure

It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously

The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants

The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]

It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question

ACKNOWLEDGMENTS

A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript

REFERENCES

[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016

[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016

[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016

[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016

[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016

[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016

[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017

[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012

[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017

[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and

S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011

[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml

[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010

[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015

[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security

under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler

Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016

[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017

[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016

[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016

[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010

[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for

safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016

[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical

systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013

[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111

[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016

[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013

[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx

[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007

[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016

[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3

[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event

[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008

[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016

[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011

[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf

[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment

of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007

[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016

[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016

[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003

[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014

[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014

[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011

BIOGRAPHY

Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems

Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research

TABLE VIIRESULTS FROM S3 2017

Target of Attack Noofattacks

WD WDH

Physical Process Attacks

State of motorised valves 2 100 100

State of water pumps 4 75 75

Pressure in UF 2 100 100

Water tank level 4 100 100

Chemical dosing 4 75 75

Sensor Data Attacks

Data in historian 3 0 100

Data in HMISCADA 3 67 67

Tampering PLC communi-cations

5 100 100

Tampering Remote IO 4 0 0

Total Attacks 31 6774 7741

process and modify values entering the Historian and thusdeceive WDH In general such a situation may arise in allbehavioral intrusion detection systems where the detector takesthe decision based on incorrect input data

Indeed data that appears to be ldquolegitimaterdquo could lead theWDH into believing that there is nothing wrong with thephysical process though there actually is However doingso requires the attacker to continuously manipulate a largenumber of state variables For example consider an attackwhere the attacker turns a pump say P101 ON when it shouldbe OFF and (continually) sends the state of the pump as OFFto the Historian and the corresponding PLC If the pump isOFF then the level of the source and destination tanks must berespectively decreasing and increasing at rates determined bythe pump characteristics Creating ldquolegitimate-lookingrdquo datathus requires an attacker to manipulate several state variablesas explained next (a) Two state variables that correspondto tank levels Two sensors (in SWaT) measure these statevariables (see Figure 2) Thus the attacker must have accessto these level sensors (b) If pump P101 is actually ON whilethe Historian receives its state as OFF then FIT201 must showno flow Thus the attacker will also need to manipulate FIT201to avoid detection This argument can be carried forward tosubsequent stages to show that many sensors will need to bemanipulated by an attacker to ldquohiderdquo a simple attack such asldquochange the state of a pumprdquo In summary yes incorrect dataat the Historian could prevent detection though doing so wouldbe a significant challenge for the attacker due primarily to thedistributed nature of the invariants

VIII DISCUSSION

A Challenges faced

We faced several challenges during S3 For example aftereach teamrsquos performance the operator was required to bringSWaT back to a predefined normal state It was necessary to

keep SWaT in a normal state before another team launched at-tacks Bringing SWaT to its normal state required (a) resettingnetwork communications to ensure that all the communicationchannels are operating as expected (b) the operator to ensurethat all physical processes in SWaT are stable with respectto the control logic (c) the operator to bring back SWaT tothe normal state of that particular device such as a pump or amotorized valve in the case of any physical or manual attacksby the previous team and (d) that the Historian and SCADAservers were reverted to their original state ie the state thatexisted prior to the launch of attacks

B Research questions

RQ1 How do attackers compromise the security of an ICS InSection VI we presented and categorized the attacks based onattacker profiles An attacker can launch physical attacks wheninside the plant such as manually operating a motorized valveor tampering with network cabling Several attacks launchedby the attack teams had not been launched by the authorsin their evaluation of WD [1] and WDH Thus S3 raisedour confidence in the effectiveness of the attack detectionmechanisms based on invariants derived from plant designs

RQ2 How effective is WD in detecting attacks launched byindependent attack teams As mentioned earlier while bothWD and WDH were found to detect a number of attacksthey did fail in several cases Given that the invariants derivedare intended to detect process anomalies it is clear that suchmechanisms must be used in conjunction with other attackdetection tools such as those in [24] [27] [21]

C Assessment by the authors and by independent teamsTable VIII lists the number of attacks launched by the

authors in an experimental evaluation performed prior to S3-2016 [1] Note that the WD detection rate observed by theauthors (89) was higher than the combined rate observedduring the two S3 events (6326) The difference in perfor-mance is due to different attack vectors used in the three setsof experiments WDH detection rate observed during S3 eventis (7741) which is much higher than the WD detection rateSome of these attack vectors are explained in Section VI andthe remaining may be found in [20]

TABLE VIIIPERFORMANCE OF WD AS EVALUATED BY THE AUTHORS AGAINST THOSE

BY PARTICIPANTS IN S3

Experiments by Attacks

Launched Detected (WD) Detected (WDH)

Authors 37 33 (89) NA

S3-2016 18 10 (555) NA

S3-2017 31 21 (677) 24 (774)

NA WDH did not exist at the time of experimentation by the author andduring S3-2016

The data in Table VIII is indicative of the value of orga-nizing S3 events Specifically in the case described in this

paper the two S3 events led to an increased confidence inthe effectiveness of the invariant-based approach in detectingcyber attacks The hackfests also led to the creation of newtypes of attack vectors that were not used earlier to assess theperformance of WD and WDH in detecting cyber attacks

D False alarms

The performance of any attack detection method ought tobe assessed using its detection accuracy ie how many of thelaunched attacks it detects as well as the rate at which falsealarms are raised During S3 each team attempted to launchseveral attacks The attacks listed in Tables III and IV are theones that were successful in realizing the stated attacker intentand were scored by the judges The remaining attacks werenot recorded and hence any alarm generated by such attackswas not considered Some of these unrecorded alarms couldbe false though no specific claims can be made about theirnature

Since S3-2017 the authors have observed no false alarmsfrom WD during normal operation of SWaT WDH has beenin operation since a few weeks prior to S3-2017 Againduring the normal operation of SWaT no alarm has beengenerated by WDH This observation should not be construedto imply that an invariant-based attack detection mechanismwill not generate any false alarmndash in fact it could Howeverif the invariants generated are complete in the sense that theyaccurately capture all aspects of process behavior and theirimplementation is correct and tuned properly the likelihoodof false alarms is low

Even though SWaT is a relatively new plant (2-years sinceits inauguration at the time of writing this paper) we doobserve intermittent failures in a few motorized valves Forexample sometimes MV101 in Stage 1 takes much longer toopen than expected by its controlling PLC1 The PLC itselfdetects such cases In such a case WD or WDH dependingon the time it takes for the valve to finally open will raisean alarm We do not consider this as a false positive simplybecause whether an anomalous behavior is due to a naturalcause or a cyber attack cannot be distinguished by WD orWDH While such distinction is important to make additionalresearch is needed to distinguish process anomalies due tocyber attacks and those arising due to natural componentfailures

E Benefits of S3

S3 exposed the organisers participants and researchers tohow an attacker might design and launch attacks on ICS Bene-fits of S3 include the following 1) An improved understandingof how an ICS operates and the consequent formulation ofnew research directions 2) Opportunity for participants fromindustry and academia to learn from the event and focus onthe limitations of their work 3) An aid to the ICS managementteam to observe the defense teams thus leading to possibleadoption of technology embedded in WD or WDH

F Placement of WD

The placement of WD is another question that ought tobe looked into carefully In this work WD is placed insidePLCs However an exceptionally large number of invariantsmay prevent adding code to the existing control code in a PLCThis may happen due to the computational load requirementson a PLC This aspect led us to create WDH that is placedon the plant network and gets its data from the Historian toevaluate the invariants

G Forensics

One advantage of the invariant-based approach for attackdetection appears while determining the area of impact ofan attack When a single invariant is violated it indicatesclearly the source of process anomaly For example an alertis generated if valve MV101 is closed when the water intank T101 is at or below the L level marker While this alertdoes not indicate how an attacker entered the system or ifthe valve or the level sensor is defective it does assist inlocalising the reason for the alert The analysis becomes abit more complex when multiple invariants raise alerts Thisaspect of an invariant-based detection mechanisms remains tobe analyzed in further detail

H Attacker capabilities

We do not have any validation of the professionalism of theS3 attack teams As mentioned earlier [20] [35] [36] attackteams were from a variety of backgrounds including fromthe industry and academia from Europe and Asia During S3-2017 one team consisting of four membersndashall from outsideof Singaporendash focuses on ethical hacking and cyber-warsinvolving critical infrastructure This team is part of a globalalliance The other teams consist of hackers interested inknowing how vulnerabilities in software can be exploitedand passes this information to others for improving systemssecurity Coverage of attacks launched by the attack teams andattacker profiles is discussed in Section IV and summarizedin Tables I II V and VII

I Attack trees

It is possible to use attack trees [37] [42] to model attackslaunched during the two hackfests reported in this paper Doingso would enable mapping each attack to a specific path inthe attack tree and reveal which attack paths in SWaT weretraversed Such modeling and analysis has not been attemptedin this work and is a possible subject for future research

IX RELATED WORK

S3 is a Capture-The-Flag [15] event on ICS TraditionalCTF events generally attract the attention of both industrialand academic teams and currently enjoy increasing popularityas indicated in [15] The number of such events is graduallyincreasing [13] [16] Such events aid in learning about secu-rity vulnerabilities how these could be exploited nature ofattacks and strength of the deployed [18] [33] [45] defensemechanisms To the best of our knowledge S3 is the first CTF

style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed

The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks

There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc

The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies

Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS

X CONCLUSION

There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense

While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the

actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure

It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously

The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants

The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]

It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question

ACKNOWLEDGMENTS

A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript

REFERENCES

[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016

[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016

[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016

[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016

[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016

[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016

[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017

[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012

[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017

[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and

S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011

[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml

[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010

[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015

[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security

under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler

Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016

[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017

[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016

[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016

[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010

[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for

safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016

[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical

systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013

[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111

[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016

[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013

[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx

[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007

[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016

[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3

[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event

[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008

[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016

[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011

[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf

[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment

of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007

[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016

[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016

[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003

[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014

[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014

[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011

BIOGRAPHY

Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems

Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research

paper the two S3 events led to an increased confidence inthe effectiveness of the invariant-based approach in detectingcyber attacks The hackfests also led to the creation of newtypes of attack vectors that were not used earlier to assess theperformance of WD and WDH in detecting cyber attacks

D False alarms

The performance of any attack detection method ought tobe assessed using its detection accuracy ie how many of thelaunched attacks it detects as well as the rate at which falsealarms are raised During S3 each team attempted to launchseveral attacks The attacks listed in Tables III and IV are theones that were successful in realizing the stated attacker intentand were scored by the judges The remaining attacks werenot recorded and hence any alarm generated by such attackswas not considered Some of these unrecorded alarms couldbe false though no specific claims can be made about theirnature

Since S3-2017 the authors have observed no false alarmsfrom WD during normal operation of SWaT WDH has beenin operation since a few weeks prior to S3-2017 Againduring the normal operation of SWaT no alarm has beengenerated by WDH This observation should not be construedto imply that an invariant-based attack detection mechanismwill not generate any false alarmndash in fact it could Howeverif the invariants generated are complete in the sense that theyaccurately capture all aspects of process behavior and theirimplementation is correct and tuned properly the likelihoodof false alarms is low

Even though SWaT is a relatively new plant (2-years sinceits inauguration at the time of writing this paper) we doobserve intermittent failures in a few motorized valves Forexample sometimes MV101 in Stage 1 takes much longer toopen than expected by its controlling PLC1 The PLC itselfdetects such cases In such a case WD or WDH dependingon the time it takes for the valve to finally open will raisean alarm We do not consider this as a false positive simplybecause whether an anomalous behavior is due to a naturalcause or a cyber attack cannot be distinguished by WD orWDH While such distinction is important to make additionalresearch is needed to distinguish process anomalies due tocyber attacks and those arising due to natural componentfailures

E Benefits of S3

S3 exposed the organisers participants and researchers tohow an attacker might design and launch attacks on ICS Bene-fits of S3 include the following 1) An improved understandingof how an ICS operates and the consequent formulation ofnew research directions 2) Opportunity for participants fromindustry and academia to learn from the event and focus onthe limitations of their work 3) An aid to the ICS managementteam to observe the defense teams thus leading to possibleadoption of technology embedded in WD or WDH

F Placement of WD

The placement of WD is another question that ought tobe looked into carefully In this work WD is placed insidePLCs However an exceptionally large number of invariantsmay prevent adding code to the existing control code in a PLCThis may happen due to the computational load requirementson a PLC This aspect led us to create WDH that is placedon the plant network and gets its data from the Historian toevaluate the invariants

G Forensics

One advantage of the invariant-based approach for attackdetection appears while determining the area of impact ofan attack When a single invariant is violated it indicatesclearly the source of process anomaly For example an alertis generated if valve MV101 is closed when the water intank T101 is at or below the L level marker While this alertdoes not indicate how an attacker entered the system or ifthe valve or the level sensor is defective it does assist inlocalising the reason for the alert The analysis becomes abit more complex when multiple invariants raise alerts Thisaspect of an invariant-based detection mechanisms remains tobe analyzed in further detail

H Attacker capabilities

We do not have any validation of the professionalism of theS3 attack teams As mentioned earlier [20] [35] [36] attackteams were from a variety of backgrounds including fromthe industry and academia from Europe and Asia During S3-2017 one team consisting of four membersndashall from outsideof Singaporendash focuses on ethical hacking and cyber-warsinvolving critical infrastructure This team is part of a globalalliance The other teams consist of hackers interested inknowing how vulnerabilities in software can be exploitedand passes this information to others for improving systemssecurity Coverage of attacks launched by the attack teams andattacker profiles is discussed in Section IV and summarizedin Tables I II V and VII

I Attack trees

It is possible to use attack trees [37] [42] to model attackslaunched during the two hackfests reported in this paper Doingso would enable mapping each attack to a specific path inthe attack tree and reveal which attack paths in SWaT weretraversed Such modeling and analysis has not been attemptedin this work and is a possible subject for future research

IX RELATED WORK

S3 is a Capture-The-Flag [15] event on ICS TraditionalCTF events generally attract the attention of both industrialand academic teams and currently enjoy increasing popularityas indicated in [15] The number of such events is graduallyincreasing [13] [16] Such events aid in learning about secu-rity vulnerabilities how these could be exploited nature ofattacks and strength of the deployed [18] [33] [45] defensemechanisms To the best of our knowledge S3 is the first CTF

style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed

The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks

There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc

The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies

Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS

X CONCLUSION

There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense

While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the

actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure

It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously

The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants

The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]

It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question

ACKNOWLEDGMENTS

A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript

REFERENCES

[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016

[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016

[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016

[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016

[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016

[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016

[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017

[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012

[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017

[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and

S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011

[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml

[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010

[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015

[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security

under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler

Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016

[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017

[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016

[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016

[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010

[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for

safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016

[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical

systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013

[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111

[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016

[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013

[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx

[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007

[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016

[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3

[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event

[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008

[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016

[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011

[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf

[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment

of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007

[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016

[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016

[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003

[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014

[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014

[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011

BIOGRAPHY

Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems

Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research

style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed

The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks

There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc

The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies

Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS

X CONCLUSION

There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense

While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the

actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure

It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously

The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants

The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]

It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question

ACKNOWLEDGMENTS

A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript

REFERENCES

[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016

[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016

[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016

[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016

[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016

[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016

[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017

[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012

[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017

[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and

S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011

[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml

[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010

[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015

[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security

under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler

Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016

[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017

[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016

[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016

[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010

[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for

safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016

[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical

systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013

[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111

[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016

[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013

[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx

[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007

[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016

[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3

[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event

[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008

[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016

[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011

[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf

[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment

of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007

[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016

[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016

[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003

[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014

[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014

[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011

BIOGRAPHY

Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems

Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research

[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017

[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012

[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017

[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and

S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011

[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml

[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010

[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015

[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security

under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler

Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016

[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017

[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016

[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016

[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010

[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for

safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016

[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical

systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013

[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111

[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016

[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013

[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx

[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007

[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016

[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3

[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event

[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008

[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016

[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011

[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf

[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment

of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007

[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016

[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016

[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003

[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014

[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014

[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011

BIOGRAPHY

Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems

Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research