® Copyright© Intel Corporation 2000-2004 Industrial Applications for Sensor Networks Condition based monitoring pilot project Lama Nachman Researcher Intel

®

Copyright© Intel Corporation 2000-2004

Industrial Applications Industrial Applications for Sensor Networksfor Sensor NetworksCondition based monitoring pilot Condition based monitoring pilot projectproject

Industrial Applications Industrial Applications for Sensor Networksfor Sensor NetworksCondition based monitoring pilot Condition based monitoring pilot projectproject

Lama NachmanLama Nachman

ResearcherResearcher

Intel Research and System Intel Research and System Technology LabTechnology Lab

Lakshman KrishnamurthyLakshman Krishnamurthy

Hans MulderHans Mulder

Ralph KlingRalph Kling

Mark YarvisMark Yarvis

Jasmeet ChhabraJasmeet Chhabra

Carl DellarCarl Dellar

• 2 •Intel Research


AgendaAgenda Introduction & Problem Statement Introduction & Problem Statement

Equipment Health monitoring Pilot in Intel FABEquipment Health monitoring Pilot in Intel FAB Application requirementsApplication requirements

Current Implementation (Phase 2)Current Implementation (Phase 2) MICA & iMote clustersMICA & iMote clusters

Reliability protocolReliability protocol

Network ConfigurationNetwork Configuration

Power Saving ProtocolPower Saving Protocol

Status & Next stepsStatus & Next steps

Key learningsKey learnings



Use vibration signatures to Use vibration signatures to identify problems with equipment identify problems with equipment Avoid failureAvoid failure

~5000 Sensor points in each fab~5000 Sensor points in each fab 4 years of archived data4 years of archived data Done by sneaker net todayDone by sneaker net today Move to wireless sensor networkMove to wireless sensor network

Demonstrate a commercially feasible ROI for sensor and Demonstrate a commercially feasible ROI for sensor and mesh network deploymentsmesh network deployments

Fab Pre-emptive Maintenance Application at Intel



Problem StatementProblem Statement Equipment failures in live production fabs is Equipment failures in live production fabs is

extremely costly ($Millions)extremely costly ($Millions) Shutdown results in opportunity lossShutdown results in opportunity loss

Cost of evacuation and requalifying all the toolsCost of evacuation and requalifying all the tools

Possible loss of wafer lots in the pipelinePossible loss of wafer lots in the pipeline

Need to predict equipment failures early Need to predict equipment failures early enough and perform preemptive maintenance enough and perform preemptive maintenance during pre-scheduled down-timeduring pre-scheduled down-time

Monitor equipment health using vibration Monitor equipment health using vibration signaturessignatures



Case study (RA FAB)Case study (RA FAB) ~5000 sensing points already instrumented~5000 sensing points already instrumented

40% permanent sensors, 60% portable sensors40% permanent sensors, 60% portable sensors

Vibration and RPM sensors (Wilcoxon & Honeywell)Vibration and RPM sensors (Wilcoxon & Honeywell)

Manual data collection using handheld devicesManual data collection using handheld devices Time domain data is collected, spectrum and magnitude Time domain data is collected, spectrum and magnitude

plots are generatedplots are generated

Data is downloaded to Rockwell Enshare softwareData is downloaded to Rockwell Enshare software Sensors are manually configured in DBSensors are manually configured in DB

Type, location, direction, collection frequency, etcType, location, direction, collection frequency, etc

Alarms are generated, further manual collection is Alarms are generated, further manual collection is performed on specific sensorsperformed on specific sensors



Case Study (RA FAB)Case Study (RA FAB) Prevention estimatesPrevention estimates

Once per month -> catch (80-85)%Once per month -> catch (80-85)%

Once per week, and selective daily collections -> catch Once per week, and selective daily collections -> catch ~97%~97%

Manual collection method is currently usedManual collection method is currently used Target is once per monthTarget is once per month

Headcount cost : ~$500,000 in one FABHeadcount cost : ~$500,000 in one FAB

Rockwell based solution (EnWatch)Rockwell based solution (EnWatch) Ethernet based on-line system (~$5000)Ethernet based on-line system (~$5000)

16 channels, data collection and analysis16 channels, data collection and analysis

Controlled by EnShare backendControlled by EnShare backend



Application RequirementsApplication Requirements Interface to Wilcoxon vibration sensors and Honeywell RPM Interface to Wilcoxon vibration sensors and Honeywell RPM

sensorssensors 0.5 Hz – 5KHz range0.5 Hz – 5KHz range 3000 Samples, 16 bits each3000 Samples, 16 bits each

Collect once per week (optional selective collection)Collect once per week (optional selective collection) Battery life Battery life

No access to power or Ethernet at sensing locationsNo access to power or Ethernet at sensing locations 6 months @ 1 collection per month6 months @ 1 collection per month 4 months @ 1 collection per week4 months @ 1 collection per week

ReliabilityReliability MTBF : 6 monthsMTBF : 6 months Identify bad data (especially false good data)Identify bad data (especially false good data)

Interface to Rockwell EnShare backendInterface to Rockwell EnShare backend Automatic network configuration and maintenanceAutomatic network configuration and maintenance



Pilot Network ArchitecturePilot Network ArchitectureIntranet

802.11 Mesh

Fab Equipment

Mote + Vibration Sensors

Ad Hoc Mote Network

Intranet isolation

Root Node

Cluster Heads



Solution componentsSolution components Ad hoc Mote networkAd hoc Mote network

MICA based clustersMICA based clusters Imote based clustersImote based clusters

End to End Reliable datagram transport protocol End to End Reliable datagram transport protocol (sensor node -> Root Node)(sensor node -> Root Node)

802.11 overlay mesh network using stargates802.11 overlay mesh network using stargates Cluster head manages data collection and Cluster head manages data collection and

sleep/wake schedulesleep/wake schedule Root Node collects the raw data, stores in EnShare Root Node collects the raw data, stores in EnShare

format and sends it to serverformat and sends it to server EnShare data base imports the raw dataEnShare data base imports the raw data



Reliability ProtocolReliability Protocol Runs on Mica motes, iMotes and StargatesRuns on Mica motes, iMotes and Stargates TinyOS implementationTinyOS implementation

Provides VarSend, VarRecv interfaces to app layerProvides VarSend, VarRecv interfaces to app layer Uses Generic Packet interface to abstract network layerUses Generic Packet interface to abstract network layer

Sliding window protocolSliding window protocol Connection parameter negotiation (fragment size, Connection parameter negotiation (fragment size,

window size, timeout info)window size, timeout info) Receiver sends an ACK bitmap within windowReceiver sends an ACK bitmap within window Sender retransmits NACK’d fragmentsSender retransmits NACK’d fragments 3 phases3 phases

Connection setup (light weight, 2 packets)Connection setup (light weight, 2 packets) Data exchange (data and NACK packets)Data exchange (data and NACK packets) Final ACK (2 packets)Final ACK (2 packets)



Mote Cluster ImplementationMote Cluster

Implementation



Data Collection / Power SavingData Collection / Power Saving Cluster head sends a command to each Cluster head sends a command to each

sensor node to start data collectionsensor node to start data collection Sensor node initiates reliable transport Sensor node initiates reliable transport

protocol with RootNode for each connected protocol with RootNode for each connected sensorsensor

Sensor node informs cluster head when data Sensor node informs cluster head when data transfer is completetransfer is complete

Cluster repeats the process for each sensor Cluster repeats the process for each sensor nodenode

When all sensors have been collected, the When all sensors have been collected, the complete cluster is put to sleep until next complete cluster is put to sleep until next collectioncollection



iMote Cluster DetailsiMote Cluster Details



Intel Mote: an enhanced wireless network research platformIntel Mote: an enhanced wireless network research platform

Hardware featuresHardware features High platform integration level (core, High platform integration level (core,

radio, memory…)radio, memory…)

Low power operationLow power operation

Small physical sizeSmall physical size

Modular HW/SW designModular HW/SW design

Low cost and volume production Low cost and volume production potentialpotential

Intel Mote is a modular, Intel Mote is a modular, stackable designstackable design Main board (ARM core, SRAM, Main board (ARM core, SRAM,

FLASH, BT radio)FLASH, BT radio)

Power supply board (battery, Power supply board (battery, AC, solar, …)AC, solar, …)

Sensor board(s)Sensor board(s)

Other boards (alternate radio, Other boards (alternate radio, debug, actuator, …)debug, actuator, …)

Power board

Main board

Sensor board

Backbone interconnect

TinyOS applications

TinyOS base components

Network layer (multihop)

Intel Mote layer

Firmware (BT-LLS)

Hardware



Network ConfigurationNetwork Configuration Automatic scatternet formation algorithmAutomatic scatternet formation algorithm

Forms a tree structure Forms a tree structure Clusterhead is the root of the tree (Master Role)Clusterhead is the root of the tree (Master Role) Intermediate nodes have dual Master/Slave rolesIntermediate nodes have dual Master/Slave roles Leaf nodes are slave only nodesLeaf nodes are slave only nodes

Free nodes alternate between BT Inquiry & scan Free nodes alternate between BT Inquiry & scan modes to discover other nodesmodes to discover other nodes

Free nodes can join at different levels in the tree, Free nodes can join at different levels in the tree, depending on which node they connect todepending on which node they connect to

Connected nodes only scan to eliminate the Connected nodes only scan to eliminate the possibility of creating loopspossibility of creating loops

Simple routing algorithmSimple routing algorithm



Power Saving ProtocolPower Saving Protocol Leverage low power modes in BluetoothLeverage low power modes in Bluetooth Cluster head broadcasts a “network sleep” message Cluster head broadcasts a “network sleep” message

down the tree.down the tree. Once the message reaches a leaf node, a response Once the message reaches a leaf node, a response

is sent up the treeis sent up the tree When a master hears a response from all its slaves, When a master hears a response from all its slaves,

it will put all the links on hold, and propagate the it will put all the links on hold, and propagate the response up the treeresponse up the tree

Messages can still flow through the network in Messages can still flow through the network in between hold intervals (20 second response time between hold intervals (20 second response time per level in the tree)per level in the tree)

The cluster will broadcast a “network wake up” The cluster will broadcast a “network wake up” message down the treemessage down the tree



Network ObservationsNetwork Observations 1 minute to form a cluster of 16 nodes1 minute to form a cluster of 16 nodes

BT links are very stable once establishedBT links are very stable once established

Network formation overhead is amortized Network formation overhead is amortized over long connection timeover long connection time

BT link layer reliability is very effective, BT link layer reliability is very effective, hence reducing the end to end NACKshence reducing the end to end NACKs

Need to optimize the scatternet formation Need to optimize the scatternet formation algorithm to select connections based on link algorithm to select connections based on link quality, and reducing hop countquality, and reducing hop count



iMote Cluster integrationiMote Cluster integration iMote clusteriMote cluster

Simple routing algorithmSimple routing algorithm iMote MHOP header (src, dest, channel)iMote MHOP header (src, dest, channel) TOS Message is not usedTOS Message is not used

MICA cluster & RootNodeMICA cluster & RootNode DSDV & flood protocolsDSDV & flood protocols

iMote cluster head translates between domainsiMote cluster head translates between domains Route update messages from rootnode intercepted to get Route update messages from rootnode intercepted to get

the RootNode IDthe RootNode ID Reliability protocol hdr/data is repackagedReliability protocol hdr/data is repackaged

Sensor -> RooNode (iMote packet -> DSDV packet)Sensor -> RooNode (iMote packet -> DSDV packet) RootNode -> Sensor (Flood -> iMote packet)RootNode -> Sensor (Flood -> iMote packet)



Sensor BoardSensor Board 18V power supply18V power supply

10kHz+ 24bit A/D10kHz+ 24bit A/D

Programmable antialiasing filterProgrammable antialiasing filter

PLD bridges SPI to UART interfacePLD bridges SPI to UART interface

Intel MoteSensor voltage supply,

A/D, filterVoltageoutput

UART900kb/s

SRAM 64kBFLASH 512kB

Wilcoxon sensor



Time domain dataTime domain data



Frequency domain dataFrequency domain data

80Hz reference

signal



StatusStatus Phase 2 development completePhase 2 development complete

Testing will begin in the JF3 chiller room with Testing will begin in the JF3 chiller room with MICA & iMote clusters next weekMICA & iMote clusters next week

Hardware is installed CUB3Hardware is installed CUB3

CUB3 deployment is scheduled for mid June CUB3 deployment is scheduled for mid June

Collecting performance and power data for Collecting performance and power data for platform comparison by end of Juneplatform comparison by end of June



JF3 Pilot DeploymentJF3 Pilot Deployment

FacilitiesRooms

FacilitiesRooms



Next StepsNext Steps Finalize phase 3 requirementsFinalize phase 3 requirements

Choose one Mote platform based on the Choose one Mote platform based on the phase 2 dataphase 2 data

Move to TinyDB/TASKMove to TinyDB/TASK



Key learnings (platform)Key learnings (platform) Size requirementsSize requirements

Not very sensitiveNot very sensitive Current solutions are much largerCurrent solutions are much larger Mote size is negligibleMote size is negligible

Power ConsumptionPower Consumption Sensor + A/D consume a lot (~60 mW)Sensor + A/D consume a lot (~60 mW) Can use large batteriesCan use large batteries

Large RAM is very usefulLarge RAM is very useful Adding more capabilities to the mote simplifies the Adding more capabilities to the mote simplifies the

sensor board designsensor board design Fast I/O on the mote is usefulFast I/O on the mote is useful



Key learnings (Network)Key learnings (Network) Automatic configuration of the network is requiredAutomatic configuration of the network is required Reducing hop count is keyReducing hop count is key Heterogeneous networks are very usefulHeterogeneous networks are very useful Simple power saving protocols are sufficient Simple power saving protocols are sufficient

(Cluster based)(Cluster based) Matching radio bandwidth to application Matching radio bandwidth to application

requirements can save powerrequirements can save power Fast network response time is needed, even if Fast network response time is needed, even if

collection frequency is infrequentcollection frequency is infrequent Polling specific sensors and adding streaming Polling specific sensors and adding streaming

modes will be very usefulmodes will be very useful



Key learnings (Network)Key learnings (Network) Debug modes would be very usefulDebug modes would be very useful

Tracing network topology and data flowsTracing network topology and data flows

Performance and power monitoringPerformance and power monitoring

Isolating bad data and recovering from Isolating bad data and recovering from failures failures



Key Learnings (backend)Key Learnings (backend) Interfacing into existing tools is extremely Interfacing into existing tools is extremely

importantimportant Want to use Rockwell EnShare for network Want to use Rockwell EnShare for network

control/commandcontrol/command

Getting info into/out of Rockwell was very painfulGetting info into/out of Rockwell was very painful

Easing the installation process is very Easing the installation process is very desirabledesirable Automatically recognizing sensors and their Automatically recognizing sensors and their

location is very useful (sensor -> equipment location is very useful (sensor -> equipment mapping)mapping)

Documents

® Copyright© Intel Corporation 2000-2004 Industrial Applications for Sensor Networks Condition based monitoring pilot project Lama Nachman Researcher Intel