Streaming data with PI Integrators · Water Rate: 8% fic1001.C FR5001 AQUA2 -TI 201.PV TIC-181 BGE 03 FI-111 02T100 AF_FLOW3 02:T103 ACEDemo.Unit1.Output TI-178 B352_W778 0_CMP _SVLV

© Copyright 2018 OSIsoft, LLC© Copyright 2018 OSIsoft, LLC

Daniel Chaves, Systems Engineer

Dineshkumar Ambalavanan, Systems Engineer

27th November, 2018

Streaming data with PI Integrators

© Copyright 2018 OSIsoft, LLC 2

Agenda

Introduction

Challenges for advanced analytics projects

How do you address?

Streaming Analytics Live Demo

Key Takeaways

Appendix – Product Briefing

© Copyright 2018 OSIsoft, LLC

1Challenges for advanced analytics


What challenges

may slow or prevent success of

advanced analytics?


knowledge workers waste up to 50% of time hunting for data, identifying and correcting errors,

and seeking confirmatory sources for data they do not trust1

1 Harvard Business Review

2 Forbes

84% of CEOs are concerned about the quality of the data they’re basing their decisions on2

“

“

Challenge 1: Many versions of the truth exist

https://hbr.org/2013/12/datas-credibility-problem

https://www.forbes.com/sites/forbespr/2017/05/31/poor-quality-data-imposes-costs-and-risks-on-businesses-says-new-forbes-insights-report/#3254ae4c452b


RiskTime Expense

1https://hbr.org/2014/04/the-sexiest-job-of-the-21st-century-is-tedious-and-that-needs-to-change/

50-80% of data scientist time spent

collecting & preparing data1

Custom data integration requires

ongoing upkeep

Project will stall if business does not

understand “the why?”

0

20

40

60

80

100 Data prep &

cleaning

Analysis 47

16

Which are more difficult to address?

Organizationa

l challenges

Technical

challenge

s

Challenge 2:Just raw data and algorithms are not enough


Challenge 3: Integrate time series data with analytics platforms

Operationalizing insights

Automation and

productionisation of algorithms

CHALLENGES

Data extraction (70 sites): • Initial effort: 1-2 months

• Monthly effort: 1 month

Preparation:

• Initial effort: 1 week

• Monthly effort: 3 days

CHALLENGES


Capture Collect data from many

sources

Predict with a known level of accuracyApply

Algorithms CorrelateIdentify patterns

visually

Assumptions in approach:

1. Data has predefined schema

2. One data stream for one product

3. No relationships between data streams

Examples of assumptions failing:

1. Measurements from sensors in series have time offset

2. Tank temperature readings apply to one product

3. Algorithm predicts controller logic

Challenge 4: Pure replication is a trap


Example of challenges and success cases

Customer What

was done

Why

it matters

Industry

category

Marathon Oil Centralized, trusted source for IT

and OT data sets

• Decreased time to analyze holistic data set from 3 months to 3

weeks

• Decreased unplanned well downtime

Oil & Gas

White House

Utility District

Sped up identification of water

leaks across a large rural area

• Optimized services team’s workflow to save $30,000 per year

• Saved $900,000 in 2 years by preventing ongoing water leaks

Water

CEMEX Democratizing data for decision

makers

• Reduced time to begin analysis from months to minutes

• Reduced production data variations significantly

Metals,

Mining, &

Materials

Deschutes Predicting events in brewing

process across all brands

• Avoided $750,000 cost to automate density measurements

• Saving on average 48 hours of production time per batch of beer

Food & Bev

Centrica Automated daily reports

composed of different data types

• Inherited aging manual reporting process created by someone

who left the company 5 years ago

• Took 3 - 4 days to prepare each updated report, now updates

are automated with an initial creation of only 30 minutes

Power

Generation

http://www.osisoft.com/Presentations/The-PI-System-and-SAP-HANA--Operating-at-the-Speed-of-Business/

https://www.osisoft.com/Presentations/Maximizing-Operations-Utilizing-Real-Time-Spatial-Information/

http://www.osisoft.com/Presentations/Transforming-Process-Data-Into-Information--PI-Integrator-for-Business-Analytics/

http://www.osisoft.com/Presentations/Deschutes-Brewery--Reducing-Beer-Production-Time-with-Predictions/

https://www.osisoft.com/Presentations/PI-Integrator-for-Business-Analytics--or-as-I-Like-to-Call-It---Bunny-Catcher-/


2 How do you address?


What technology can

enable an agile process,

speed up time to value,

and utilize internal or external talent

for advanced analytics?


BGE003 FI-111 02T100AF_FLOW3 02:T103 AT401

ACEDemo.Unit1.Output

TI-178 B352_W778 0_CMP_SVLV_PCT

02F104 CD:F161

GE01_DT 409510395_Wind Speed QI-109GE01_DT Cooling Fan-711.Feed Rate

1-8.Net

Volume03LBA32CT0

01-2

DC.SJ.ITLoad.P

WR

TI-145FR2001

TI-178

GE04_OS

FT9001

FT9001FR5001

AF_NOISE

DC.SJ.PUE TI-102 DC.Zero DY-108DC.SJ.C1.Z3.R3.PDU1.PF GE01_A_DT

GE04_Status

02:F101.C

FIC-14402F100 fasttag

FI-1510_ENG_AUX_STS

AQUA2-SI005.PV GE02_Energy0_ENG_MODE_STSFI-151 02T103.QDC.Z1R

0_CMP_HDR

GE05_Energy C1:14AT5 AC03.Air FlowFeedBin.Cmt Boiler Cold Reheat Pressure

B737_FG117 DC.TimeLoad

D-110.Tank Pressure.PVGE04_DT QI-121 GE03_V_WIN

DC.Rk07RDC.Srv06R GE04_Energy

TI-121 FT9001

FAC.OAK.Power-Kh-Val.PV

fic1001.C GE02_OT

02F102.1HR

AVGBGT001

PI-111facility_output

DM-05:BW.R

DC.SJ.C1.Z1.R1.Rk06.S2.O03.PWR

QI-111FinalProductBin.On

94:GRDIDX.ProdIDBoiler-209.Fuel

Gas Flow

GE01_A_DT

fic1001.C

02:F101.C

FR2001 TIC-121

Aso AT401 DC.Srv01R

TI-178

asset1_outputActive Meters

GE01_A_DT aso

AF_NOISE PI-115DM-05:BW.R

FI-101Volume

0_CLR_FINAL_OUT_B_TMP F506_E990339511775_Clear SkyGlobalHoriz GE01_DT

FI-111

% CO2GE05_ES T

GE01_CONAlarmTest.Input.

Float32.1

DC.NY.Actual.PWR.day.Tot

80-5.Net VolumeGE01_A_DT

45-2.Net Volume

Boiler-209.Fuel Gas

Flow DC.Srv01R

94:GRDIDX.Tr

trigger

AC09.Power

403511195_Wind

Speed

DC.C2Z1.Pwr.Rippl

e GE01_A_DT

AF_NOISE

1-16.Net VolumeCB1992_MS

0_CMP_FLOW_TOTAL

FT9001

fasttag

FeedBin.Cmt

DC.Zone1.Number

FT9001

AlarmTest.Input.Float32.1

FT9001

DC.Srv01RBoiler-

125.Fuel Gas

Volume

1-13.Net

VolumeB045_FG978

AT401

Anacortes Refinery.Alkylation.Asset

Problems B210_FG005.KPIExcursion

D-110.Tank

Pressure.P

V Boiler Feed

Pump #1

Provide Context : Map raw data to the physical world

94:BW.R TI-101F723_E889

369512185-TempCompressor-439.Feed Rate

DC.CH.DCE FIC-172:210

FI-121 AF_FLOW3

0_ENG_MODE_STS

GE03_Q

DY-131:166 GE01_TD

Boiler-334.Feed Rate

GE04_OSAsset1.Problems QI-122 FI-151

DC.SJ.SiteRealTim

eITLoad.PR FT9001

Weather

Conditions

Relative Humidity: 34%

Current Temp: 85 F

High: 92

Low: 57 F

Wind: 8 mph/N

Crude Furnace

Draft Pressure: -0.5 WC

Stack Temp: 316 F

Oxygen: 2.5%

Outlet Temp: 840 F

Cold Oil Velocity: 6 ft/sec

Crude Desalter

Operating Pressure: 110 psi

Charge Rate: 14 gph

Mix Valve Pressure: 8 psi

Water Rate: 8%

fic1001.C FR5001

AQUA2-TI-201.PV

TIC-181

BGE003 FI-111 02T100

AF_FLOW3 02:T103

ACEDemo.Unit1.Output

TI-178 B352_W7780_CMP_SVLV_PCT

02F104 CD:F161

AQUA2-SI005.PV GE02_Energy

0_ENG_MODE_STSFI-151 02T103.Q

DC.Z1R 0_CMP_HDR_SUC_PRS

DM-05:BW.R

DC.SJ.C1.Z1.R1.R

k06.S2.O03.PWR

QI-111FinalProductBin.On

94:GRDIDX.ProdID Boiler-

209.Fuel GasFlow

FI-101 bf5e1d1d-

39c9-4b5b-b3d3-

c2ce05fa3a26DM-05:BW.R AT401

AT401

DailyTrigger FrqPrbCost_ER

GE01_DT 409510395_Wind Speed

QI-109 GE01_DT Cooling Fan


364511575-AC Power

DY-101 02:F101.C

0_CLR_FINAL_OUT_B_TMP F506_E990

339511775_Clear Sky Global

Horiz GE01_DT

80-13.Net

DY-131DC.SJ.PUE


GE01_DT

FI-101 bf5e1d1d-39c9-

4b5b-b3d3-c2ce05fa3a26DM-05:BW.R AT401

4-36.NetVolumeAC04.Air Flow

02T100 03LBB02CT001-2

DC.SJ.SiteRealTim

eITLoad.PR FT9001

02F102.1HRAVGBGT001

PI-111 facility_output

AQUA2-SI005.PV GE02_Energy

0_ENG_MODE_STS FI-151

02T103.Q DC.Z1R 0_CM P_ H D R _SU C _ PRS

403511195_Wind Speed

GE02_Energy

FI-101 bf5e1d1d-39c9- 4b5b-b3d3-c2ce05f


Status

Power

Overall Efficiency

Total Power

Unit % Uptime

Status

Power Power

Status

Both variables are potential features!!

Aggregate data: Raw data isn’t the only feature set for an

algorithm


Shaping the data: time-series data is not naturally aligned


Need a product that can embed these capabilities

15

Capture Collect data from many

sources

Predict with a known level of accuracyApply

Algorithms CorrelateIdentify patterns

visually

Model View operations with required granularity

Calculate Elevate raw data with rollups and equations

Contextualize Wrap user-friendly layer around data

Combine Merge with other business data

Why it matters?

Data has connection to the physical world

Required for feature selection

An analysis defines data granularity needed

Time-series data is not naturally aligned

© Copyright 2018 OSIsoft, LLC3 Streaming Analytics


RESULTSCHALLENGES SOLUTION

In a chemical reaction it is not easy to

monitor the change of the relative

concentration of the chemical substances

Due to regulatory requirements and quality

assurance it is critical to determine the time

in which the concentration is just right.

Using PI Integrator BA Advanced Edition, live

production data can be streamed to a ML model

with results written back to the PI System.

Operators are guided in their decisions through

the visualization of the predictions on a PI Vision

display.

Standardization of the information available to the

operators to enable better decisions and increase

predictability.

17

Predicting the end of a chemical production process

• Lack of real-time monitoring of the concentration of chemical substances during the reaction.

• Utilizing the measureable process values to predict the concentration of a substance.

• Usage of PI Integrator for BA (advanced edition) for data streaming.

• PI Vision as the link between the prediction system and the operator.

• Saving of training time by replacing

experience with digitalization.

• Optimizing batch run time

for predictability and higher output


Challenge

1. Start time of reaction

2. Weight of materials

3. Temperature

4. Energy from the reaction

GivenEnd time of the current reaction (when the concentration of NCO just reaches the optimal value)

Predict


Approach

Data PreparationData

Cleansing/Feature Extraction

Streaming target Machine Learning Operationalizing the results

PI AF PI Integrator Random Forest PI VisionEvent hub


Event Hub Machine

Learning

INGESTPREPAREOT INFRASTRUCTURE ANALYZE PUBLISH CONSUME

SQL DatabasePI System

On Cloud

Predictions

PI Integrator BA

Advanced Edition

20

Mobile/Desktop

PI Vision

Power BI


Streaming Analytics Demo


Demo Steps

Data Preparation• Context, analytics,

aggregation using PI AF

Data Shaping• CAST using PI

Integrators

Cortana Integration• Event Hubs

• Streaming Analytics

• Azure SQL DB

Prediction• Random Forest

using Azure Machine Learning

Operationalization• Operators guidance

using PI Vision

• Microsoft Power BI

1 2 3 4 5

© Copyright 2018 OSIsoft, LLC4 Conclusion


Leverage operations data. Utilize PI Integrators to support your business intelligence, machine

learning, and GIS projects.

Take advantage of flexibility. PI Integrators enable best of breed analytics platforms, allowing you

to start projects with lower risk.

Save time. Without any custom code, reduce the time and effort for data

scientists to prepare data and generate predictions, for field crews to

perform work, and for projects to deliver business value.

PI

Integrators

Key Takeaways


PI System integrates to many advanced analytics platforms

30

PI Integrator for Business Analytics 2018 Capabilities enabled

(requires non-OSIsoft

software)Target Destination Format Standard

Edition

Advanced

Edition

PI Views via ODBCrow-column

Combine OT and IT

data

Report holistic data

sets to the business

Find visual correlations

in PI System data

Flat Files

SQL Server

row-column

Oracle RDBMS

Apache Hive

Azure SQL Data Warehouse

Azure SQL Database

Hadoop HDFS

row-column

Train & retrain machine

learning algorithms with

PI System dataAzure Data Lake Store

Apache Kafka

stream

Operationalize

predictive models with

most recent PI System

data

Azure IoT Hub

Azure Event Hubs

data warehouses

messaging hubs

data

lakes


Coming Next - AWS

PI Integrator for

Business Analytics


Contact Information

Daniel Chaves

Systems Engineer

OSIsoft Australia

[email protected]

Dinesh Ambalavanan

Systems Engineer

OSIsoft Australia

[email protected]

32

mailto:[email protected]

mailto:[email protected]


Thank You

© Copyright 2017 OSIsoft, LLC© Copyright 2018 OSIsoft, LLC

Get started on your analytics journey now

© Copyright 2018 OSIsoft, LLC5 Appendix


PI Integrators speed the process that brings trustworthy datato many unique analytics tools

PI Integrators


Benefits of PI Integrators

36

Integrate operational data to your choice of advanced analytics platforms

Avoid delays from analyzing cryptic data sets

Eliminate the need to create & manage scripts that prepare operational data for other systems

HARMONY

CEMEX:

“Now we don't spend any time preparing data - It’s there.

Now, your time is spent in value-add activities…”

http://www.osisoft.com/Presentations/Transforming-Process-Data-Into-Information--PI-Integrator-for-Business-Analytics/

SAP HANA


PI System accelerates and operationalizes advanced analytics

Automation SystemsAssets Edge Devices / Sensors IoT solutions

data

warehousemessaging

hub

dat

a

lake

machine

learning

business

intelligence

tool

Enterprise Operations Infrastructure

push trained algorithm


2018 version

Choose Your Own BI Tool

Optimized Workflows

Enhanced Security

Improved Performance

Easily make changes to published

views; plus new logs and stats.

Changes to archived data pushed

automatically to supported views.

Hadoop target now supports

Kerberos authentication.

Distributed processing to improve

performance and scale.

PI Integrator for Business Analytics 2018:

Standard Edition

Includes multiple targets

Sync Changes to Your Data


PI Integrator for Business Analytics 2018:

Advanced Edition

39

2018 version released in

May 2018

Superset of Standard Edition

New Data Pattern

Support for New Platforms

Data Governance x2

Live Updates to Analytics

Transmit attribute-value pairs or

raw "packages" of data.

Stream PI data to Apache Kafka,

Azure IoT Hub and Event Hubs.

Support for Confluent Avro

Schema Registry with Kafka

Stream updated PI data when

a new snapshot is received.

All existing targets from 2018

Standard Edition included.


Documents

Streaming data with PI Integrators · Water Rate: 8% fic1001.C FR5001 AQUA2 -TI 201.PV TIC-181 BGE 03 FI-111 02T100 AF_FLOW3 02:T103 ACEDemo.Unit1.Output TI-178 B352_W778 0_CMP _SVLV