Geert Baeke & Tim JacobsXylos Clients’ Day – 18/10/2016
Let “the things” do the work for you:
IoT, from idea to practical solution
Agenda
• ThingTank?
• Measuring Air Quality
• Anatomy of a device
• Receiving data in the cloud
• Processing data in the cloud
• Making predictions with data
• Conclusions
ThingTank and you
• Internet of Things:• Consultancy: business case & TCO/ROI, project planning, IoT architecture
• Hardware: custom development or leverage existing devices
• Backend systems: data reception, (big) data storage & handling
• Analytics:• Reporting & dashboarding: interactive + realtime
• Predictive analytics: statistic modeling, log analytics, data mining, anomaly detection, ….
• In IoT context, or standalone context, using device generatedor your own data
Agenda
• ThingTank?
• Measuring Air Quality
• Anatomy of a device
• Receiving data in the cloud
• Processing data in the cloud
• Making predictions with data
• Conclusions
Measuring air quality
• What to measure in training & meetingrooms? • We decided to focus on:
• WHO / EPA / LNE+VITO typically add:• Radon
• O3 – Ozone
• NOx – Nitrogen Oxides
• CO – Carbon Monoxide
• Other VOC: Benzene / Naphthalene
Substance Indicator for…
CO2 Air pollution by people
Formaldehyde (HCHO / CH2O) Air pollution through office equipment
PM10 (fine dust) Air pollution through external sources
Temperature & Relative Humidity General air quality
smoking & combustion related
Functional requirements of the solution
• Common-Of-The-Shelf (COTS) components• Standard components & sensors
• Cheap & disposable
• Did we say: cheap?
• As many “as-a-service” components
• Scalable
VISU
AL
REA
LTIMEB
ATC
HC
LOU
D
REC
EPTIO
NTHIN
GS
Solution ArchitectureLegendMicrosoft
Azure Service
External Service
Hardware
Agenda
• ThingTank?
• Measuring Air Quality
• Anatomy of a device
• Receiving data in the cloud
• Processing data in the cloud
• Making predictions with data
• Conclusions
Hardware choice
• Arduino• Open-source hardware & software since 2005
• Dozen different configurations / complexity
• Arduino Mega 2560 + Arduino GSM shield• Atmel ATmega2560 processor, 8-bit at 16 MHz
• 8 kB of RAM, 256kB flash memory
• Need ±10ms time measurement accuracy
Assumptions we made
• Sensors are plug & play• Try connecting 5 sensors instead of 1
• Data sheet galore
• Reliability out of the box• Supplied code stable?
• Exception handling is non-existent
(IoT) Security
• Authentication: “shared secret” requirement (per device)
• Transport security: TLS requirement• SSL/TLS on 8-bit Arduino?
• Protocol Gateway introduced in architecture
• Cellular: private APN to avoid public internet
• Physical security
Agenda
• ThingTank?
• Measuring Air Quality
• Anatomy of a device
• Receiving data in the cloud
• Processing data in the cloud
• Making predictions with data
• Conclusions
IoT Hub Messaging
MQTTAMQPHTTP
D2C&
C2D
Send cloud-to-device messages
Receive device-to-cloud messages(Event Hubs-compatible endpoint)
IoT Hub Device Registry
• Every device is registered and has a secret key
•Basic UI
•Not extensible
IoT Hub DocumentDB
IoT Hub Scaling
• Three tiers: S1, S2, S3 and F1 (Free)
• Pricing per unit
• No automatic scaling• You set the amount of units
• Limit is 200 units (default)
• Throttling will occur if you exceedthe limits*
* See https://azure.microsoft.com/en-us/documentation/articles/iot-hub-devguide/#throttling
How to process device messages?
Azure Stream Analytics
• Multi-tenant managed service
• Direct input from IoTHub or Event Hub
• Subset of T-SQL to select from and transform the stream of data
Azure HDInsight Storm
• Deploy HDInsight cluster of type Storm
• Direct input from IoTHub or Event Hub using a Storm spout
• To process the stream of data, you need to write code
• Java and other languages
EventProcessorHost
• Write .NET code to process the stream of data
• EventProcessorHosthides the complexities and provides checkpointing
• Host the code in a Cloud Service or WebJob
Azure Stream Analytics (ASA)
Event Hub-compatible endpoint
D2C&
C2D
Stream Analytics
Input Query Output
Job
Stream Analytics Query
SELECT firmware,
version,
time,
count,
IoTHub.ConnectionDeviceId AS deviceId,
avgTemp
INTO "airqdl“
FROM "airQ-IoTHub";
Demo
• Scaling….. IoT Hub & Stream Analytics
• Could run into several problems:• Device creation throttling
• Device connections
• Device send to cloud limits
Agenda
• ThingTank?
• Measuring Air Quality
• Anatomy of a device
• Receiving data in the cloud
• Processing data in the cloud
• Making predictions with data
• Conclusions
Data Lake and ELT
• Data Lake is like a really big file server• Structured with folders
• Store our telemetry data in .csv files! Really!!!• Extract (E) and Load (L) the data
• Transform (T) as per later requirements
• Why?• Low cost and easy to use
• Scale
• Azure Active Directory integration
• Play with new stuff!!!
Data Lake Analytics
Azure Data Factory
Telemetry data in Azure Data Lake
Azure Data Lake Analytics Job
Output results to Azure Data Lake
Azure Data Factory
1 - Run this job
2 – Grab the results
3 – Send to Azure SQL Server
Realtime Processing
Event Hub-compatible endpoint
D2C&
C2D
Stream Analytics
Event Hub
EventProcessorHostWebJob
Agenda
• ThingTank?
• Measuring Air Quality
• Anatomy of a device
• Receiving data in the cloud
• Processing data in the cloud
• Making predictions with data
• Conclusions
Learning from data (in development)
• Azure Machine Learning• Predict subjective experience of Indoor Air Quality
• Predict warning/critical threshold traversal
• Anomaly Detection• Find outliers in measurements
• Hard-coded limits, e.g. CO2 1200 ppm
• Soft limits?
Anomaly detection for Air Quality solution
Azure Machine Learning
Anomaly Detection Database
Azure Function
Azure SQL Database
IoT Hub(MQTT bidirectional)
Agenda
• ThingTank?
• Measuring Air Quality
• Anatomy of a device
• Receiving data in the cloud
• Processing data in the cloud
• Making predictions with data
• Conclusions
Key takeaways
• Devices: take it step by step: go from Proof-of-Concept to Prototype
• Security is key, in hardware & software!
• Backend systems: choose a platform that can be used for small & big projects, that scales along.
• Analytics: don’t think of the most complicated analytics first. Start simple & grow as you learn from your data
Call to action
• Where are you in your digital transformation? • How can your business grow
using IoT and analytics?
• How can we help in your digital journey?