Protecting workers today…

© 2021, Amazon Web Services, Inc. or its Affiliates.

Gavin Adams, IoT Specialist Solutions Architect September 23, 2021

Protecting workers today…

… and tomorrow using AWS IoT services


Agenda

AWS IoT overview

Defining worker safety, and safety over time

AWS for Industrial

Demonstrate object detection for worker safety


Fundamentals of AWS IoT


If you knew the state of every thing and could reason on top of that data…

what problems would you solve?


Predictive maintenance

Wellness &health solutions

Connected buildings and city systems

Maintaining device fleets

Monitoring energy efficiency

Enhance the safety in the home, the office, and

industrial sties

IoT payment and connected commerce

Remote patient monitoring

What are customers doing with AWS IoT?


AWS IoT architecture

How can I make sense of my IoT data and take appropriate actions?

How can I control, manage, and secure my devices at scale?

How can I build devices that operate at the edge?

Analytics Services

Connectivity & Control Services

Device Software


Device software

Edge Cloud

Law of Economics

Law of Physics

Law of the Land

AWS IoT Greengrass helps connect your devices to AWS, while reducing latency, reducing cost, and improving regulatory compliance

AWS IoT Greengrass


AWS IoT Greengrass

ML Inference Container SupportConnectors Secrets

ManagerStream

Manager

Perform ML Inference locally

Extend edge devices with connections to external

services

Deploy secrets to edge devices

Use AWS Lambda, Docker, or a

combination of both

Collect, process, & export high-volume data streams from

edge devices


Worker safety solutionsCommon solutions and more recent events


7

RIR by Industry (US) – (Industrial sectors)

425K+cases


Personal protective equipment (PPE)

Normal considerations for worker safety:PPE and specific job


Other ways to protect workers

Fall detection – fainting, slipping

Ergonomic risk factors– awkward postures, repetition, static postures

No-go areas – geo-fencing, lanes of traffic, overhead material


Achieving safety through machine learning

Object detection model Plume dispersion model

Credit: Los Alamos National Laboratory


Safety over time example - natural gas extraction site

Sitepreparation

Drilling Wellcompletion(hydraulicfracturing)

Production ReclamationProjecttimeline

Safetytimeline

PPE,heavyequipment

PPE,falling,material

PPE,respirationprotection,fluid spill

PPE*,methane,authorized personnel

PPE,heavyequipment


Natural gas extraction: site preparation

heavy equipment

person

Detection Modelclassifications


Natural gas extraction: drilling

person

hard hat



Natural gas extraction: well completion

person

hard hat

respirator

fluid spill

hydrogen sulfide (⌖⌖⌖)



Leveraging detection at scale

prepare-v3.pt

drill-v6.pt

completion-v11.pt

produce-v1.pt

Machine Learning Models


Worker safety for compelling events

face cover detection hand cover detection head cover detection


Worker safety for compelling events

Amazon Rekognition PPE detection


Summary: worker safety solutions over time

Industries will have specific requirements, almost all can go beyond typical PPE

Solutions change over course of projects or during changes to environment

Multiple approaches can be used or combined (ML + sensors)

Compelling events require timely changes to the norm


AWS for IndustrialImplementing worker safety solutions


What is AWS for Industrial?

aws.amazon.com/industrial

Making it easy for customers to digitize, monitor and optimize their industrial operations

http://aws.amazon.com/industrial


Key customer needs across common industrial workloads

Accelerate time to market

Lower infrastructure costs

Improve collaboration

Lower costs (energy, machines, labor)

Improve machine or asset productivity

Reduce production downtime

Automate quality inspection

Improve inspection accuracy

Reduce product defects

Improve employee productivity

Monitor safety

Reduce worker injuries

Improve forecast accuracy

Reduce inventory costs

Improve capacity utilization

Enhance user experiences

Create new revenue streams

Improve product or service quality

Engineering & Design

Production & Asset Optimization

Quality Management

Worker Safety & Productivity

Supply Chain Management

Smart Products & Machines


End-to-end capabilities for industrial use cases

AI & MLAnalytics APIs and Developer Tools

Cloud

Operational Data Historians MES

Industrial IoT

Data Platform

Factory Edge

SCADA

Edge

Asset Models Industrial Data Lake

Time Series Data Connectors Relational

Databases

Visualization Dashboards Alerts Anomaly Detection

Computer Vision

Connectivity Data Ingestion Data Ingestion Event Processing

Disconnected Mode Local Decisions OTA Security


Private subnet

AWS Industrial reference architecture

AWS IoTGreengrass

AWS IoT SiteWise Edge

Industrial Environment

OT App

AWS SnowballEdge

Historian

OPC-UA

AWS Cloud

Amazon Kinesis Firehose

AWS IoT Core

Amazon SageMakerNeo

Amazon QuickSight

Amazon Athena

AWS IoT SiteWiseAWS Glue(ETL job)

Amazon API Gateway

AWS AppSync

PLC/DCS

S3 Bucket

Amazon Cognito

Amazon CloudFront

Web App

Amazon Route 53

Data Lake

AWS Glue(Crawler/Catalog)

Amazon S3(data)

Amazon Redshift

Amazon Neptune

Cameras

Connected Worker

MQTT

Amazon Lookoutfor Equipment

Asset Amazon Monitron

AmazonTimestream

AWS Panorama

Amazon Lookoutfor Vison

Prot

ocol

Con

vert

or

Model

EC2 instance

AWS IoTGreengrass

AWS Storage Gateway

Lambda function

AWS IoT Events

Amazon Forecast

Smart Machine

AWS IoT Core

Amazon RDS Amazon EC2

VPC – Enterprise Apps / HPC

Amazon EBS

ELB

Notification

Amazon SNSNotifications

AWS Glue

Amazon DynamoDB AWS Lambda

Amazon KinesisData Firehose

NotificationAWS Fargate Amazon Kinesis

Data Streams

Amazon S3

Amazon SageMaker

Operator

Business Users

Data Scientist / Engineer

Version 1.0

Amazon S3

Amazon AppStream 2.0


Amazon SageMaker

Fully managed data processing jobs and

data labeling workflows

One-click collaborative notebooks and built-in, high performance

algorithms and models

One-click training

Debugging and optimization

One-click deployment and

autoscaling

Visually track and compare experiments

Automatically spot

concept drift

Fully managed with auto-scalingfor 75% less

Prepare Build Train & Tune Deploy & Manage

101011010010101010000011110

Collect and prepare

training data

Choose or bring your own

ML algorithm

Set up and manage environments

for training

Train, debug, and tune models

Deploy model in

production

Manage training runs Monitor models

Add human review of

predictions

Web-based IDE for machine learning

Automatically build and train models

A managed service that helps customers build, train, and deploy machine learning models


Mapping EHS Functions to AWS Stack

Simple Incidents (Safety Saves)Safety Observations (Floor Safety Inspections/Behavior Based Safety)Industrial HygieneWaste ManagementComplex IncidentsInspectionsAuditsRisk Register/Risk Assessment/Job Safety AnalysisAction Tracking/Compliance Calendar/Permits

“Typical” EHS Functions

Many-to-many = messy = no obvious starting point


Mapping AWS Stack to Safety Use Cases

Simple Incidents (Safety Saves)Safety Observations (Floor Safety Inspections/Behavior Based Safety)Industrial HygieneWaste ManagementComplex IncidentsInspectionsAuditsRisk Register/Risk Assessment/Job Safety AnalysisAction Tracking/Compliance Calendar/Permits

“Typical” EHS Functions

Precedent exists for computer vision and predictive analytics – recommend starting there

1

2


Computer vision reference architecture

Industrial Environment AWS Cloud

Cameras

AWS Panorama

Model

AWS IoTGreengrass

AWS Storage Gateway

Amazon SageMaker

EnvironmentalHealth

Safety Staff


Amazon S3

AWS IoT Core

MQTT

Trained Models

Inference Results

Inference Results

InferenceCode

- Investigate- Train- Test- Deploy- Review

Inference Results


AI/ML: AWS Panorama

Turn existing onsite cameras into powerful edge devices

Analyze video feeds from multiple cameras in parallel, and generate highly accurate CV predictions within milliseconds

Bring-your-own or select partner CV applications such as PPE detection, social distance monitoring, license plate detection, and more from Bigmate, Clarifai, CVEDIA, Indus.ai, Pilot AI, Siemens Mobility, and TIBCO Software

Panorama system integration and consulting service partners: Accenture, Deloitte, Capgemini, Slalom, TensorIoT, Quantiphi, Provectus, and LTI

Get started with the AWS Panorama Appliance Developer Kit

“The Vancouver Fraser Port Authority is the third-largest Port in North America, processing 3.5 million shipping containers a year. We’re constantly looking for new ways to improve the performance of the gateway, including through the use of technology, such as AI. The computer vision capabilities Deloitte, through its Future of Mobility team, is developing on AWS Panorama has the potential to be a powerful new tool to help us increase capacity and ensure the safe movement of shipping containers through the Port, and to bring increased visibility to our supply chain partners.”

Port of Vancouver

Build and manage computer vision applications at the edge


Summary: AWS for Industrial

Uniting Partner and AWS solutions in a single initiative

New purpose-built solutions focused on AI/ML

Addresses all industrial use-cases, not only worker safety

Greenfield and Brownfield supported


Worker safety demonstration


Demonstration architecture

Amazon SageMaker

Video source file

Environment Health

Safety Staff


Default ModelRun

JupyterLab

PPE ModelRun

Industrial Environment

Camera input Initial non-PPE trained model

PPE trained model

AWS IoTGreengrass

InferenceCode


Getting started


Model training (transfer learning) and deployment

One model per use-case or potentially per-location

Use at least 1,500 images per classification for best accuracy

Virtuous cycle of model trainingSave live images/videoCreate ground truth jobsRetrain and validate model results (mAP scores)

Target “good enough” for model size versus accuracy


AWS programs for industrial builders

Engage with a solution architect dedicated to your account and industry

Brainstorm use cases and get support for solution implementation

Solution Architects Identify key areas where machine learning

could help you with an AI or IoT expert

Let AWS experts build POCs and custom solutions for your industrial use cases

ML and IoT Solutions Lab

Discovery, Advisory and Implementation services working along with Partners

Training workshops and hackathons for builders in your organization

AWS Professional Services Let AWS validated partners help you with

the POC and build integrations with your current systems to productionize

Industrial domain-specific expertise

AWS Partner Network


AWS Partner solutions to accelerate time to value

aws.amazon.com/marketplacepartners.amazonaws.com

http://aws.amazon.com/marketplace

http://partners.amazonaws.com/


Summary


In closing…

Works safety is a continuum over time

Use reference architectures and offerings such as AWS for Industrial

Pick a specific use-case and iterate from there

Engage your AWS account team for IoT and ML best practices and approaches


In closing…

You can do this!


Q&AGavin Adams, IoT Specialist Solutions Architect

Documents

Protecting workers today…