Embed Size (px)
Citation preview
“AI/ML solutions for low-power Edge platforms -challenges and opportunities”
Amit MateBangalore, India - October 17, 2020
PAGE 3| Confidential Presentation ©2020 Deeplite, All Rights Reserved
VISIT bit.ly/Deeplite FOR MORE INFO
WE USE AI TO MAKE OTHER AI FASTER, SMALLER ANDMORE POWER EFFICIENT
Automatically compress SOTA models like MobileNet to <200KB withlittle to no drop in accuracy for inference on resource-limited MCUs
Reduce model optimization trial & error from weeks to days usingDeeplite's design space exploration
Deploy more models to your device without sacrificing performance orbattery life with our easy-to-use software
Copyright © EdgeImpulse Inc.
TinyML for all developers
Get your free account at http://edgeimpulse.com
Test
Edge Device Impulse
Dataset
Embedded and edge
compute deployment
options
Acquire valuable
training data securely
Test impulse with
real-time device
data flows
Enrich data and train
ML algorithms
Real sensors in real time
Open source SDK
Maxim Integrated: Enabling Edge Intelligence
Sensors and Signal Conditioning
Health sensors measure PPG and ECG signals critical to understanding vital signs. Signal chain products enable measuring even the most sensitive signals.
Low Power Cortex M4 Micros
The biggest (3MB flash and 1MB SRAM) and the smallest (256KB flash and 96KB SRAM) Cortex M4 microcontrollers enable algorithms and neural networks to run at wearable power levels
Advanced AI Acceleration
AI inferences at a cost and power point that makes sense for the edge. Computation capability to give vision to the IoT, without the power cables. Coming soon!
Wide range of ML methods: GBM, XGBoost, Random
Forest, Logistic Regression, Decision Tree, SVM, CNN, RNN,
CRNN, ANN, Local Outlier Factor, and Isolation Forest
Easy-to-use interface for labeling, recording, validating, and
visualizing time-series sensor data
On-device inference optimized for low latency, low power
consumption, and a small memory footprint
Supports Arm® Cortex™- M0 to M4 class MCUs
Automates complex and labor-intensive processes of a
typical ML workflow – no coding or ML expertise required!
Industrial Predictive Maintenance
Smart Home
Wearables
Qeexo AutoML for Embedded AIAutomated Machine Learning Platform that builds tinyML solutions for the Edge using sensor data
Automotive
Mobile
IoT
QEEXO AUTOML: END-TO-END MACHINE LEARNING PLATFORM
Key Features Target Markets/Applications
For a limited time, sign up to use Qeexo AutoML at automl.qeexo.com for FREE to bring intelligence to your devices!
is for
building products
Automated Feature
Exploration and Model
Generation
Bill-of-Materials
Optimization
Automated Data
Assessment
Edge AI / TinyML
code for the smallest
MCUs
Reality AI Tools® software
Reality AI solutions
Automotive sound recognition & localization
Indoor/outdoor sound event recognition
RealityCheck™ voice anti-spoofing
[email protected] @SensorAI Reality AIhttps://reality.ai
SynSense (formerly known as aiCTX) builds ultra-low-power(sub-mW) sensing and inference hardware for embedded, mobile and edge devices. We design systems for real-time always-on smart sensing, for audio, vision, bio-signals and
more.
https://SynSense.ai
Next tinyML Talks
Date Presenter Topic / Title
Tuesday,October 27
Kristopher ArdisExecutive Director, Maxim Integrated
Manuele RusciEmbedded Machine Learning Engineer, Greenwaves Technologies
Cutting the AI Power Cord: Technology to Enable True Edge Inference
GAP8: A Parallel, Ultra-low-power and flexible RISC-V based IoT Application Processor for the TinyMLecosystem
Webcast start time is 8 am Pacific timeEach presentation is approximately 30 minutes in length
Please contact [email protected] if you are interested in presenting
tinyML India Committee
Chetan Singh ThakurAssistant ProfessorIndian Institute of Science
Anup RajputDirector (R & D)SAAR Microsystems Pvt. Ltd.
Sandipan ChatterjeeLead Data ScientistDXC Technology
Abhishek NairPhD Student (ML Accelerators)Indian Institute of Science
tinyML India LinkedIn
tinyML India Meetup
www.tinyml.org
Reminders
youtube.com/tinyml
Slides & Videos will be posted tomorrow
tinyml.org/forums
Please use the Q&A window for your questions
Amit Mate
Amit has 20+ years of experience in leading cross-
functional Engineering teams on ML and Wireless
projects from concept through commercialization. He
has delivered commercial grade software on several
deep-technologies ( 3G/4G, OCR, VR, Femtocells)
with Industry leaders such as Qualcomm and Nokia.
Amit earned his master’s degree in electrical
communication engineering from IISc, Bangalore and
bachelor's in electronics and communication from
NIT, Nagpur. He has been awarded 10+ patents
including 3GPP essential patents.
AI/ML SOLUTIONS ON LOW-POWER EDGE PLATFORMS - CHALLENGES
AND OPPORTUNITIESAMIT MATE
GMAC INTELLIGENCE
INTRODUCTION TO GMAC INTELLIGENCE
We are a B2B AI/ML Software company
Our mission is to build world’s best AI/ML software for robots and consumer devices
15• NVIDIA Inception cohort and GTC 2020 presenter
• IISc Deep-tech cohort
• 4th globally in Google visual-wake-word challenge-2019
• Recipient of Google TFRC Compute Grant
AI/ML software => real-time, on-device implementations of DNN models with state of the art accuracy and power efficiency for Edge devices.
GMAC INTELLIGENCE – CURRENT EDGE AI PRODUCTS
16
DMSANPR Robotics Speech recognition
Thanks to NVIDIA and Google TFRC team that has enabled us to train these models on GPU and TPUs and Qualcomm for
providing the platforms for these solutions
WHAT IS EDGE AI? HOW BIG IS THE MARKET?
Endpoints Smart Nodes Gateway
Clie
nt
Hom
eIn
dust
rial
17
GMAC’s
current focus
By 2025
TAM > 25B devices,
SAM > 6B Edge AI devices
SOM > at least one model on
2% of the devices
Source: ARM
WHY AI/ML IS MOVING TO THE EDGE?
On-device or Edge AI enables new applications and new revenue streams
For IoT/Edge device vendors and system integrators
Cloud based AI is often slow, expensive and raises security/privacy concerns
18
WHAT ARE THE MAIN CHALLENGES IN MOVING AI/ML TO THE
EDGE?
Constrained environment - IoT/Edge devices have limited memory (10KB++) , storage
(1MB++) , compute (50MHz++) and power (mW++)
Fragmented technology landscape – Tensorflow or Pytorch or TensorflowRT – uC or DSP
or CPU or GPU or NPU – Android or Yocto linux or Ubuntu
19
20
GMAC’S SOLUTION – ON-DEVICE ACCELERATION API + AI/ML
MODELS
Ready to deploy AI/ML
models
Biometrics, ANPR. DRL, OCR, Depth
Off-the shelf platforms – NXP, Qualcomm,
ARM, NVIDIA
SOTA performance
small footprint
high accuracy
low power
GMAC’s on-device
inferencing + training API
On-device inferencing in a
platform/compute-type/os agnostic way!!
On-device learning for a Edge-only
solutions
On-device & real-time
high fps
linux, android, native implementations
APPLICATIONS
VISION,ROBOTICS,SPEECH..
DNN DRL
ARCHITECTURE INNOVATIONS, LOSS FUNCTION, DATA
AUGMENTATION INNOVATIONS
HIGH LEVEL MODEL
G-MOT AI-MET
TFLITE SNPE/Pytorch
QC,NXP,NVIDIA,ARM
DSP/NPU CPU/uC GPU
GMAC’S ON-DEVICE INFERENCING AND TRAINING API
What API do I need to build any conceivable application?
registerModels(models, callbacks)
infer(inputs)
Can I train my models on an Android Edge device ?
Yes
21
EDGE ML WORKFLOW
SMART DATA COLLECTION AND
AUGMENTATION
INNOVATIVE MODELS & SMART
TRAINING TECHNIQUES
DEPLOYMENT READY MODELS ON
OFF-THE-SHELF EDGE-AI
PLATFORMS
CONVOLUTIONS – TYPES AND COST
Reference: Yusuke – Apr 2018
23Quiz#1 : What is the Moore’s law equivalent for Edge AI?
Quiz#2 : What is the god code for ANNs?
Quiz#3: Can we assemble a VON Neuman or Harvard architecture machine with MLP elements?
SEARCH FOR HIGHEST PERFORMANCE EDGE AI PLATFORM
24
Reference: http://ai-benchmark.com/news_2019_10_27_npus_review_2019.html
• Qualcomm Gen 3 NPUs better than
Huawei’s Gen 4 NPU
• Performance achieved using proprietary
quantization techniques
• Effort ongoing to mainstream it with rest of
the ecosystem!!
DEEP DIVE INTO PERFORMANCE COMPARISONS – FP16 BASELINE
25
Reference: http://ai-benchmark.com/news_2019_10_27_npus_review_2019.html
• Gen 4 NPUs approaching performance of
PC grade AI accelerators
• Mobile SoCs will continue to evolve at a
faster pace incorporating multi-core NPUs!!
CHALLENGES IN “ALWAYS-ON EDGE-AI”
Thermal throttling on ARM based systems – always-on , data/compute intensive AI/ML triggers it!!
TinyML or Ultra-low power ML to the rescue!!
26
TOUCHLESS ATTENDANCE USE CASE
Community Gardens
27
Community Staff
ATTENDANCE AND PAYROLL
Attendance and payroll are linked
Attendance mandatory for payroll, biometric is a must for many organizations!!
Pre-covid biometric was fingerprint – trust on card based systems is very low!!
What is the new touchless solution?
28
GMAC’S SOLUTION
29
Key Features
5 seconds/attendance, 20 seconds/registration, upto 10K registrations per device
Easy registration that can be enabled by security staff with few clicks
Edge security measures in place to protect data and system tampering
Thermal aspects
Display cant be on all time
Reliable detection of intentional presence at low-power, proximity dosen’t work
TinyML aspects
Can we build an ultra-low power yet accurate intentional presence detection?
Can we leverage TinyML thinking and extend this on-device solution to 1M -1B people ?
Copyright Notice
This presentation in this publication was presented as a tinyML® Talks webcast. The content reflects the opinion of the author(s) and their respective companies. The inclusion of presentations in this publication does not constitute an endorsement by tinyML Foundation or the sponsors.
There is no copyright protection claimed by this publication. However, each presentation is the work of the authors and their respective companies and may contain copyrighted material. As such, it is strongly encouraged that any use reflect proper acknowledgement to the appropriate source. Any questions regarding the use of any materials presented should be directed to the author(s) or their companies.
tinyML is a registered trademark of the tinyML Foundation.
www.tinyML.org
