Embed Size (px)
Citation preview
1 Copyright © 2014 Tata Consultancy Services Limited
Dr. Arpan PalPrincipal Scientist and Head of ResearchInnovation Lab, KolkataTata Consultancy Services
Research Challenges in Internet of Things (IoT)
May 3, 2023
2
Internet-of-Things
M2M Communication
Sensing the human – quantified selfEmbedded software and Hardware
Cloud, Mobile, Big Data and Analytics
Wireless Sensor Networks, Pervasive Computing
Sensorsand Actuators
Revenue Potential - $300+ Billion for Technology and ServicesEconomic Value - $1.9 Trillion
IoT - “a world-wide network of uniquely addressable and interconnected objects, based on standard communication protocols”.
Objects that are -• uniquely addressable• aware of their “characteristics, context and situation” • share information about themselves and surroundings• actively participate in business processes and offer services• have embedded sensors / actuators • enable data collection, monitoring, decision making & optimizations
3
Pervading all aspects of our life – Internet-of-Everything
Humans
Physical Objects and Infrastructur
e
Computing Infrastructur
e
Peop
le
Cont
ext
Disc
over
y
PhysicalContext Discovery
INTERNET OF EVERYTHING
Physical Context Discovery
What is happening, where and when
People Context Discovery
Who is doing what, where and when, who
is thinking what
Internet of
Digital
Internet of
Things
Internet of
Humans
ABI Research. May 7, 2014
• New Business / Pricing Models, Always On–Anytime–Anywhere, Secure, Context-aware - need to guarantee ROI for sustainability
• Customer becomes the focus, not the product or service – key is understanding the Customer
• Analytics need to understand the Physics and Chemistry of the Physical World and the Physiology and Psychology of the Humans
4
IoT Architecture – complex Ecosystem and complex Technology stack
Sensor Manufacturers
Board Manufacturers
Cloud Infrastructure Providers
BAN
KIN
G
INSU
RAN
CE
AGRI
CULT
URE
HEAL
THCA
RE
GOVE
RNM
ENT
UTI
LITY
MAN
UFA
CTU
RIN
G
TRAN
SPO
RT
APPLICATION SERVICES
INFRASTRUCTURE PLATFORM
INTERNET
GATEWAY
RESP
ON
DSE
NSE
ANAL
YZE
EXTR
ACT
Processor and Semiconductor Manufacturers
Network Equipment Manufacturers
System Integrators and Application Developers
Embedded System Developers
Domain Experts
Telecom / M2M Providers
Data Scientists
Edge
Network
Cloud
Embedded Devices - gateway, mobile, wearable
Sensor Signal Processing
Protocols and
Networking
Parallel and
Distributed
Computing
Analytics
Security and
Privacy
Model-driven
Development (MDD)
5
Fall Detection
PPG extraction
Eye Image / Video
Cardiovascular Model
Pulse Oxymetry
Pupilometry
FingertipVideo
Lung Function
Blood Pressure
Microphone
Accelerometer
Digital Stethoscope
Heart Rate
Using Mobile Phone Sensors for Physiological Measurements
Activity / Calorie
ECG
Respiratory RateHRV / Stress
Signal Processing for Noise Cancellation and Feature ExtractionMachine Learning on top of Physical Models for human Physiology
6
Mobile Phone based Automotive Insurance
• Phone Picture – VIN identification and damage assessment – OCR and real-time 3D reconstruction under noisy conditions
Need to simplify and speed up car accident insurance claim
• Driving behavior analysis and Road Condition Monitoring using mobile phone accelerometer – Noise Modeling, Signal Processing, Statistical Processing
Need to promote safe driving and preventive maintenance
Acceleration a(t) = f (H(t), v(t), R(t), D(t))
7
• Shopper Localization in Retail Stores• Emergency Evacuation in Large Buildings• Occupancy Estimation for Energy Savings
Need to localize people indoors
Mobile Phone based Indoor Localization
Geo-fencing • Using Magnetometer
Proximity
Detection
• Using Bluetooth RSSI
Inertial Navigati
on
• Step Count + Stride Length (personalized model)
• Gyroscope and Magnetometer-corrected Inertial Navigation
Wi-Fi based
Zoning
• RSSI based using attenuation modeling of the building - Unsupervised Learning
Fusion • Kalman Filter based Tracking with Particle Filter based Correction
8
• Personalize education based on real-time measurement of cognitive load
• Getting unbiased feedback from subject on usability
Why Measure Cognitive Load
Cognitive Load on Human Brain – EEG and GSR processing
Cognitive Load 23+45=? 1846890129 + 2374609823=?
EEG GSR
Signal Processing for Noise Cancellation and Feature ExtractionMachine Learning on top of Cognitive Models for human Psychology
9
• Unobtrusive Human Identification at Home – TRP analytics• Neuro-rehabilitation
Application for Skeleton Analytics
Kinect Signal Processing
Research– 20 joints of skeleton data– Gait cycle detection– Feature extraction from skeleton
joints– Training– Recognition– Gait Analytics
• 2D Camera with IR depth sensor
• Excitation by IR light pattern
10
Multi-sensor Fusion for Robot-assisted Sensing
www.ese.wustl.edu
Cloud point from 3D vision
Possible gas / heat
source (ROI)
Source direction
and intensity
• Robot carries 2D camera and thermal / microphone array• 3D reconstruction from the 2D vision• Estimation of Heat / sound Source through passive directional
signal processing• Fusion of thermal / acoustic map with optical 3D –
computational thermography and audiography• Gas Sensors planned in future
Application in remote sensing in hazard-prone areas
11
Requirements for IoT Platform
Applications need support for
VisibilityCapture & store data from sensors
InsightsPatterns, relationships and models
Control Optimize and actuate
TCUP – TCS Connected Universe Platform - horizontal platform to address IoT Software and Services market
TCUP Platform
• To balance between energy cost, communication cost and computing cost
Distributed Computing on Edge Devices
• To reduce network congestion
Adaptive, Lightweight yet Secure Communication Protocols
• For economical scaling of sensor data storeEfficient Compression
ManageScale,
Reduce Cost,
Improve Battery
Life
Handle Privacy
Easy to Use Analytics
Semantic Interoperability
12
Horizontal operators(semantic integration) operates on data from heterogeneous sources to created integrated data streams.
Sensor Data Analytics and Semantics - From Data to Wisdom
temperature
humidity
odor
image
high temperature
gaseous odor
light
concentrated light
high temperature indicates fire
gaseous odor indicates gas discharge
Fire from Gas Leak, evacuate
immediately, send fire fighting team
equipped with gas leakage
data
information
knowledge
wisdom
Vertical operators(semantic abstraction) operates on artifacts at each level and transcends them to the next level
F PCS(Data, KB*) → Information
F PCS(Knowledge, KB) → Wisdom
F PCS(Information, KB) → Knowledge
KB: Knowledge base
Adopted from: Physical-Cyber-Social Computing: An early 21st Century Approach, Amit Sheth et. al.
13
A bigger challenge for Analytics – a wide variety of stakeholdersI only know the business logic, I do not know how to code, nor do I
understand analytics
algorithms…
I know how to code, but I do not know
algorithms, nor do I know about the business logic…
Oh, I know algorithms, but I
can’t code for your mobile devices…
I have all these cloud and edge
nodes which you can use to deploy
the app…
Need for Knowledge based Model-driven-development
14
Source: www.winlab.rutgers.edu/~gruteser/papers/fp023-roufPS.pdf
Privacy Breach in IoT Applications
Pattern of living, activity, occupancy revealed
Even Sleeping Smartphones Could Soon Hear Spoken CommandsNuance is working with chipmakers on technology that would enable “persistent listening” apps. http://www.technologyreview.com/news/429316/even-sleeping-smartphones-could-soon-hear-spoken-commands/MIT Technology Review, Sept. 2012
Vehicle Trip Overlay Over a Year reveals your hub locations (home, office??)Source: https://www.aclu.org/technology-and-liberty/meet-jack-or-what-government-could-do-all-location-data
Data cannot be both contextually useful as well as forever privacy preserving
Need Balance between Privacy and Security
15
Innovation Lab Kolkata -at-a-glance
• Associates in R&D100+
• Researchers40+
• PhDs6
• Pursuing Higher Study8
• Papers published in last two years – www, SenSys, Mobihoc, UbiComp, Infocomm, ICASSP, …..
125+
• Patents filed in last two years60+
• Patents granted till date15+
• Standard Body Participation and Contribution
IETF, GISFI, TSDSI
Partnering Institutes (RSP, Research Collaboration) Indian Statistical Institute Institute of Neuroscience IIT Kharagpur, Mumbai, Guwahati Jadavpur University Calcutta University
Missouri S&T SMU University of Maryland MIT, University of Toronto /
Waterloo (Exploring)
Long term Masters / PhD
interns
16
Awards and Mentions
TCUP - Winner in Leading Edge Proven Technology
CoAP - IETF Fellowship from ISOC
Mobile Blood Pressure - Best Demo Award
Editorships for IEEE and ACM Transactions
17
References1. Philip B. Gibbons, et.al, IrisNet: An Architecture for a Worldwide Sensor Web, October 2003 IEEE Pervasive
Computing , Volume 2 Issue 42. Open Geospatial Consortium, OGC Sensor Web Enablement Architecture,, December 20083. Deborah Estrin , Participatory Sensing: Applications and Architecture, January/February 2010, IEEE Internet
Computing 4. Michael Chui, et.al, The Internet of Things, McKinsey Quarterly 2010, Number 25. W3C Incubator Group, Semantic Sensor Network XG Final Report, Report 28, June 20116. Dennis Pfisterer et.al, SPITFIRE: Towards a Semantic Web of Things, November 2011, IEEE Communication
Magazine7. S Bandyopadhyay, P Balamuralidhar, A Pal, Interoperation among IoT Standards, Journal of ICT
Standardization, 20138. P Balamuralidhara, P Misra, A Pal, Software Platforms for Internet of Things and M2M, Journal of the Indian
Institute of Science, 20139. Bandyopadhyay, S. and Bhattacharyya,
Lightweight Internet protocols for web enablement of sensors using constrained gateway devices , ICNC 201310. S. Bandyopadhyay, A. Bhattacharyya, and A. Pal, Adapting protocol characteristics of CoAP
using sensed indication for vehicular analytics SenSys, 201311. A. Ukil, S. Bandyopadhyay, A. Bhattacharyya, and A. Pal,
Lightweight security scheme for vehicle tracking system using CoAP, ACM ASPI-Ubicomp Adjunct, 2013.12. A. Ukil, S. Bandyopadhyay, A. Bhattacharyya, A. Pal and T. Pal, Auth-Lite
: Lightweight M2MAuthentication reinforcing DTLS for CoAP, IEEE Percom, 2014.13. A Bhattacharyya, S Bandyopadhyay, A Pal,
ITS-Light: Adaptive Lightweight Scheme to Resource Optimize Intelligent Transportation Tracking System (ITS)–Customizing CoAP for Opportunistic Optimization, Mobiquitous 2014
14.Arpan Pal, Aniruddha Sinha, Anirban Dutta Choudhury, Tanushyam Chattopadyay, Aishwarya Visvanathan, A robust heart rate detection using smart-phone video, ACM MobiHoc workshop on Pervasive wireless healthcare, 2013.
15.Anirban Dutta Choudhury, Aishwarya Visvanathan, Rohan Banerjee, Aniruddha Sinha, Arpan Pal, Chirabatra Bhaumik, Anurag Kumar, HeartSense: estimating blood pressure and ECG from photoplethysmograph using smart phones, ACM Conference on Embedded Networked Sensor Systems, 2013.
16.A Pal, A Visvanathan, AD Choudhury, A Sinha, Improved heart rate detection using smart phone , ACM SAC, 2014.
17.A Visvanathan, A Sinha, A Pal, Estimation of blood pressure levels from reflective Photoplethysmograph using smart phones BIBE 2013.
18
References
18.Vivek Chandel, Anirban Dutta Choudhury, Avik Ghose, Chirabrata Bhaumik, AcTrak-Unobtrusive Activity Detection and Step Counting Using Smartphones , Mobiquitous 2013
19.Avik Ghose, Provat Biswas, Chirabrata Bhaumik, Monika Sharma, Arpan Pal, Abhinav Jha, Road condition monitoring and alert application: Using in-vehicle Smartphone as Internet-connected sensor , PerCom Workshops 2012
20.Tapas Chakravarty, Avik Ghose, Chirabrata Bhaumik, Arijit Chowdhury MobiDriveScore-A system for mobile sensor based driving analyis: a risk assessment model for improving one’s driving, ICST 2013
21.Tanushyam Chattopadhyay, V Ramu Reddy, Utpal Garain, Automatic Selection of Binarization Method for Robust OCR , ICDAR 2013
22.Arindam Saha, Brojeshwar Bhowmick, Aniruddha Sinha, A System for Near Real-Time 3D Reconstruction from Multi-view Using 4G Enabled Mobile, IEEE MS 2014
23.A Mukherjee, A Pal, P Misra, Data Analytics in Ubiquitous Sensor-Based Health Information Systems, NGMAST, 2012
24.A Mukherjee, S Dey, HS Paul, B Das, Utilising condor for data parallel analytics in an IoT context—An experience report,, 9th IEEE International Conference on Wireless and Mobile Computing, Networking and Communications - IoT 2013 workshop
25.Felix Büsching et. al, DroidCluster: Towards Smartphone Cluster Computing--The Streets are Paved with Potential Computer Clusters, ICDCSW 2012
26.DP Anderson, Boinc: A system for public-resource computing and storage, Fifth IEEE/ACM International Workshop on Grid Computing, 2004.
27.A Banerjee, A Mukherjee, H S Paul, S Dey, Offloading work to mobile devices: an availability-aware data partitioning approach, MCS 2013.
28.S Dey, A Mukherjee, HS Paul, A Pal, Challenges of Using Edge Devices in IoT Computation Grids, ICPADS 2013
29.A Mukherjee, HS Paul, S Dey, A Banerjee, ANGELS for distributed analytics in IoT, WF-IoT 201330.R. Arasanal and D. Rumani, Improving MapReduce
performance through complexity and performance based data placement in heterogeneous hadoop clusters, ICDCIT 2013.
31.Pankesh Patel, Brice Morin, Sanjay Chaudhary, A model-driven development framework for developing sense-compute-control applications, MoSEMInA 2014
32.Bonomi Flavio, Rodolfo Milito, Jiang Zhu, and Sateesh Addepalli. Fog computing and its role in the internet of things, MCC workshop on Mobile cloud computing 2012.
33.Arpan Pal, Arijit Mukherjee, Balamuralidhar P, Model-driven Development for Internet of Things: Towards easing the concerns of Application Developers, IoTaaS, IoT 360, 2014
