Sensors/IOT · What are data? • Not general agreement! • Usually agreed on properties: • material (matter or energy at bottom) • this material basis can vary (lack of uniformity)

Sensors/IOT

Lecture Overview

• Data vs information• Data infrastructures• Open data• Introduction to IOT• Big data technologies for IoT data • Use of IoT in disaster management

10/5/2018 INFO319, Autumn 2018, Session 4 2

What are data?• Not general agreement!• Usually agreed on properties:

• material (matter or energy at bottom)• this material basis can vary (lack of uniformity)• the variations (or lack of v.) represent something

• Representation:• direct correspondence:

“the property/state of the data corresponds to some property/state of something else” (natural/intentional)

• symbolic correspondence:“the data contain symbolic language that describes something else” (intentional)

10/5/2018 INFO319, Autumn 2018, Session 4 Luciano Floridi: Information – A Very Short Introduction 3

What are data?• “Nondata”: material variations (or lack of v.) that do not represent anything

(rare!)• “Natural” data, data in the wild: material variations (or lack of v.) in nature

that represent something else• Human-made, artificial data: ← We are (mostly) here! material variations

(or lack of v.) that represent something else by human action• direct, “hand-made” artificial data• indirect, machine-generated artificial data

– non-recorded, recorded ← We are (mostly) here!• Non-rivalrous, non-excludable, marginally free• Data do not only represent, they also constitute reality

10/5/2018 INFO319, Autumn 2018, Session 4 Luciano Floridi: Information – A Very Short Introduction 4

Data are not information!• A common distinction:

– data may accommodate multiple interpretations– information = data + meaning

• ...the interpretation has become (more) fixed• Information is carried by (constituted by) data, but is not bound to

particular data:• a letter can be scanned into a PDF file.• when the letter is shredded, the data are lost.• but the information is still there in the PDF

• Data are in themselves, but the same information can be carried by different data in different forms at the same time (or different times)


(Some) types of data• Analog and digital• Qualitative and quantitative

•nominal, ordinal, interval, ratio for quantitative• Structured, semi-structured and unstructured• Primary (main purpose) and exhaust (side effect)

• secondary, tertiary• Metadata:

• about content: syntax, semantics• about dataset: descriptive, structural, administrative

• Indexical and attributive• Small and big!


Shapers of data• Collected data are not neutral, but shaped by:

• prevailing power structures• background and interests of collectors• data generation context• field of view / sampling frame• technology and platform used• data model / ontology• regulatory environment:

• e.g., privacy, data protection, security• Big data tend to be opportunistic / convenient

• small data tend to be purposeful / targeted


Data Infrastructures(Kitchin ch.2)


Big and small data


Data holdings, archives and infrastructures• Data holding:

• gathering, storing and making data available• Data archive:

• structuring, curation and documentation practices• institutional structures (e.g., to ensure longevity)• public services, e.g., cultural heritage, libraries• may be required by law

• Data infrastructure• institutional, physical and digital gathering, storing and making data available over the internet• adds data archiving functions


Data archive / infrastructure challenges• Data generation procedures• Data formats and data standards• Metadata and documentation• Data preparation, ingestion and cleaning procedures• Data quality and assurance measures• Preservation, backup and auditing policy• Software and hardware(!)• Security and data protection• Access, licensing, use, reuse, privacy and ethics policies• Ownership, copyright and IP rights policies• Administrative arrangements, management organization, governance mechanism• Funding of the infrastructure, its services and management


Qualities of open data sets and sites

• Clean, high-quality, validated, interoperable• Comply with data standards• Associated metadata and documentation• Preservation, backup and auditing policies• Reuse, privacy and ethics policies• Administrative arrangements, management organisation, governance

mechanism, financial stability• Long-term plan for development and sustainability


Disincentives to share (research context)

• Lack of rewards• Effort to prepare and archive• Expertise, resources and tools• Concerns over being first to extract value• Concerns over use: misuse, mishandling, misinterpretation• Worries about additional work: queries and requests• Concerns about transparency, being exposed, alternative interpretations• Intellectual property (IP) issues• Fear of no or little use: wasted effort


Data brokers and markets

• Large data collections are being built as central business resources, e.g.,

• detailed data about every person in the US• every household in Norway

• Recombined from diverse sources


Open Data(Kitchin ch.3)


Open data• Example definition:

• Knowledge is open if anyone is free to access, use, modify, and share it —subject, at most, to measures that preserve provenance and openness.

• Requirements:• technically open: open, standard format, physical availability, no-DRM or

similar constraints (DRM: Digital Rights Management software)• legally open: no legal restrictions, explicit open licenses

• Examples:• Open Definition of Knowledge: http://opendefinition.org/od/2.1/en/• Open Government Data, 8+7 principles: https://opengovdata.org/

• From product to service thinking?


Why open data?• Long tradition in some countries

• ...other are opening up• Drivers:

• measure success of (public) organizations, decision making, transparency, accountability, value for money

• active and informed citizenship: choosing schools and hospitals, political involvement, participative democracy, social innovation

• evidence-based monitoring and decision making, improved operational efficiency, competence and productivity, using information across departments, broader (“holistic”) views of organizations, more eyes

• low economic value → high commercial value, e.g. map data• brand enrichment, customer contact, trust and reputation


Why open data?

• Obstacles:– first-time preparation has a cost

• requires repurposing• curation (anonymity, aggregation)• developing new systems / services

– partly market-financed state agencies– legal limitations:

• public / private sector competition– lobbying from third-party resellers


Funding open data• Arguments for direct government backing:

– increased societal costs are offset by reduced company costs– free additional labor, improved data quality, crowd innovation– simpler, better, more efficient customer-handling– diverse consumer surplus value– new innovations and markets (GPS!), corporate revenue, corporate tax


Funding open data• Funding models for open data:

– premium version of free product/service– freemium product/service (graded options)– open source– free trial (razor), then paid (blades)– value-added services (i.e., semantics)– product/service store– advertising– customization

...resemble the funding models for open software!


Concern: neoliberal and market interests• Open data are not neutral• Example claims:

– driven by commercial forces– exploiting public goods for private benefit

• in turn weakens public data resources• must perhaps buy back from private sector

– public accountability drives neoliberal, NPM reorganization– transparency talk is just rhetorical

• business interests in disguise• not similar support for whistleblowing, IP liberalization, DRM-

restrictions etc.


Concern: benefits the already empowered

• Open data is not all– also: which data, and how they can change society

• Example claims:– open public data enhance value of privately held data– data are not neutral:

• which data to collect, generate and make open about who and what is highly political

• which interests are included, which excluded?• leveraging open data is labor and skill intensive:

technological, contextual, argumentative...– two types of public data: operational and citizen


Concern: sustainability and utility

• Supply focus– many open data sites / sets are low-hanging fruit– more interesting data sets may require curation

• e.g., repurposing, privacy concerns, regulation– created by volunteers, short-term projects– less focus on maintenance over time– danger of vicious cycles– shift needed:

• holdings → archives → infrastructures


Introduction to IOT


Introduction to IOT

• IoT = 1998, by Kevin Ashton• Definition: ”world-wide network of interconnected objects uniquely

addressable, based on standard communication protocols”.• Enabling new forms of communication between people and things,

and between things themselves.


Introduction to IOT• Internet of Things (IoT):

– sensors, actuators, other devices are on theinternet– TCP / IP → IP4 / 6 addresses of their own– physical endpoints are connected through uniquely identifiable IP addresses

• data can be gathered, aggregated, communicated and analyzed via embedded electronics and software.

10/5/2018 INFO319, Autumn 2018, Session 4 Source: https://www.i-scoop.eu/internet-of-things-guide/ 26

How Big is IoT?

10/5/2018 INFO319, Autumn 2018, Session 4 Source: https://www.i-scoop.eu/internet-of-things-guide/ 27

Visions of IoT

10/5/2018 INFO319, Autumn 2018, Session 4 Figure: https://arxiv.org/pdf/1105.1693.pdf 28

IoT: From connection to benefit


Key Technologies Involved in Internet of Things• Identification technology • IoT architecture technology• Communication technology • Network technology• Network discovery technology• Softwares and algorithms• Hardware technology• Data and signal processing technology• Discovery and search engine technology• Relationship network management technology• Power and energy storage technology• Security and privacy technologies, and • Standardization

10/5/2018 INFO319, Autumn 2018, Session 4 Source: https://arxiv.org/pdf/1105.1693.pdf 30

Architecture of Internet of Things

10/5/2018 INFO319, Autumn 2018, Session 4 Figure: https://arxiv.org/pdf/1105.1693.pdf 31

Logical architecture for IoT.

10/5/2018 INFO319, Autumn 2018, Session 4 32Source: https://docs.microsoft.com/en-us/azure/architecture/data-guide/big-data/

Unifying IoT:

10/5/2018 INFO319, Autumn 2018, Session 4 Figure: http://www.sensormeasurement.appspot.com/?p=m3 33

Benefits of IoT• Improved Customer Engagement

- e.g, issue in the car will be automatically detected by the sensors.

• Technical Optimization- e.g., The manufacturer can collect data from different car sensors

• Reduced Waste- e.g., For example, if a manufacturer finds fault in multiple

engines, he can track the manufacturing plant of those engines and can rectify the issue with manufacturing belt.

10/5/2018 INFO319, Autumn 2018, Session 4 Source: https://www.edureka.co/blog/iot-tutorial/#Benefits_of_IoT 34

Overlapping concepts / ideas• Web of Things (WoT):

– web APIs, HTTP, HTML / CSS / JS dashboards / UIs– sensoring, actuating etc. -as-a-Service

• Cloud of Things (ClouT):– gateways and services can be hosted– digital twins with histories, extrapolations, simulations...– the cloud, the edge and the fog

• ...a bit hyped as usual, but at least a quantitative change going on


Big data technologies for IoT data


Hortonworks Data Platform (HDP):

10/5/2018 INFO319, Autumn 2018, Session 4 Figure:https://hortonworks.com/blog/using-hdp-hadoop-platform-service/ 37

• Apache Hadoop, is a massively scalableand 100% open source platform for storing, processing and analyzing large volumes of data.

• It consists of the essential set ofApache Hadoop projects including MapReduce, Hadoop Distributed File System (HDFS), HCatalog, Pig, Hive, HBase, Zookeeper and Ambari.

Tutorial: Analyzing Machine and Sensor Data with Hadoop’s Hortonworks Sandboxhttps://github.com/costin/hadoop-

tutorials/blob/master/Sandbox/T14_Analyzing_Machine_and_Sensor_Data.md


https://github.com/costin/hadoop-tutorials/blob/master/Sandbox/T14_Analyzing_Machine_and_Sensor_Data.md

Use of IoT in disaster management


IoT in disaster management

• Monitoring of forest fires:- sensors on trees can take measurements that indicate when a

fire has broken out, or there is a strong risk, e.g. temperature, moisture, CO2 and CO levels.

• Detecting earth movements:- microwave sensors that can be used to measure earth

movements before and during earthquakes• Detect and measure floods:

- infrared sensors


IoT for predicting next Natural Disaster

• IoT technologies can’t stop disasters from happening, but can be very useful for disaster preparedness, such as prediction and early warning systems.

• Smartphones come with built-in accelerometers.• Other sensors can be very expensive.• Sensors do not need to be operated by experts


IoT for Helping Relief Efforts

• In the aftermath of Typhoon Pablo, which devastated the Philippines in 2012, UN Global Pulse were able to identify and analyze the time each post was uploaded, the GPS coordinates, and the types of damages in photos.

• After Haiti earthquake, chart the movement of displaced populations using their phone’s subscriber identity module or SIM number.

10/5/2018 INFO319, Autumn 2018, Session 4 Source: https://datafloq.com/read/how-big-data-will-be-used-predict-next-disaster/2434 42

What to do in Two Weeks?...and in the meantime :-)


Documents

Sensors/IOT · What are data? • Not general agreement! • Usually agreed on properties: • material (matter or energy at bottom) • this material basis can vary (lack of uniformity)