Portals and Infrastructure for Communicating Smart Objects · 2003-12-30 · Portals and...

Portals and Infrastructure for Communicating Smart Objects

SAP International Research and Applications Congress Feb. 2001

Friedemann MatternETH Zurich

EEEETTTTHHHH EidgenössischeTechnische HochschuleZürich

© F.Ma. 4

Swiss Federal Institute of Technology (ETH)

founded 1854, 330 professors, 12000 students

Department of Computer Science21 professors, 1000 students, 100 PhD students

Distributed Systems Group middlewaremobile computing ubiquitous computing

ETH Zurich, Switzerland

ETH Zurich

© F.Ma. 5

What If All Things Were Smart?

And communicate with each other? I‘m

smart

hello!

© F.Ma. 6

My Bag

What If Every Object Had Its Own Internet Home Page?

Or even better - its own portal?

© F.Ma. 7

What If Every Object Had a Smart Proxy in the Internet?

Real World

Internet

© F.Ma. 8

Outline

Smart Objects

5 Reasons for Ubiquitous Computing

Real and virtual worlds: bridging the gap

Smart labels

Commercial interest

Infrastructure

Friedemann Mattern, ETH Zurich

© F.Ma. 9

Smart Objects

Embedded processorsin everyday objectssmallcheaplightweight

Wireless communicationspontaneous networks

Sensors

Real world objects are enriched with information processing capabilities

© F.Ma. 10

MediaCup from TeCo, University of Karlsruhe

An Example of a Smart Object

© F.Ma. 11

Are responsivecommunicate with their environmentrequires new user interfaces

touching, moving, using them ; speaking to them ; ...?networked with other smart objects

Smart Objects

May find their place everywherework, home, entertainment ,...

Can remember pertinent eventsthey have a memory

Show context-sensitive behaviorthey may have sensors

- e.g., location awareness- or situation awareness

.... ...............

© F.Ma. 12

Ubiquitous Computing

Mark Weiser1952 - 1999 XEROX PARC

„In the 21st century the technology revolution will move into the everyday, the small and the invisible…“

© F.Ma. 13

An Application Domain:Smart Clothing

Sensors based on fabrice.g., monitor pulse, blood pressure, body temperature

Invisible collar microphones

Kidswearintegrated GPS-driven locatorsintegrated small cameras (to keep the parents calm)game console on the sleeve?

© F.Ma. 14

Outline

Smart Objects

5 Reasons for Ubiquitous Computing

Real and virtual worlds: bridging the gap

Smart labels

Commercial interest

Infrastructure

Friedemann Mattern, ETH Zurich

© F.Ma. 15

Processing speed and storage capacity double every 18 months(at same price)

Exponential increasewill probably go on for the next 10 years at same rate

First Reason for UbiquitousComputing: Moore‘s Law (1965)

© F.Ma. 16

2nd Reason: New Materials

Example: Light Emitting Polymerorganic semiconductorsplastic displays (~ 1 mm thick)applications soon (e.g., curved or flexible displays)

© F.Ma. 17

Another Example:Smart Paper, Electronic Ink

An electronically charged pencil rotates the “pixels”

Detailed view of the micro capsules

© F.Ma. 18

3rd Reason: Progress in Communication Technologies

Bandwidth of single fibers ~ 10 Gb/s2002: ~ 20 Tb/s with wavelength multiplex (often at no cost for laying new cable!)

Powerline techniquecoffee maker „auto-matically“ connected to the Internet

Wirelessmobile phone: GSM, UMTS wireless LAN (up to 10 Mb/s)

Room networks , body area networks

A bluetoothmodule

Nostalgia

© F.Ma. 19image source: “Die Zeit”

Ubiquitous Networking

Today, the Internet connects all computers

Tomorrow everyday objects will become smart and they will all be interconnected

© F.Ma. 20

4th Reason: Better Sensors

Miniaturized cameras, microphones,... pattern recognition, assisted by heuristicsspeaker recognition, speech controlled devices

Fingerprint sensor on mobile objects

Radio sensorswithout powersupply

Location sensorse.g., based on GPS

POSITION N 39°

43’17’’W 105°

01’26’’

© F.Ma. 21

E.g., spontaneous networkingobjects in an open, distributed, dynamic world find each other and form a transitory community

devices recognize that they „belong together“

5th Reason: New Concepts

may I help you?

I‘m a smarthome gateway,

let‘s all worktogether!

that‘s my localtext-to-speech

server!

not with methis time!

© F.Ma. 22

Outline

Smart Objects

5 Reasons for Ubiquitous Computing

Real and virtual worlds: bridging the gap

Smart labels

Commercial interest

Infrastructure

Friedemann Mattern, ETH Zurich

© F.Ma. 23

Virtual Worlds - It All Started with Data Processing

Data

- Data processing- Information processing- Simulation- Virtual Reality

Results

© F.Ma. 24

How to Bridge the Gap?

Virtual worldReal world

© F.Ma. 25

Narrowing the Gap

Virtual world

Real world

time

bar codelabels

manualdata entry

databases

files

© F.Ma. 26

Narrowing the Gap

Why not attribute every object a unique representation in cyberspace?

„virtual counterpart“

RFID tags

Virtual world

Real world

time

bar codelabels

manualdata entry

virtual counterpartsdatabases

files

© F.Ma. 27

Virtual Counterparts

Virtual world(Internet,Cyberspace)

Real world

virtual counterparts

pure virt. object(e.g. email)

© F.Ma. 28

Making Things Smart with Virtual Counterparts

Extend artifacts by information processing facilities embedded processorssensorstagging (bar code, RFID tags)

Real world

Virtual world

Virtual counterpartsrepresent their real-world artifacts in a virtual world

passive („homepage“) [e.g., HP‘s Cooltown project]

active („proxy“) orservice interface („portal“)

© F.Ma. 29

Display Virtual Counterparts of Labeled Artifacts

Label = Internet-URL (pointing to the bag‘s „home page“)e.g., recipe „on“ food for microwave oven

1

Internet

WWW server

HTTP get

Bought on 20 Aug 2001; last travel: to London Sep 2003; contained shirt no. 1342 and 1349; was on 17 Nov 2002 in Hotel Atlantic, room 317 ...

object of thereal world

displayinstrument and labelreader

2

label

© F.Ma. 30

Artifact Memory

1) Aug. 3rd, 2001: ….2) Aug. 5th, 2001, 10:34 …..3) Aug. 5th, 2001 10:37 ...4) ...

Proxies act as memoriesfor their artifacts

Updates triggered by events

Queries from the real world return memory content

Arrived in room 564 Bayview Hotel

10:34, Sue K.opens bag

who? where?when?

Sensors generate events

© F.Ma. 31

Active Proxies Replay Real-World Manipulations

new coordinates:

N 39.34.17E 13.26.43

operation“insert”

Context #317:duck “Traxi”

bag #5744

create contextobject

put duck into bag

© F.Ma. 32

An Old Paradigm?

Real-world objects have virtual peerswhich record a state and meta-information (type, owner, references, ...)

Actions in one world are reflected in the otherreal transactions manipulate virtual objectsvirtual actions trigger real-world actors

© F.Ma. 33

... But a New Quality!

Real-world objects have virtual peerswhich record a state and meta-information (type, owner, references, ...)

Actions in one world are reflected in the otherreal transactions manipulate virtual objectsvirtual actions trigger real-world actors

A new quality: everything has a unique virtual counterpart bridges are ubiquitousinteractions are immediate

© F.Ma. 34

How Can This be Realized?

Required: an adequate model of the real world and techniques for bridging the gap

RFIDs (i.e „smart labels“) could be such a bridging technique

?

?

© F.Ma. 35

Outline

Smart Objects

5 Reasons for Ubiquitous Computing

Real and virtual worlds: bridging the gap

Smart labels

Commercial interest

Infrastructure

Friedemann Mattern, ETH Zurich

© F.Ma. 36

RFID:„Radio Frequency Identification“

Identify objects from distanceSmall IC with RF-transponder

Wireless energy supply~ 1 m magnetic field (induction)

ROM or EEPROM (writeable) ~ 100 Byte

Price ~ $ 0.1 ... $ 1consumable and disposable image source: Portolano project

© F.Ma. 37

RFID tagRFID„reader“

energy

antenna

application

Components of an RFID System

~ 1 m

2 x 2 mm~ 3 cm

data

© F.Ma. 38

Flexible tagslaminated with paperself-adhesiveprintable (e.g., bar code)

RFIDs as „Smart Labels“

image source: Portolano project

© F.Ma. 39

Application Domains for RFIDs

Electronic article surveillance„EAS“ - anti-theft functionality

Inventory controlshops or mini bar in hotel rooms

Libraries, video rentalBaggage labels

© F.Ma. 40

Application Domains for RFIDs

Access token (e.g., ski pass)Ear clips for animalsTransport of mail and parcelsTracking of goods„Radio signature“ of documents...

© F.Ma. 41

~ 20 bytesHuman readability

if number printed as textNot writeableSensible to dirtVisual line of sight necessaryDependent on positionLow cost

label and reader electronicsFraud by illegal change of data is easyLow speedMax ~ 50 cm

> 100 bytesHuman readability

if label is printed WriteableNot sensible to dirtMay be coveredLargely position independentHigher cost

Fraud almost impossible

High reading speedMax ~ 2 m

© F.Ma. 42

A Context Sensitive CookbookK

itch

en

Sce

nari

o

Place grocery items on the kitchen counter

groceries are wrapped and equipped with RFID tags

Nearby display shows dishes that can be prepared with available ingredients

© F.Ma. 43

Outline

Smart Objects

5 Reasons for Ubiquitous Computing

Real and virtual worlds: bridging the gap

Smart labels

Commercial interest

Infrastructure

Friedemann Mattern, ETH Zurich

© F.Ma. 44

Commercial Interest inLinking „Atoms to Bits“

Connect the real world to the digital worlde.g., by using bar code readers or RFID readers

Number is transformed to an URL, associated Web page is returned

Identify object instance,not object type (e.g., UPC)

Business opportunity, e.g. www.connectthings.com(launched in Oct 1999)

© F.Ma. 45

Application Scenarios

Get information about real objectsWhat is contained in this medicine? Listen to music found in an ad.

Smart assistantWhat parts need maintenance? What is the layout of this machine?

Context awarenessIs this tool available here?

Smart home, car, office, ...Adapt to people‘s preferences...

© F.Ma. 46

Clicking on Real-World Objects?

www.bar-mail.orgaccess Internet sites withoutmanually entering URLsfind informationorder products...

Bar code reader connected to amobile phone

send codes via SMS to bar-mail serverserver may also send back an e-mail

„BarMailer“, a snap-in bar code reader for mobile phones

© F.Ma. 47

The CueCat Case

Bar code scanner (shape of a cat)LED based; attaches to the computer via the keyboard port

10 million free scanners distributed in the US by the end of year 2000

50 millions planned in 2001 estimated cost of $ 5 - $ 10 per CueCatsomeone spends at least $ 500 000 000

Sends the Web browser directly to the „right“ location when scanning the bar code of an ad in a magazine

© F.Ma. 48

Mapping of Bar Code to Web Page

bar code + CueCat Serial Number

User

Provider

user has to register to use the software

- map bar code to URL- log user data- send targeted

advertisements

Manufacturer, Advertiser, ...

bar code + serial number + user data

Web page related to product or document

1

2

3

4

© F.Ma. 50

Hackers knowhow to decrypt the code how to avoid sending the serial number

Provider doesn‘t like reverse engineeringopen bar code directories for productsfree applications (e.g., download amazon.com information from ISBN numbers on books)LINUX drivers

CueCat Reverse Engineering

The CueCat dissected

„Our revenue model is being the gate keeper between codes and their destination online“

© F.Ma. 51

Patent US5978773

„System and method for using an ordinary article of commerce to access a remote computer.“NeoMedia Technologies, Inc., Fort Myers, FLIssued / Filed Dates: Nov. 2, 1999 / Oct. 3, 1995

A system and method for using identification codes found on ordinary articles of commerce to access remote computers on a network. Inaccordance with one embodiment of the invention, a computer is provided having a database that relates Uniform Product Code ("UPC") numbers to Internet network addresses (or "URLs"). To access an Internet resource relating to a particular product, a user enters the product's UPC symbol manually, by swiping a bar code reader over the UPC symbol, or via other suitable input means. The database retrieves the URL corresponding to the UPC code. This location information is then used to access the desired resource.

© F.Ma. 52

Outline

Smart Objects

5 Reasons for Ubiquitous Computing

Real and virtual worlds: bridging the gap

Smart labels

Commercial interest

Infrastructure

Friedemann Mattern, ETH Zurich

© F.Ma. 53

Infrastructure for Smart Objects

“A Dancing Toaster” (Rich Gold, XEROX PARC)

© F.Ma. 54

Why Infrastructure for Smart Objects?

Guaranteesecurity privacy availability reliability

Provide serviceslocation („where am I?“) context („are we in a meeting?“) event delivery („tell me when... happens“)brokering („find a something that...)directoryregistry …

How do we organize billions of mobile smart objects that arehighly dynamic, short living,…?

for applications built with smart objects

for smart objects

© F.Ma. 55

More Infrastructure Tasks

Enable spontaneous networkingcooperation among smart objectscommunicationmobilityservice creationservice discovery (“is a service available that ...?”)...

Facilitate linking the real world to the virtual world

Challenge for practical computer science research!

for communities of smart objects

© F.Ma. 56

Using Events to Link the Two Worlds

Events in the real world are propagatedto their virtual counterparts

may carry parameters

Event distribution schemese.g., publish / subscribe

Event structurestandard eventsevent composition

Events may be generated by RFIDs e.g., appearance of an object

arrived in room 564 Bayview Hotel

new coordinates:

N 39.34.17E 13.26.43

&

© F.Ma. 57

Representation of Object Proxies?

Remember: real-world objects should have virtual proxies

Object orientedit‘s a simulation of the real world! (remember Simula?)

Structured data representationopen standard (e.g., XML)

Interface (API)object state, event history,...services connected to the object

Integrate all issues related to an object at some virtual place

wouldn´t one call that a portal?

My Bag

© F.Ma. 58

What Exactly is a Portal?

A buzzword?A Web site called „my xxx“?A Web site that integrates

contentservices (relevant to me)?

A single access point formemy smart devices (mobile phones,...)?

A Web page that can be configured and customized to increase its value?

A marketing instrument?B2C: stickiness (e.g., Yahoo), B2E: assistance, business operation, collaboration

My Portal

© F.Ma. 59

Object Portals –Characteristics and Benefits

An embodiment of a virtual proxyfor a real-world objectaggregating data and services

A single place to accessobject information

it‘s state, data, events, history,...

Access point to all relevant servicesother related objects

Firewallsecurityprivacy

state

firewall

service 1 service n

eventlist------------------------------

…

otherportals

APIAPI

application

APIAPI

application

MyD

uck

#081

5471

1

history

© F.Ma. 60

Object Portals...

Built upon the open Internet infrastructureprotocols (http,...), formats (XML,...), standards,...linking to Internet-aware backend systems (repositories, data bases,...)

Other issues (to be discussed at some other time):privacy?engineering paradigms, e.g., mobile code?emerging functionality?new industry for underlying infrastructure?...

Are object portals a good paradigm?

© F.Ma. 61

Will we ever get a common infrastructurefor smart objects?

scalableextensible...

Will it be open? based on common, open standardssimilar to the Internet and its protocols

Or will we get different proprietary solutions?could we invent the open Internet (TCP/IP,...) today?

Open or Proprietary Infrastructure?

Is it realisticor a dream?

© F.Ma. 62

Who Owns the Key?

Who knows the mapping from object IDs to data (such as a product homepage)?

copyright? trade secret? intellectual property?can one own such mappings?e.g., are phone book entries common knowledge?

Commercial value!linking bar codes on ads to the “right” web sitecustomer profiles when “clicking” real-world objectswhy share this knowledge with others?

Political issue? different attitudes in the US, in Europe,...?

© F.Ma. 63

Outline

Smart Objects

5 Reasons for Ubiquitous Computing

Real and virtual worlds: bridging the gap

Smart labels

Commercial interest

Infrastructure

Friedemann Mattern, ETH Zurich

Conclusions

© F.Ma. 64

The European „Disappearing Computer“ Initiative

New information artifactspossibly with sensors, micro mechanical systems, wireless connections,...

Emerging new functionalityfrom collections of interacting artifacts

awareness of other artifacts in the environment

Emphasis: new people-friendly environments design of new user interfaces

16 trans-European projects started in 2001more will follow

© F.Ma. 65

Vision: enable everyday objects as smart interconnected information artifacts by attaching computing devices to them

makes objects „self aware“

Next generation of „smart labels“ processor & memory wireless communication (e.g. Bluetooth)various sensors perceive the environment

customizable behaviorremote update

cheap, small, compact, autonomous

Project partners in Germany, Sweden, Finland(part of the European „Disappearing Computer“ Initiative)

The „Smart-Its“ Project

MediaCup (Teco, Univ. of Karlsruhe)

© F.Ma. 66

What Else Are We Doing atETH Zurich / Switzerland?

Research:General infrastructure for Ubiquitous ComputingCommunication and service environment for Smart-ItsEvent-based middleware for smart labelsSmart proxies for real-world objects

e.g., object portals...

My Bag

F.Ma. 56

arrived in room 564 Bayview Hotel

new coordinates:

N 39.34.17E 13.26.43

&

© F.Ma. 67

Research:General infrastructure for Ubiquitous ComputingCommunication and service environment for Smart-ItsEvent-based middleware for smart labelsSmart proxies for real-world objects

e.g., object portals...

Projects, industrial cooperations, and consulting within the M-Lab:M-Lab – a common institution of ETH Zurich(polytechnical) and University of St. Gallen(business)

Ubiquitous ComputingPervasive ComputingSmart Devices

What Else Are We Doing atETH Zurich / Switzerland?

© F.Ma. 68

Two Worlds that Collide ?

If there is tight interaction between the physical and the virtual world – what happens?

what is gained?what is lost?

Can we make a better world, or just better business?Which techniques are needed? Which are suitable?What are the limits?

????