Supporting Interactions across the Paper-Digital Divideweibel.ucsd.edu/talks/SBB_IT_Interactive Paper.pdf · 2009. 10. 13. · Vincent E. Giuliano of Arthur D. Little, 1975 “The

SBB IT Developer Day, December 16 2008, Muntelier

Interactive Paper

Supporting Interactions across the Paper-Digital Divide

Nadir Weibel [email protected] ETH Zurich Submission 5762

Nadir Weibel, ETH Zurich

SBB IT Developer Day, December 16 2008, Muntelier Nadir Weibel, [email protected]

The Myth of the Paperless Office…

-  “I don't know how much hard copy printed paper I'll want in this world.“

  George E. Pake, heads of Xerox PARC, 1975

-  “The use of paper should be declining by 1980.”

  Vincent E. Giuliano of Arthur D. Little, 1975

“The last paper edition of The New York Times will appear in 2018”

Dick Brass, vice-president of Technology Development

at Microsoft Research, 2001

For decades, people predicted the office of the future as a paperless office

The Office of the future


3

The Myth of the Paperless Office…

0

10

20

30

40

50

60

1975

1977

1979

1981

1983

1985

1987

1989

1991

1993

1995

1997

1999

2001

2003

2005

World Paper Consumption (kg/person/year)

What happened to this imminent revolution?


Affordances of Paper   The physical properties of an object determine how people use that

object

  Properties of Paper -  light, flexible, robust, porous, opaque, transparent, ….

  Human Actions -  grasping, folding, tearing, carrying, writing on ….

  Paper supports forms of collaboration and interaction difficult to mimic in the digital world


Integrate Paper and Digital Technologies

"Rather than pursue the ideal of the paperless

office, we should work toward a future in which

paper and electronic document tools work in

concert and organizational processes make

optimal use of both."

5

Abigail Sellen and Richard Harper, The Myth of the Paperless Office, 2002


iPaper General Architecture


Input Devices


Digital Pen and Paper   Encodes positions into a non-repetitive pattern

printed on paper

  Pattern space   60 million km2

  Developed by Anoto AB (Sweden)

  Different pen manufacturers   Logitech, Maxell, Nokia,   Adapx, Livescribe ...


FLY Pentop Computer

  Everything integrated into the pen

  Voice feedback

  Applications on cartridges -  english to spanish translator -  calculator (draw interface) -  ...


Pulse Smartpen

  Record and replay synchronised ink & audio

-  1 GB memory -  16'000 pages -  200 hours voice recording -  96x18 OLED Display -  Open application platform

(SDK to be released soon)


iPaper Framework and Interfaces   Developed in Java   Based on a Java Object-Oriented DBMS (OMS)   Interfaces based on XML/XSLT

-  XHTML -  VoiceXML -  Web Services

  Bridges to different technologies and API -  Interactions, printing and publishing services (Anoto) -  Voice services (AT&T voice engine,…) -  Microsoft COM services (Jawin) -  OpenOffice (UNO Runtime) -  Adobe Flex (Adobe Air) -  Character recognition (Vision Objects MyScript) -  Gesture recognition (iGesture)


Infrastructure


Interactive Paper Applications

  Enhanced Reading   interactive paper maps   interactive festival brochures   educational materials   auction brochures   scientific publications

  Paper-Based Interface   presentation tool   video analysis tool   educational games

http://www.ipaper.ethz.ch/applications/

  Enhanced Writing   proof-editing system   photo album   researcher's notebook   Mammography

  Artists‘ installations   Lost Cosmonaut   Generosa Enterprise

  Interactive TableTops   iTable

SBB IT Developer Day, December 16 2008, Muntelier Nadir Weibel, [email protected] 14

PaperPoint Presentation Tool

  Mobile presentation tool

  Non-linear presentations

  Paper-based real-time annotations

  Digital whiteboard

  Multi-pen support -  brainstorming


PaperPoint Architecture


Active Content / Active Components

  Execute code (services) on the client and server side


Active Components... Button

Slider

Slider2D

DateSelector

Browser

MapSelector

Rating

Capture

CaptureAndSpeak

Image

Sound

Movie

SQLClient

XCMRequest

ClientCtrlRequest

PaperPointControl

PaperPointAnnot

...


Active Components <?xml version="1.0" encoding="UTF-8" ?> <iserver> <activeComponent id="s1a" creator="beat"> <name>Show slide 1</name> <properties> <parameter> <key>org.ximtec.iserver.ac:command</key> <value>showSlide</value> </parameter> <parameter> <key>org.ximtec.iserver.ac:slide</key> <value>1</value> </parameter> </properties> <identifier>PAPERPOINT_CONTROL</identifier> </activeComponent> ... </iserver>

PAPERPOINT_CONTROL

Show slide 1

command showSlide

slide 1


Digital Link Authoring Tool


20

PaperProof, Paper-Digital Proof-editing Tool


Paper-Digital Structural Mapping

21

  Dual document representation in digital and physical document spaces


PaperProof

22


PaperProof


PaperProof Active Components

public void processEvent(ACEvent event) { //process pen input and store the operation //in an operation buffer }

public void processEvent(ACEvent event) { //process pen input and store the text //in the operation buffer }

public void processEvent(ACEvent event) { //retrieve all operations from the buffer //and execute them in OpenOffice }

public void processEvent(ACEvent event) { //delete last operation from //operation buffer }

  Event-based execution of program code


Replace Operation Example

1.  Interpret gesture: “scratch-out”

2.  Recognise handwriting: “selecting”

3.  Resolve command: “replace with selecting”

4.  Retrieve service from (x,y) position: “PaperProof synchronise service”

5.  Transfer positional metadata to PaperProof: “doc1, page1, (10, 25)”

6.  Map positional metadata to the digital element: “touching”

7.  Apply command to source document: “replace touching with selecting”


Conclusion

  General interactive paper framework -  different forms of paper-based interactions (enhanced reading,

enhanced writing, …) -  integration of new types of resources and services -  integration of new input devices

  The key to a highly integrated interactive paper solution lies in the introduction of a platform for general cross-media information management

SBB IT Developer Day, December 16 2008, Muntelier

Nadir Weibel http://www.globis.ethz.ch/weibel

ETH Zurich [email protected]

http://www.ipaper.ethz.ch

Documents

Supporting Interactions across the Paper-Digital Divideweibel.ucsd.edu/talks/SBB_IT_Interactive Paper.pdf · 2009. 10. 13. · Vincent E. Giuliano of Arthur D. Little, 1975 “The