Upload
beat-signer
View
939
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Presentation given at USI research seminar, Faculty of Informatics, University of Lugano, May 2007
Citation preview
USI Seminar, May 30, 2007
Bridging the Paper-Digital Divide:
The iPaper Interactive Paper Framework
Beat Signer
Institute for Information Systems, ETH Zurich
8092 Zurich, Switzerland
http://vub.academia.edu/BeatSigner
USI Seminar, May 30, 2007 Beat Signer, [email protected] 2
The Paperless Office (1975)
USI Seminar, May 30, 2007 Beat Signer, [email protected] 3
The "Paperless Office" (2003)
USI Seminar, May 30, 2007 Beat Signer, [email protected] 4
The Myth of the Paperless Office
For decades, people have predicted the
office of the future as a paperless office
documents generated, published and
distributed electronically
documents read electronically
What has happened to this
imminent revolution?
USI Seminar, May 30, 2007 Beat Signer, [email protected] 5
Affordances of Paper
The physical properties of an object determine
how people use that object
Properties of paper
light, flexible, robust, porous, opaque, transparent, …
Human actions
grasping, folding, tearing, carrying, writing on, …
Paper supports forms of collaboration and
interaction difficult to mimic in the digital world
USI Seminar, May 30, 2007 Beat Signer, [email protected]
E-Paper versus Digitally Augmented Paper
Electronic Paper (E-Paper)
Digitally Augmented Paper
USI Seminar, May 30, 2007 Beat Signer, [email protected]
Outline
Motivation
Interactive paper framework (iPaper)
Cross-media link server (iServer)
Interactive paper applications
functionality
authoring
Future work
USI Seminar, May 30, 2007 Beat Signer, [email protected]
Augmented Paper Projects
Digital Desk, Wellner, 1991 Listen Reader, Back et al., 2001
Audio Notebook,
Stifelman et al., 2001
Books with Voices
Klemmer et al., 2003 ButterflyNet, Yeh et al., 2005
USI Seminar, May 30, 2007 Beat Signer, [email protected] 9
Digital Pen and Paper
Developed by Anoto
Different pen manufacturers
Logitech, Maxel, Nokia, ...
Pattern space
60 million km2
USI Seminar, May 30, 2007 Beat Signer, [email protected]
Problems of Existing Applications
Many projects focus on the input device, paper,
printing and other hardware technologies rather
than on the data integration and information
management aspects isolated solutions
The linking of paper tends to be based on
physical rather than information-centric concepts
difficult to integrate new input / output devices
10
USI Seminar, May 30, 2007 Beat Signer, [email protected]
Our Approach
General interactive framework
different forms of paper-based interactions (enhanced
reading, enhanced writing, …)
rapid prototyping as well as large scale publishing
integration of new types of resources and services
integration of new input devices
The key to a highly integrated interactive paper
solution lies in the introduction of a platform for
general cross-media information management
11
USI Seminar, May 30, 2007 Beat Signer, [email protected]
Active Areas and Virtual Page Layers
image
video clip
USI Seminar, May 30, 2007 Beat Signer, [email protected]
iPaper Architecture
USI Seminar, May 30, 2007 Beat Signer, [email protected]
iServer Metamodel and Resource Plug-ins
iPaper
Shape Page
Application Database
DB ObjectData/Metadata
Query
Movie Clips
Time Span /
Spatial InfoMovie Clip
HTML Documents
XPointer XHTML
Document
Selectors
Entities
Resources
link
USI Seminar, May 30, 2007 Beat Signer, [email protected]
iServer
layer
|Layers|
layer
Active
Layers
resource
Containers
activeComponent
Active Com-
ponents
medium
Media
medium
Movies
medium
Images
medium
Webpages
OnLayer
Contains
(1,1)
(1,1) (0,*) (0,*)
document
Documents(0,*)
shape
Shapes
partition (2,*)
(0,*)
rectangle
Rectangles
ellipse
Ellipses
polygon
Polygons
circle
Circles
complexShape
Complex
Shapes
HasShapes
page
PagesOnPage
Contains
Pages
(0,*)
(1,1)
(1,1)
(0,*)
Resources
parameter
Preferences
link
Links
user
Users
group
Groups
individual
Individuals
HasMembers
HasTarget
(0,*)
(0,*)(0,*)
(0,*)
(0,*)
(0,*) (0,*)
(1,*)
(0,*)
(0,*)
(1,1)
(0,*)
partitionpartition
(1,*)
CreatedBy
RefersTo
Has
Preferences
parameter
Properties
selector
Selectors
resource
Resources
HasSource
(0,*)
AccessibleTo
InaccessibleTo
(0,*)
(0,*)
entity
EntitiesHasProperties
Users
Plug-ins
Links
(0,*)
Layers
iPAPER
iSERVER
USI Seminar, May 30, 2007 Beat Signer, [email protected]
Active Content
Execute code (services) on the client and server side
Active Component
A
Active Component
B
Active Component
C
iS
erv
er
AC Manager
Server
R
eq
ue
st S
en
de
rActive Component
Stub B
AC Names
ACS Manager
C
lien
t D
evic
e
AC Names
Client
HT
TP
USI Seminar, May 30, 2007 Beat Signer, [email protected]
Active Components
Button
Slider
Slider2D
DateSelector
Browser
MapSelector
Rating
Capture
CaptureAndSpeak
Image
Sound
Movie
SQLClient
XCMRequest
ClientCtrlRequest
PaperPointControl
PaperPointAnnot
...
USI Seminar, May 30, 2007 Beat Signer, [email protected]
Active Components …
<?xml version="1.0" encoding="UTF-8" ?>
<iserver>
<activeComponent id="s1a" creator="beat">
<name>Show slide 1</name>
<properties>
<parameter>
<key>org.ximtec.iserver.ac:command</key>
<value>showSlide</value>
</parameter>
<parameter>
<key>org.ximtec.iserver.ac:slide</key>
<value>1</value>
</parameter>
</properties>
<identifier>PAPERPOINT_CONTROL</identifier>
</activeComponent>
...
</iserver>
PAPERPOINT_CONTROL
Show slide 1
command showSlide
slide 1
USI Seminar, May 30, 2007 Beat Signer, [email protected]
PaperPoint Presentation Tool
USI Seminar, May 30, 2007 Beat Signer, [email protected]
XML-Based Authoring
...
<document id="handout" creator="beat">
<name>Slide Handout</name>
<id>h1</id>
<size>
<width>210</width>
<height>297</height>
</size>
<content>handout.pdf</content>
</document>
<page id="p1" creator="beat" document="h1">
<name>Page 1, Slide Controller</name>
<number>1</number>
</page>
...
<rectangle id="fs" creator="beat"
layer="default" resource="p1">
<name>First Slide</name>
<upperLeft>
<point><x>24</x><y>12</y></point>
</upperLeft>
<size>
<width>25</width><height>8</height>
</size>
</rectangle>
...
...
<activeComponent id="fsa" creator="beat">
<name>First slide action</name>
<properties>
<parameter>
<key>org.ximtec.iserver.ac:command</key>
<value>firstSlide</value>
</parameter>
</properties>
<identifier>PAPERPOINT_CONTROL</identifier>
</activeComponent>
...
<link id="fsl" creator="beat" sources="fs"
targets="fsa">
<name>Show the first slide</name>
</link>
...
USI Seminar, May 30, 2007 Beat Signer, [email protected]
Digital Link Authoring Tool
USI Seminar, May 30, 2007 Beat Signer, [email protected]
PaperPoint Architecture
USI Seminar, May 30, 2007 Beat Signer, [email protected] 23
Edinburgh Fringe Festival
World's largest
international arts festival
4 weeks
~1700 events
~27000 performances
USI Seminar, May 30, 2007 Beat Signer, [email protected] 24
EdFest Components
USI Seminar, May 30, 2007 Beat Signer, [email protected] 25
EdFest Booklet
USI Seminar, May 30, 2007 Beat Signer, [email protected]
Content Publishing Framework
XCMcontent publishing
XML data
of festival
venues
and events
iServercross-media link server
• paper link data
• active areas on paper
• layers
• digital resources
• user modelimport
publish
XML link
definition
data
iPublishpublishing framework
Structure Style
XSL CSS
USI Seminar, May 30, 2007 Beat Signer, [email protected]
Print-n-Link
Citations build the basis
for a scientific web of
publications
Use digital services to
search / retrieve cited
publications
Automatic authoring
USI Seminar, May 30, 2007 Beat Signer, [email protected]
Print-n-Link Architecture
Display
Printer
Driver
Document DB
iPAPER
iDOC
Publish
Document
Pattern
Semantic Mapper
Citation
Analyser
Citation
Retrieval
Document
Lookup
Citation DB
Print-n-Link Engine
iSERVER
Active Content
Link DB
iPaper Plug-in
Publish
Positions and
Active Content
Definition
Publish
Citations
Desktop
(x,y)
Coordinates
Text-to-
Speech
Web Search
Services
iPaper
Client
Print-n-Link: Weaving the Paper Web
Moira C. NorrieDept. of Computer Science
ETH Zurich8092 Zurich, Switzerland
Beat SignerDept. of Computer Science
ETH Zurich8092 Zurich, Switzerland
Nadir WeibelDept. of Computer Science
ETH Zurich8092 Zurich, Switzerland
ABSTRACT
Citations form the basis for a web of scientific publications.Search engines, embedded hyperlinks and digital librariesall simplify the task of finding publications of interest onthe web and navigating to cited publications or web sites.
However the actual reading of publications often takes placeon paper and frequently on the move. We present a systemPrint-n-Link that uses technologies for interactive paper toenhance the reading process by enabling users to access dig-ital information and/or searches for cited documents from a
printed version of a publication using a digital pen for inter-action. A special virtual printer driver automatically gener-ates links from paper to digital services during the printingprocess based on an analysis of PDF documents. Depend-ing on the user setting and interaction gesture, the systemmay retrieve metadata about the citation and inform the
user through an audio channel or directly display the citeddocument on the user’s screen.
Categories and Subject Descriptors
D.2.11 [ Software Engineering ]: Software Architectures;H.4.m [ Information Systems Applications ]: Miscella-neous; H.5.m [ Information Interfaces and Presenta-tion ]: Miscellaneous
General Terms
Design, Algorithms, Experimentation
Keywords
Interactive paper, document integration, digital library, ci-tation management
1. INTRODUCTIONNowadays we are all familiar with the notion of hyperlinks
as a means of creating interconnected document spaces thatcan be easily searched and navigated with the help of search
Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.DocEng’06, October 10–13, 2006, Amsterdam, The Netherlands.Copyright 2006 ACM 1-59593-515-0/06/0010 ...$5.00.
engines and web browsers. The ease with which we cannot only find documents on the web, but also follow linksto related documents is considered to be one of the mainadvantages of working with digital documents as opposedto paper equivalents. Yet many studies have shown thatpaper is still the preferred med ium when it comes to reading
documents for reasons of comfort, mobility and also ease ofannotation.
In this paper, we show how emerging technologies for in-teractive paper enable a bridge to be built between paperand digital document spaces that allows users to travel backand forth between the two using the medium most appropri-
ate to the current task and sit uation. Users can print doc-uments for reading and, by activating hyperlinks on paperwith a digital pen, they can not only access digital informa-tion about related documents, but also initiate web searchesto retrieve copies of those documents. The retrieved docu-
ments may in turn be printed for reading with automaticallygenerated embedded hyperlinks on the paper. Mobility issupported by providing disconnected users on the move withaudio information about related documents and the optionof adding them to a request list to be processed when recon-nected.
As an application, we take the everyday academic task ofreading scientific publications. Whether the reading activ-ity is for the purpose of knowledge discovery or reviewing,readers regularly look up citations to other publications byflicking to the reference list at the back of the paper. In somecases, a search for the cited publication may follow and, if
found, often it is printed and added to a pile of documents tobe read. Citations are the established meansof providing thelinks that form the basis of the scientific web of publicationsthat spans the digital and paper worlds. Just as citationswithin digital documents can be used to automatically gen-erate hyperlinks to cited documents, we can automatically
generate links within paper documents through an analysisof the PDF file when the document is printed.
Section 2 presents a more detailed look at the activityof reading scientific publications and a discussion of relatedwork. Section 3 describes the digital pen and paper tech-
nologies and how these can be used to support interactionbetween paper and digital services. We then present thegeneral functionality of our enhanced reading system in Sec-tion 4 and the overall architecture in Section 5. Details ofthe virtual printer component used to analyse PDF docu-ments for automatic link generation as well as printing are
given in Section 6. Concluding remarks follow in Section 7.
Interactive Paper Architecture
PrintingSemantic Mapper
Document Retrieval
USI Seminar, May 30, 2007 Beat Signer, [email protected]
Interactive Paper Applications
Enhanced Reading
interactive paper maps
interactive festival brochures
educational materials
auction brochures
scientific publications
…
Enhanced Writing
photo album
researcher's notebook
mammography
search system
…
Paper-Based Interfaces
PaperPoint
digital libraries
video analysis tool
educational games
…
Artists' Installations
Lost Cosmonaut
Generosa Enterprise
Interactive TableTops
USI Seminar, May 30, 2007 Beat Signer, [email protected]
Current Projects
PaperWorks (FP6-516895)
Interweaving Paper and Digital Documents
King's College London, Acreo AB, Anoto, ETH Zurich,
Arjowiggins SAS, Brunel University, Malmo
University, Pearson Education
DELOS
Network of Excellence on Digital Libraries
Paper-based user interfaces for image retrieval
(collaboration with Heiko Schuldt, University of Basel)
USI Seminar, May 30, 2007 Beat Signer, [email protected]
Future Work
Design guidelines for interactive paper interfaces
functionalty (active components) and design
Cross-media authoring and publishing
analysis of existing publishing processes (Pearson)
general document model for mapping between printed
and digital document instances
Extensions of iServer
collaborative cross-media information sharing
new resource plug-ins
USI Seminar, May 30, 2007 Beat Signer, [email protected]
Mapping between Printed and Digital
Instancesstatic physical rendering
dynamic logical document
USI Seminar, May 30, 2007 Beat Signer, [email protected]
Conclusions
Interactive paper framework (iPaper)
information-centric approach supports different
technologies, functionalities and applications
rapid prototyping (reusable active components)
Extensible platform for cross-media information
management (iServer)
Cross-media authoring / publishing tools
manual, content-driven and automatic link authoring