Collaboration through Cross-Media Annotation and Authoring

Prof. Moira Norrie

Global Information Systems GroupDepartment of Computer ScienceETH Zurich Switzerland

Application Modelling

Database & Application Design

Implementation

OM Semantic Object Data Model

OMS ProRapid Prototyping Systemand Lightweight DBMS

OMS Java, eOMS,…….

OMS Database Development Suite

OMS Application Frameworks

OMS Connect Peer-to-Peer database

connectivity

XIMA Multi-channel access, Voice

Interfaces

OMSwe, XCM Web engineering, versioning

mechanism for context-

dependent delivery

iServer Cross-Media Link Server

MDDQL Metadata driven collection and

querying of scientific data

1. Cross-Media Linking and Annotation

Myth of the Paperless Office

Reforming Paper

Affordances of Paper

– Cheap, Light, Mobile, Flexible,….

– Actions of Folding, Tearing, …..

– Supports Multiple Sheets, Multiple Users, ……

Bridging the Paper and Digital Divide

– Try to combine media and obtain best of both

worlds

– Range of technologies for digital paper and pens

Anoto Technologies

Pen manufacturersSony EricssonLogitech Nokia

Virtual paper space of 60 million km2

HP Forms Automation System

Aimed at Large Enterprises e.g. Insurance

Print-on-demand Forms

Each Form Instance

Identifiable

HP Pen based on Logitech

Pen

Form uploaded when pen

placed in cradle

Related Technologies

Many other related technologies, including:

Acreo - papers displays

Papergate - paper biometrics, electronic

delivery notes

Plastic Logic - plastic screens, flexible e-paper

NTERA - electrochromic display

technology (paper-quality displays)

EU Project Paper++

video clip

image

Linking of active areas to digital resources

Investigating low cost solutions for enhanced reading

web page

Paper++ Technologies

Digitally Augmented Paper

This is an example paragraph of text that is selectable within this page.

document position

transform positions to shapes

Based on conductive inks and special position encoding

General Cross-Media Link Server

Paper++

Shape Page

Application Database

DB ObjectData/Metadata

Query

Movie Clips

Time Span /Spatial Info

Movie Clip

HTML Documents

XPointer XHTMLDocument

Selectors

Entities

Resources

link

Layer 10

Layer 5

Anchor 1

Anchor 2

P(x,y)

Virtual Page Layers

Can introduce layers such that shapes belong to a layer and we return shape of “top active layer”

iServer Model

layer

|Layers|

layer

ActiveLayers

resource

Containers

textComp

Texts

medium

Media

medium

Movies

medium

Images

medium

Webpages

OnLayer

Contains

(1,1)

(1,1) (0,*) (0,*)

Paper++

document

Documents(0,*)

shape

Shapes

partition (2,*)

(0,*)

rectangle

Rectangles

ellipse

Ellipses

polygon

Polygons

circle

Circles

complexShape

ComplexShapes

HasShapes

page

PagesOnPage

ContainsPages

(0,*)

(1,1)

(1,1)

(0,*) Resources

parameter

Preferences

link

Links

user

Users

group

Groups

individual

Individuals

HasMembers

HasTarget

(0,*)

(0,*)(0,*)

(0,*)

(0,*)

(0,*) (0,*)

(1,*)

(0,*)

(0,*)

(1,1)

(0,*)

partitionpartition

(1,*)

CreatedBy

RefersTo

HasPreferences

parameter

Properties

selector

Selectors

resource

Resources

HasSource

(0,*)

AccessibleTo

InaccessibleTo

(0,*)

(0,*)

entity

EntitiesHasProperties

UsersPlug-ins

Links

(0,*)

Layers

Natural History Museum Study

iServer

General cross-media link server

Independent of specific digital pen and

paper technologies

Authoring tools developed for creation of

links between paper and digital media

Peer-to-Peer architecture for link sharing in

user communities

Annotation of Scientific Data

b) Annotating images

The nucleus also contains the nucleolus whichhelps to synthesize ribosomes. The nucleoluscontains nucleolar organizers which are partsof chromosomes with the genes for ribosomesynthesis on them. Copious amounts of RNAand proteins can be found in the nucleolus aswell.

The nucleus also controls the synthesis ofproteins in the cytoplasm through the use ofmessenger RNA. Messenger RNA is producedin the nucleolus of the cell and travels to thecytoplasm through the pores of the nuclearenvelope.

USA

N. America

Japan

Canada

UK

Germany

W. Europe

France

Korea

Australasia

S. America

Asia

E. Europe

Africa

1989

305

1993

315

1996

317

1999

344

297

220

220

166

210

158

150

89

139

26

20

42

5

303

222

217

190

214

163

170

92

137

27

23

39

4

317

236

231

210

211

171

176

112

143

31

26

32

4

330

238

247

217

213

190

180

141

145

35

27

20

5

Worldwide Paper Consumption

0 50 100 150 200 250 300 350

USA

N. America

Japan

Canada

UK

Germany

W. Europe

France

Rep. of Korea

Australasia

S. America

Asia

E. Europe

Africa

kg per annum

1999

1989

1989

1

2

3

4

5

6

7

8

9

10

11

12

13

14

Graphic paper grades make uparound 50% of the EU's paperproduction, packaging papergrades account for 40% andhygiene and specialty papers foraround 10%.

a) Annotating tables

Video

Webpage

Text

Chart(table)

Chart(column)

Text(cell)

Mammography Demonstrator

Based on

iServer, Anoto

Technologies,

Logitech io pen

iServer Annotation Framework

core model

link authoring

model

anchor authoring

visualisation

content authoring

iServer

resourceplug-ins

links

model annotations

annotationplug-ins

Annotation Framework

Edinburgh Festival Demonstrator

Interactive Paper Maps

and Brochures

Audio i/o channel

Dynamic Authoring of

Reviews

Authoring of Restaurant

Locations on Map and

Reviews

Sharing of Authored

Information among

Tourists

Based on iServer, OMSwe and modified Anoto pen for instant interaction and authoring

2. Collaborative Editing

Collaborative Editing

Real-Time Collaborative Editor based on Operation

Transformation and Hierarchical Document Model

…

…

Document

Pa 1 Pa 2 Pa 3

Se 3.1

Se 3.2

W 3.1.1

W 3.1.2

C 3.1.2.1

“C”

Document History

Pa3 History

Se 3.1 History

W3.1.2 History

…

…

…

W 3.1.3

C 3.1.2.2

“A”

C 3.1.2.3

“R”

History for operations at

paragraph level

History for operations on sentences in paragraph Pa3

History for operations on words in sentence

Se3.1

History for operations on characters in word

W3.1.2

Collaborative Editors

Real-Time (Synchronous) Textual Editor

Real-Time Graphical Editor with Group/Ungroup

Asynchronous Textual Editor based on

Operational Transformation for Merging

Asynchronous Graphical Editor based on

Operational Transformation for Merging

Asynchronous Graphical Editor with State-

Based Merging

Collaborative Cross-Media Editing

Synchronous or asynchronous merging

Relevance to e-Science

Important to support scientific work practices

– Co-located and remote synchronous and asynchronous

collaboration

– Mobility within and between environments an important

factor

– The design, planning, logging and documentation of

experiments important as well as data collection and

analysis

– Important to consider the role that paper plays in these

activities and how it can be integrated rather than replaced

– Important to develop infrastructure that can adapt to

emerging technologies

Collaborators

Anoto, Sweden

HP Barcelona and HP Research Labs, Bristol

Work, Interaction and Technology Group,

King’s College London

Arjo Wiggins Fine Papers, UK and France

Acreo, Sweden

Anitra Medienprojekte, Germany