The Reading Desk:Applying Physical Interactions to Digital Documents

Jennifer Pearson1, George Buchanan2, Harold Thimbleby1

1 FIT Lab, Swansea University 2 Centre for HCI Design, City University, Londoncsjen@swan.ac.uk, csharold@swan.ac.uk george.buchanan.1@city.ac.uk

ABSTRACTReading is increasingly being performed interactivelyon-screen; for instance, new novels are now routinelyreleased in electronic format for viewing on PCs andmobile devices. Unfortunately, on-screen reading losesmany of the natural features of conventional physicalmedia, such as the ability to annotate, slip in book-marks, turn page corners, and so on. How best shouldthese features be represented electronically? Can com-puterized representations give bene�ts that excel theconventional bene�ts of paper? We describe the de-sign and implementation of a novel reading system thatmimics key properties of paper and surpasses them byincorporating digital techniques. A comparative userstudy evaluating the system con�rmed the e�ectivenessof the features and the value of the system as a whole.

Author KeywordsDocuments, Note Taking, Annotation, Bookmarking

ACM Classification KeywordsH.5.2 User Interfaces: input devices and strategies; in-teraction styles; prototyping

INTRODUCTIONReading is an ubiquitous human activity that is increas-ingly done on-screen, whether using traditional PCsor speci�c reader devices such as the Amazon Kindle.It is therefore timely to examine and optimize the in-teraction design of digital reading. Previous ressearchdemonstrates numerous de�cits in screen reading com-pared to printed text, particularly for close, attentivereading that involves annotation, bookmarking and nav-igation to support the cognitively demanding task.

In the physical world, the act of note-taking requiresvery little conscious e�ort. The lightweight [5] prop-erties of paper, coupled with years of learned behav-ior, facilitate easy manipulation and use of paper-basedtools. The equivalent digital interactions do not o�erthe same a�ordances and su�er (comparatively) from

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, orrepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.CHI 2011, May 7–12, 2011, Vancouver, BC, Canada.Copyright 2011 ACM 978-1-4503-0267-8/11/05...$10.00.

poor rates of use [5, 6]. For example, when a note is nolonger required on a physical document, one can simplyremove and discard it without the comparatively labori-ous extra task of clicking and selecting delete. Anotherinstance is found in place-holding and note-taking: asingle piece of paper can be used to take notes and toserve as a bookmark (Figure 1), whereas digital readersoftware separates these two functions.

This paper presents a novel digital reading system thathas been designed to support the needs of users engagedin attentive reading. Our design acts like a real booksitting open on a desk, with digital features includingsearch. It uses a drag-and-drop interface to create asingle form of note that can be used as a bookmark orannotation. We used a large (desktop) screen, ratherthan a small (handheld) screen, to better study inter-action rather than device issues. This paper presentsthe main results from a larger comparative study.

BACKGROUNDResearch consistently demonstrates that current digi-tal note-taking tools su�er from poor interaction de-sign [5, 6, 8]. By focusing our attention on how usersinteract with physical documents we can gain a usefulgrounding for understanding the shortcomings of digitaldocument interaction. A diary study on work-relatedreading tasks concluded that reading occurs more of-ten in conjunction with writing than without [1]. Usersform a contextual understanding of the text, and var-ious forms of writing and drawing (e.g., underlining)support this activity with little cognitive attention be-ing paid to these secondary tasks. Hence tools withoutmarking support will have limited value in work-relatedreading activities. It is vital to the success of any digitalreader, then, that mark-up support is as e�ective andsubconscious in use as is the case in print.

Several researchers have explored the di�erences be-tween physical and digital reading. Marshall describesthe user's experience of using paper as lightweight [5]:�navigation that occurs [. . . ] in a way that is so unself-conscious that [users] aren't apt to remember it later.�Similarly, O'Hara et al [6] conclude that although digi-tal o�ers some bene�ts over paper (e.g., search, zoom),when reading for the purpose of writing, the bene�ts ofpaper outweigh those of digital documents. They con-clude that understanding the use of paper helps improvethe design of digital reading technologies. Our aim

then, is to arrive at a digital interaction where mark-uptools are transparent to the central active reading task[2]. We mirrored the interaction pattern of the physicalbook, in order to improve digital reading. Our inten-tion is not to slavishly replicate every aspect of physicalreading in our design, but to pick out key features andbuild upon them to eventually surpass the limitationsof the printed page. For example, to exactly copy thephysical world, digital Post-its would be of a �xed sizeand color, yet to make the system more e�ective we canallow users to alter such attributes, breaking the bookmetaphor and exceeding the restrictions of paper.

ProblemsPhysical documents o�er a degree of tangibility thatdigital counterparts cannot compete with. However,there are some areas of paper-based active reading [2]that are achievable but under-exploited by digital read-ers. There are three especially problematic areas:

Space: Previous work on annotation [7] shows that thespace surrounding a document (margins, desk, walls)performs an integral role in the mark-up process. Fig-ure 1 depicts a scenario where Post-its are placed bothdocument itself, and on the desk surrounding the docu-ment which is always visible. Most current digital doc-ument readers o�er no comparable space, forcing notesto be made in the documents themselves.

Tool Overload: On paper, one tool often has multiplefunctions [4]: so a ruler can be used to measure or todraw straight lines. Conversely, digital tools usuallysupport a single function, for example, bookmarkingduring reading. In physical documents, a page can bemarked simply by slipping in a scrap of paper: thisbookmark can also serve for note-taking by writing onit (Figure 1). Digitally, these functions would typicallybe distinct tools: bookmarking and note-taking.

Menu Navigation: Menus are generally displayedover the user's document, and require attention to nav-igate through their structure. Research suggests thatthis interrupts reading. In contrast, paper-based mark-up tools are much closer to direct manipulation, whichdistracts less attention from the text.

THE DESIGN OF A NEW READERPrior research demonstrates the high value of mark-up[1], yet the uptake on digital media is low. To increasethe use of digital mark-up, we designed a system, DigitalReading Desk, to bridge the gap between the physicaland digital domains by incorporating tools that behavein the same lightweight manner as paper. The DigitalReading Desk provides interaction patterns that mimicphysical tools. Drawing from physical books (Figure1), the PDF document on the desk is a double pagespread that opens in the same manner as paper books:two pages are seen when the book is open and a singlepage (front or back cover) when it is closed (Figure 2).This design creates a space around the book�a knownadvantage when reading in print [7]. We anticipated

Figure 1. Post-its being used in a physical book

that this presentation would encourage users to treatthem more like a physical book and consequently makebetter and more frequent use of the tools provided.

The Document’s Virtual Desk SpaceTo overcome the problem of restricted space for notes,we incorporated a virtual desk into our design. Thedesk acts as a backdrop for the page spread, and canbe used to make general notes or summaries about thedocument as a whole (the desk belongs to the documentso a new document generates a new desk).

The Unified Post-it ToolPaper is multi-functional: it can be used for notes or forplace-holding. To reproduce this, we amalgamate theroles into a single tool for �exibility and to reduce learn-ing. We adapt previous work on visual placeholders [3],where they protrude from the side of documents, liketabs in physical books. Our design allows users to cre-ate notes and bookmarks from the same object. Post-itscan be placed either: (1) completely on the document,(2) on the desk next to, or behind the document, or (3)protruding from the document (acting as a bookmark).This �exibility means that one tool performs three sep-arate functions: (1) to make notes about speci�c pages,(2) to make notes about the book as a whole, and (3) tomake notes that also act as placeholders. Post-its thatact as bookmarks navigate to the correct page whenclicked and, as in [3], `�ip' from one side of the bookto the other depending on which page is open. ThusPost-its that are before the current page, are on theleft, whereas those that are on pages that are after thecurrent page are on the right. See Figure 2.

Drag-and-DropTo reduce on-screen menu clutter, we use a drag-and-drop style interaction for the creation and deletion ofPost-its. On the right of the desk are three inexhaustiblepiles that can change color using the palette at the top.To create a Post-it, the user drags one from the pile ontothe document (Figure 2). Notes are removed by drag-ging them back onto the pile. As well as the additionof text to the Post-its, they can be moved, resized or`lifted up' (to reveal text underneath), borrowing fromand extending the behavior of physical notes. All inter-actions are performed without menus: to add text is adouble click, `lifting up' is a single click, etc. Removingthe need for menus promotes direct manipulation and

Post-its on desk (these will not be

visible when the

book is opened)

Post-its acting as bookmarks on pages after the current page

Post-it on desk (this will always be visible)

Post-it piles

Post-it bookmark on current

page

Post-it with text note on

PDF itself

Post-it on desk (this is only visible when the book is closed)

Color palette which

changes the color

of the Post-it piles

Post-its acting as

bookmarks on pages before the

current page

Post-it that has been shrunk to

reveal original

text under-neath

Figure 2. The Reading Desk System

reduces the cognitive attention required for the features,leaving more cognitive resource for the primary task.

EVALUATIONWe conducted a user study comparing our implementa-tion with two other designs created speci�cally as con-trols for the main features of the system. The �rst con-trol system was created to evaluate the uni�ed Post-ittool as well as the drag-and-drop interface. This system,�Traditional Desk,� consisted of the document and deskarea described above, but instead of the uni�ed Post-itsfor bookmarking and note-taking, it contained two dis-tinct tools to mimic established digital annotation andplaceholding. To evaluate the additional `desk' space, asecond control was used, �Traditional PDF,� that con-tained the same bookmarking and note-taking featuresas the �rst control, but instead contained no additionaldesk space surrounding the document area. We did notcompare our design to existing systems such as Adobeas doing so would add many variables to the experi-ment: e.g., previous experience with a system (learninge�ects or bias); di�erent function sets and implementa-tions may cause e�ects (e.g., rendering speed). We didhowever, base our two control interfaces on the methodsand tools commonly seen in existing systems (e.g., the

0

3

6

9

12

15

How do you compare the readingdesk annotation system against traditional annotation?

How do you compare the readingdesk bookmark system against traditional digital bookmarking?

Reading desk

is much

better

Reading desk is

slightly

better

They are

both th

e same

Traditional is

slightly

better

Traditional is

much bette

r

Part

icip

ants

Figure 3. Subjective User Reviews

tree list bookmark structure is directly from Acrobat).

We recruited 16 participants (12M 4F, ages 21-41) totake part in the study who we randomly selected froma cohort of computer science research sta� and post-graduate students. This group represents skilled userswho regularly take notes on documents, and would betypical users of digital texts. The structure of the studyconsisted of a short introduction to the three experi-mental systems, followed by a series of open and closedtasks on each interface, and �nally, a questionnaire andsemi-structured interview. A typical open task is �readthe document and mark-up the article with argumentsfor and against the topic. Summarize your �ndings�.We used three PDFs which we rotated amongst the tasksets and systems to ensure that the results are a productof the interface, rather than the task. The study wastherefore, run in a single blinded and counterbalancedmanner, as we also ensured that the order in which ourparticipants used the three interfaces was varied in se-quence across the six available combinations.

Desk: The post-study questionnaire included a set of5 point Likert subjective scores covering all aspects ofthe systems. One question asked how easy users foundit to create summaries for books, giving values of 2.81,4.38 and 4.63 for Traditional PDF, Traditional Desk,and Reading Desk systems respectively (Kruskal Wallisp < 0.001). This suggests that the desk area included inboth the Traditional Desk and Reading Desk systemsbut not in the Traditional PDF is useful for creatingnotes for the document as a whole. This was substanti-ated by the observed behavior of the participants whenengaged in open tasks; where the desk is available, usersmake use of it for document summaries 72% of the time.

Uni�ed Post-it Tool: One question in the post-studyquestionnaire asked how easy they found the tools touse. Average results were 3.5 and 3.81 for the Tradi-tional PDF and Traditional Desk systems, and 4.56 forthe Reading Desk (signi�cant di�erences, Kruskal Wal-lis p < 0.003). We also asked each participant how theycompared traditional annotation and bookmarking (i.e.,two separate tools) to the uni�ed Post-it tool found inthe Reading Desk system; see Figure 3.

Drag-and-Drop: The drag-and-drop portion of thesystem is concerned primarily with the creation and

deletion of Post-its and can be linked to the �ndingsshown in Figure 3. Speci�cally, 78% of participants feltthat the uni�ed drag-and-drop Post-it tool was slightlyor much better than traditional methods for annota-tion and bookmarking. Although the results for thisportion of the study are not signi�cant, qualitative com-ments such as �drag-and-drop is much the same as pa-per; chucking a note back on the pile is like screwing itup and throwing it away� and �dragging notes from pilesis simpler and more intuitive than �ddling with menusas this breaks me out of my train of thought� suggestthat drag-and-drop is preferred by participants.

Discussion of ResultsThe results of the study con�rm the e�ectiveness of thetechniques employed by the Reading Desk system. Thevirtual `desk' area, intended to overcome the issue ofspace, proved to be both popular with and well used bythe majority of participants. The behavioral and statis-tical data shows that the desk is the preferred positionfor document summaries. There is also support fromthe post-study interviews: �I always make summariesseparately so being able to put them on the desk is niceas you can always see it and make notes as you go.�

Secondly, uni�ed Post-its are more popular than sep-arate tools; there were comments such as �it [uni�edPost-its] makes life easier. In real life you wouldn't havea pile of Post-its and a pile of bookmarks�you wouldjust use a Post-it for both.� This view is also con�rmedby observed participant behavior during the compara-tive study. Interestingly, the use of bookmarks di�erssigni�cantly between the systems: 75% of participantsused them on the Reading Desk, while only 25% and19% used them on the Traditional PDF and Desk sys-tems respectively. This variation in bookmark usagesuggests that the uni�ed Post-it tool encourages theiruse; a χ2-test on the data produced a signi�cant result:χ2 = 7.684 (df = 2, p = 0.02145).

Although the Likert scale data comparing the drag-and-drop interface to traditional creation-and-deletiontechniques produced statistically insigni�cant results,our participants made comments such as �dragging anddropping notes is more intuitive, why have menus whena picture [piles of Post-its] is better?�

Finally, we asked the participants if they would usethese systems out of choice. Likert results from this por-tion of the study produced highly signi�cant (KruskalWallis p < 0.001) average results of 2.5 and 2.81 forthe Traditional PDF and Traditional Desk systems, andsigni�cantly larger 4.06 for the Reading Desk. Thoughthe Reading Desk mirrors some properties of paper, itavoids many natural limitations, including: the Post-itsare unlimited, can be resized or changed in color. Com-ments such as �the Reading Desk is better than paperbecause the notes always stick to the sides of the pagesand don't damage the book� and �you can't lose elec-tronic notes and you never have to buy more becausethe piles are in�nite� con�rm the quantitative �ndings.

CONCLUSIONSWe have presented a novel drag-and-drop Post-it systemfor digital note-taking which supports active readingtasks. The system provides a lightweight interface forannotating documents with a minimum number of in-terface tools. Rather than using extensive sets of menusand dialogs, we exploit direct manipulation and an ex-tend the user's workspace to minimize the interactione�ort for attentive reading. A comparison study on thenew interface indicates that by designing for appropri-ation [4], we were able to create an e�ective and noveluni�ed tool for placeholding and annotation that actu-ally encourages placeholder uptake. We also concludedthat an additional workspace surrounding the documentarea is a bene�cial component for the creation of notesand summaries and that a drag-and-drop interactionincreased user satisfaction of the interface.

While this paper evaluates the use of digital note-takingtools on desktops, a useful area of future work would in-clude the exploration of the design on smaller screen de-vices such as eReaders, not least because of their currentpopularity with consumers. As smaller screens pose po-tential space trade-o�s, it would be a useful area of fu-ture investigation.

ACKNOWLEDGMENTSThis work is supported by Microsoft Research Lab Cam-bridge and EPSRC, Grant EP/F041217.

REFERENCES1. A. Adler, A. Gujar, B. L. Harrison, K. O'Hara, and

A. Sellen. A diary study of work-related reading:design implications for digital reading devices. InCHI'98, pages 241�248. ACM, 1998.

2. M. J. Adler and C. Doren. How to Read a Book:The Art of Getting a Liberal Education. Simon andSchuster, 1972.

3. G. Buchanan and J. Pearson. Improvingplaceholders in digital documents. In ECDL'08,volume 5173 of LNCS, pages 1�12. Springer, 2008.

4. A. Dix. Designing for appropriation. In HCI'07,pages 27�30. BCS, 2007.

5. C. C. Marshall and S. Bly. Turning the page onnavigation. In JCDL'05, pages 225�234. ACM,2005.

6. K. O'Hara and A. Sellen. A comparison of readingpaper and on-line documents. In CHI'97, pages335�342. ACM, 1997.

7. J. Pearson, G. Buchanan, and H. W. Thimbleby.Improving annotations in digital documents. InECDL, volume 5714 of LNCS, pages 429�432.Springer, 2009.

8. B. N. Schilit, G. Golovchinsky, and M. N. Price.Beyond paper: supporting active reading with freeform digital ink annotations. In CHI '98, pages249�256, New York, USA, 1998. ACM Press.