Upload
others
View
10
Download
0
Embed Size (px)
Citation preview
Moving tangible interaction systems to the next level
Citation for published version (APA):Hoven, van den, E. A. W. H., Garde - Perik, van de, E. M., Offermans, S. A. M., van Boerdonk, K., & Lenssen, K.M. H. (2013). Moving tangible interaction systems to the next level. Computer, 46(8), 70-76.https://doi.org/10.1109/MC.2012.360
DOI:10.1109/MC.2012.360
Document status and date:Published: 01/01/2013
Document Version:Accepted manuscript including changes made at the peer-review stage
Please check the document version of this publication:
• A submitted manuscript is the version of the article upon submission and before peer-review. There can beimportant differences between the submitted version and the official published version of record. Peopleinterested in the research are advised to contact the author for the final version of the publication, or visit theDOI to the publisher's website.• The final author version and the galley proof are versions of the publication after peer review.• The final published version features the final layout of the paper including the volume, issue and pagenumbers.Link to publication
General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.
• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal.
If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, pleasefollow below link for the End User Agreement:www.tue.nl/taverne
Take down policyIf you believe that this document breaches copyright please contact us at:[email protected] details and we will investigate your claim.
Download date: 05. Sep. 2020
1
Reflecting on the Foundations and Qualities
of Tangible Interaction Elise van den Hoven1, Evelien van de Garde-Perik1, Serge Offermans1,
Koen van Boerdonk1, Kars-Michiel Lenssen2
1Industrial Design Department
Eindhoven University of Technology {e.v.d.hoven, e.m.v.d.garde, s.a.m.offermans}@tue.nl 2Philips Group Innovation, Research
High Tech Campus, the Netherlands [email protected]
Abstract
In the past decade, the field of Tangible Interaction (TI) has gained significant interest. As a result,
numerous systems, theories and frameworks have been developed with this vision in mind. This has
led to various instantiations of TI that seem developed to make digital information tangible, rather than
to optimally use and combine all important qualities of TI. We believe that TI has more to offer than
what has been used advantageously so far. Therefore, this paper reflects on the foundations of TI and
identifies three qualities of control and representation in TI based on existing systems, theories and
frameworks.
Keywords
Tangible Interaction, Foundations, Qualities, Interaction Design
Introduction
The interaction with computers, or the field of human-computer interaction (HCI), has evolved
continuously since the Graphical User Interface (GUI) started to settle in with computer users about
two decades ago. GUIs rely on graphical representation and "see, point and click" interaction, whereas
its predecessor, the Command User Interface (CUI) required the user to "remember and type"
characters (Ishii, 2008). While the interaction improved with GUIs compared to CUIs, the interactions
Digital Object Indentifier 10.1109/MC.2012.360 0018-9162/$26.00 2012 IEEE
This article has been accepted for publication in Computer but has not yet been fully edited.Some content may change prior to final publication.
2
with pixels on GUI screens appeared inconsistent compared to our interactions with the rest of our
physical environment (Ishii, 2008). Therefore, a desire emerged to provide a means of interacting with
digital data and computational power in a way that would bring these elements into our physical world.
The new class of systems would allow us to interact with computers in a more natural fashion and
would provide the next step after the Graphical User Interface.
These efforts have led to the Graspable User Interface (Fitzmaurice et al., 1995), subsequently to
the Tangible User Interface (Ishii & Ullmer, 1997; Ullmer & Ishii, 2000) and finally resulted in the
emerging field of Tangible Interaction which is the current term for work regarding this type of
interfaces. In general, one can say that Tangible Interaction distinguishes itself from the display-based
user interaction, by embracing physicality, materiality and embodiment. Typically the resulting user
interfaces consist of physical objects that can be manipulated as a means of interacting with a
computing system. Originally Graspable User Interfaces were defined to ‘allow direct control of
electronic or virtual objects through physical handles for control’ (Fitzmaurice et al., 1995, p.442).
Over the years the field has changed and moved from an engineering perspective to a wider field and,
lacking one generally accepted definition, instead several new ones arose (see e.g. Shaer & Hornecker,
2010). A common definition defines it as the interaction between human beings and computational
systems using physical objects (Ullmer & Ishii, 2000). Numerous examples have been shown and
theories and frameworks were built upon them (for a framework overview see, Mazalek & Van den
Hoven, 2009). This has not yet led to a coherent and generally accepted idea of what the community
considers TI to be. Is using a computer mouse Tangible Interaction? It is physical, but is also used for
controlling a GUI. Is interacting with your full body without any physical controller Tangible
Interaction? Your body is physical and it does leverage your capabilities as a human which could not
be reached in a CUI or GUI, but there is no physical controller beyond the human body. And what
about gestures used on touch screens? A binary distinction between Tangible and non-Tangible
Interaction is in our view not too interesting, instead we believe it is more important that the qualities
that Tangible Interaction has to offer are exploited to improve our interaction with the digital world.
This is where our paper differs from the various frameworks and taxonomies mentioned before. We do
not intend to provide a coherent framework or taxonomy for TI, but rather aim to stimulate the creation
of TI systems for their potential benefits. In order to implement these benefits we believe it is
important to understand that the foundations that are at the base of these qualities. There are good
examples of systems in which Tangible Interaction clearly adds value to the interaction, but we feel
that there are also many TI systems that are developed for the sake of tangibility without benefitting
(optimally) from what TI has to offer. We will use the indication TI to refer to the entire field of
Tangible Interaction with all its related definitions in order to cover all aspects of the field, and a
system that relies on insights from the field of Tangible Interaction will be referred to as TI system.
In this paper, we reflect on the assumed TI qualities and the foundations underlying TI (see figure
1), based on an analysis of the characteristics or advantages of TI mentioned in literature. The
foundations can be considered elements that are, at least to some extent, required for a system to be
Digital Object Indentifier 10.1109/MC.2012.360 0018-9162/$26.00 2012 IEEE
This article has been accepted for publication in Computer but has not yet been fully edited.Some content may change prior to final publication.
3
considered TI. The qualities on the other hand combine aspects of these foundations such that they
provide benefits to the user interaction with the system. These qualities relate to the mechanisms of
control and representation of digital data. With this overview we hope to trigger more people to
consciously develop TI systems for exploiting its qualities, rather than just for making the digital
tangible.
Figure 1. Benefit from the Qualities of Tangible Interaction through combining the Foundations
carefully
First we will explain the three foundations of tangible interaction: computation, human skills and
the physical world (in the Section about Foundations). Second, we will reflect on the potential qualities
that a proper combination of the foundations may offer in the interaction (in the Section about
Designing). Finally, we will provide a discussion and conclusion on the use and value of the proposed
approach.
Foundations of Tangible Interaction
From the literature study we identified the following three types of foundations: Computation, Human
Skills, and the Physical World (see figure 1, top). Each of these foundations will be discussed and
detailed below.
Digital Object Indentifier 10.1109/MC.2012.360 0018-9162/$26.00 2012 IEEE
This article has been accepted for publication in Computer but has not yet been fully edited.Some content may change prior to final publication.
4
Computation Foundation
Computation allows the execution of rapid calculations, generation of precise output, control of
multiple complex systems in general (Bellotti et al., 2002), and taking over repetitive tasks of users. It
may function as extended memory for human skills and has the potential to make the physical world
more dynamic.
Although the use of computation is widely agreed upon as a characteristic of TI, the extent of this
computation varies greatly. Simple computations are often used as a means to achieve a one-to-one
relationship between the user’s input and the system’s response, such as inTouch (Bellotti et al., 2002),
which uses computation only to literally copy input and output.
Slightly more complex systems use computation to control a state change and its effects such as
SpeakCup (Zigelbaum et al., 2008), where physical action controls a recording/playback device. Some
more complexity can be found in the SMS Slingshot (Fischer et al., 2010), which uses analysis of
physical motion to control the aim of a digital slingshot shooting a short message.
More complex computation exists in systems that have a mathematical model behind it. Ullmer and
Ishii define the use of a computational model as the most profound characteristic of TI (Ullmer & Ishii,
2000). A ‘computational model’ suggests the use of such a complex mathematical model that is able to
analyze the behavior of a complex system. Complexity often rises with the amount of parameters
involved and their interdependency on each other. Some of the typical TI examples are based on a
computational model such as ‘Urp’ (Underkoffler & Ishii, 1999). This system calculates airflow,
shadows and light reflections around buildings in real time based on the positioning of scale model
buildings and projects these on a tabletop. A more recent TI example with a mathematical model is for
instance the educational tabletop application ‘physics of light’ (Price et al., 2009).
Nowadays it is relatively easy to create more complex systems because of the vast amount of
information that is digitally available and fast growing computational power. Therefore, besides the
use of local data (e.g. from sensors), the use of external data is gaining importance, especially
regarding Internet usage. Digital data is nowadays abundantly available and can be used freely in user
interfaces. This enables TI designers to use distributed sensor data and information provided by people
across the globe, which is a relatively new area within the field.
In contrast to the foundations ‘human skills’ and ‘physical world’, the use of computational
qualities is most prominently visible in the system’s functionality, rather than in the user’s interaction
with the system. Nevertheless, it is an important differentiator as it distinguishes TI from other forms
of physical (non-digital) interaction, and enables the link between the physical and digital world.
Summarizing, all TI systems have some form of computation, ranging from just the processing of
sensor data, to complex computational models and the use of external data. Regardless of the
complexity of this computation, it allows us to interact with systems in a way that could not have been
achieved without it and is therefore a foundation of TI.
Digital Object Indentifier 10.1109/MC.2012.360 0018-9162/$26.00 2012 IEEE
This article has been accepted for publication in Computer but has not yet been fully edited.Some content may change prior to final publication.
5
Human Skills Foundation
The second foundation of TI is the broad range of innate and acquired skills that people possess.
Djajadiningrat identifies three different skills, which we adhere to in this paper, namely cognitive,
perceptual-motor and emotional skills (Djajadiningrat et al., 2004). Cognitive skills are our mental
abilities to read, interpret, and remember. Perceptual-motor skills concerns our ability to perceive with
our senses and to act physically. Emotional skills include our ability to experience, express and
recognize emotions (Djajadiningrat et al., 2004).
TI can be seen as a reaction to the traditional Human Computer Interaction (HCI) paradigm in
which interfaces such as WIMP (window, icon, menu, pointing) mainly relied on cognitive skills and
made very limited use of our other humans skills. Consequently, in TI there was a desire to exploit
other human skills as well, providing close resemblance to our interaction with the physical world. An
advantage of Graspable User Interfaces is that they ‘leverage off of our well-developed everyday skills
of prehensile behavior for object manipulation’ (Fitzmaurice et al., 1995).
The fact that TI makes use of human capabilities is not only advantageous for object manipulation
(input), but also for the system’s response (output). Ideally, in TI output is given in a physical form as
well, taking benefit of the combination of multiple senses, whereas GUIs mainly address the visual
sense. Our ability to sense and act across multiple modalities is exploited more in TI and affects input
as well as output.
While the use of emotional skills is still underdeveloped in TI, the literature provides various
examples of perceptual-motor and cognitive skills that TI takes advantage of. For example TI enables
two-handed interaction, parallel control of multiple parameters and exploits our spatial reasoning skills
(Fitzmaurice et al., 1995). Our spatial reasoning skills are better supported by TI systems than by
GUIs, because of the use of physical space for interaction. TI also has the ability to use both the
observational as well as action periphery exploiting our entire attention field as described by Edge and
Blackwell (2009) and Bakker et al. (2010). Furthermore, TI can provide information at a glance by
using abstract information (e.g. color, shape of an object) instead of concrete information (e.g. text,
numbers).
Concluding, TI offers much more opportunities for exploitation of human skills than GUI’s. The
use of these human skills allows for different interactions that suit the application depending on
desired efficiency, fun, usability, expressiveness, etc. Additionally, it will allow people to interact
more naturally with a system, in the sense that it is similar to the way we interact with our everyday
physical environment and therefore is familiar. The nature of this physical environment, the third
foundation we identified, will be explicated next.
Physical World Foundation
TI systems are characterized by their physical nature, in contrast to screen-based systems. Inherently,
they have the opportunity to use advantages of the physical world. This has been a central component
Digital Object Indentifier 10.1109/MC.2012.360 0018-9162/$26.00 2012 IEEE
This article has been accepted for publication in Computer but has not yet been fully edited.Some content may change prior to final publication.
6
in TI from the early beginning, as the emergence of TI is largely a response to the lost physical
component in GUIs (Ishii, 2008).
With the loss of the physical component in GUIs more than solely physicality got lost, as GUIs
exploit only a limited amount of the human sensory system. The visual sense is most dominant in this
paradigm (hence the Graphical in GUI), and the use of additional senses is rare. The physical world on
the other hand, explicitly uses the tactile sense, and also allows a more natural combination of multiple
sensory input and output. Any physical object can provide us with information through multiple
sensory channels. The multimodality of tangible interfaces results in a broader spectrum of
communication channels with more bandwidth.
In contrast to the virtual world, the physical world is persistent. This has advantages as objects
remain visible when you want them to. The objects are always there and therefore accessible for
interaction. Persistence can also be a disadvantage, as you cannot make objects vanish and they need
physical storage space when they are not being used. Physicality also provides affordances which are
described as the ‘actionable properties between the world and an actor’ (Gibson, 1986). Affordances
may help the user in determining how to interact with an object. A physical object may provide cues
about whether it can be squeezed or pushed, whether it is light or heavy and needs handling with one
or two hands.
The main advantage of the physical world is that we have a lot of experience with it. We can easily
comprehend the physical world as we are ourselves physical beings and have lived with the physical
world ever since our birth. We can interpret spatial location and orientation of a physical object, and
we are very well capable of understanding and manipulating objects. This again shows the interrelation
of the three types of foundations. Summarizing, using the physical world in TI may allow us to think
about and interact with the digital world using our understanding of the physical world. However,
there is no evidence for this yet (Blackwell et al., 2007; Hornecker, 2012).
Designing Tangible Interaction exploiting the Foundations
All three foundations are seen as inevitable components of a TI system. Designing systems that
combine the characteristics of these three foundations will result in qualities for interaction with TI
systems that can be advantageous over other forms of interactions. The term ‘quality’ in this paper is
used to describe a characteristic or feature (based on the definition by Cambridge Dictionary) of TI
systems. We propose three different qualities of the interaction with TI systems, that are related to the
control and representation of digital data, see also figure 1:
1. Direct control and representation of digital data;
2. Integrated control and representation of digital data;
3. Meaningful control and representation of digital data.
Directness (1) allows the user to control and retrieve digital data directly without any intermediate
steps; control and representation is immediately accessible. Integration (2) allows the user to observe
Digital Object Indentifier 10.1109/MC.2012.360 0018-9162/$26.00 2012 IEEE
This article has been accepted for publication in Computer but has not yet been fully edited.Some content may change prior to final publication.
7
the input and output of a system as a single thing. Meaningfulness (3) allows the user to understand the
possible forms of input and to correctly interpret the output of a system.
Each quality is based upon all three foundations. It is precisely through a combination of the
foundations and the actual qualities included in design that creates the real value of a TI system and its
resulting interaction experience. The extent to which the foundations and qualities are present
determines to what extent the system benefits from TI, but does not imply the overall quality of the
system. The three qualities related to interaction with TI systems will be elaborated on next.
Direct Control and Representation of Digital Data
The direct availability of information (representation) and the possibility to directly manipulate
parameters of the system (control) has been a central theme in TI. The concept of direct manipulation
has been proposed for the use in GUIs in an early stage (Shneiderman, 1997). Nevertheless, GUIs can
only do so much to provide direct control as the input is given through generic, time multiplexed
devices (Fitzmaurice et al., 1995) (i.e. one device is used to control different functions at different
points in time). In the early days, making interface elements more ‘direct’ and ‘manipulable’ was
described by Fitzmaurice et al. as one of the advantages of a Graspable UI (Fitzmaurice et al., 1995).
This means that no menus or other intermediate steps have to be taken to manipulate something, but
rather that the data are immediately manipulated through a physical object. According to Ishii, TI
provides physical form to digital information and computation, facilitating direct manipulation of data
(Ishii, 2008).
The immediate availability of the information representation allows people, besides manipulation,
to get information from the system at any time. It allows awareness of something at a glance.
Furthermore, direct availability of digital data allows for exploration. One can immediately see the
results of the actions performed. Also, direct control and representation of digital data facilitates
collaborative use of the system, because the actions of one user (and results thereof) are immediately
visible to the other users. Furthermore, because the digital data are visualized in a physical way (Ishii,
2008), this may provide additional insights in complex processes.
This direct availability is, however, always a trade-off, since whenever something is directly
available there is a risk of clutter. Secondly, direct availability impliesthat for systems with many
controllable parameters, many objects may be required as an object can only control as many
parameters as it has degrees of freedom. An advantage however is that each object can be designed
according to its function to increase understanding by the user. Good examples of direct control and
representation are the Reactable (Jordà et al., 2007) and AppMates (http://www.appmatestoys.com/),
as all the functional tokens/cars are directly available to use and provide immediate visual and auditory
feedback once a token is placed, see table 1.
Digital Object Indentifier 10.1109/MC.2012.360 0018-9162/$26.00 2012 IEEE
This article has been accepted for publication in Computer but has not yet been fully edited.Some content may change prior to final publication.
8
Example 1 Example 2
Direct
Reactable (Jordà et al., 2007)
AppMates©,
http://www.appmatestoys.com/
Integrated
Urp (Underkoffler & Ishii, 1999)
Rich Interaction (Frens, 2006)
Meaningful
Physical props (Hinckley et al., 1994)
GraffitiWall
(http://tangibleinteraction.com/chanel-
dgw)
Table 1. Overview of good examples for each of the three qualities: direct, integrated and meaningful
control and representation of digital data.
Digital Object Indentifier 10.1109/MC.2012.360 0018-9162/$26.00 2012 IEEE
This article has been accepted for publication in Computer but has not yet been fully edited.Some content may change prior to final publication.
9
Recapitulating, computers and the Internet have brought us a large amount of newly available
functionalities. As many of these functions are provided digitally through generic GUI based
interaction devices such as a (tablet-)PC or phone; it is often required to activate the device and start a
specific application to access the functionality. The ‘Direct control and representation of digital data’
quality redresses the effort-reward balance for this kind of functionalities. TI therefore has the
potential to integrate these functionalities into our everyday life and allow us to benefit more from
them.
Integrated Control and Representation of Digital Data
A major difference between interaction with our everyday physical objects and interaction with
computational systems is the location of the action by the user and the reaction by the system. For non-
digital physical objects, this is usually the same location as the only change in the object is caused by
direct physical manipulation by the user. With the personal computer, the digital effects of physical
actions (e.g. mouse movements) are traditionally presented at a different location (e.g. a screen).
Within TI, the aim is once again to unify action and perception in order to create a ‘seamless
integration of representation and control’, as is stated by Ullmer and Ishii (2000). This means that to
the user there is no longer a separation between the two, and that the system state can be ‘read’ from
the manipulated object. In literature, various terms are used to describe this reintegration of
representation and control, such as: “joining input and output space” (Hurtienne & Israel, 2007).
This integration of input and output may take place to different extents. Fishkin (2004), for
example, distinguishes four different levels of embodiment. He describes the level of embodiment as
the degree to which the input mechanism is closely tied to the output event, or to what extent the user
thinks the computation is inside the input object itself. Price et al., on the other hand, refer to actual
location on three different levels. The digital representation may be located separately from the input
device (Discrete), in a contiguous fashion (Co-located), or it may happen within the object
(Embedded) (Price et al., 2009).
We consider it most important that the action and perception space are integrated. The user has to
be able to perceive both her/his own action and the system output simultaneously. A lot of tabletop
systems work like this as they project the system output onto the physical objects. Going one step
further, integration can take place on a physical level. This means that control and representation are
physically integrated in the same object. At this level of integration, the user may feel that control and
representation are truly united.
Good illustrations of integrated control and representation of digital data is are Urp and the camera
of Frens, see table 1 (Underkoffler & Ishii, 1999; Frens, 2006). Urp leverages the integration as the
shadow falls naturally on the buildings and gives the feeling as if it were a natural shadow that was
casted by the building itself. The rich interaction camera of Frens integrates form, interaction and
function. During the interaction with the camera it is physically manipulated in such way that it
communicates the action possibilities as well as the status of the device.
Digital Object Indentifier 10.1109/MC.2012.360 0018-9162/$26.00 2012 IEEE
This article has been accepted for publication in Computer but has not yet been fully edited.Some content may change prior to final publication.
10
Summarizing, integration of control and representation provides the opportunities for interaction
that is as natural as with everyday physical objects, while simultaneously benefitting from the
advantages of the digital world.
Meaningful Control and Representation of Digital Data
The third type of quality concerns one of the challenges of TI design, which is how to map physical
objects and their manipulation to digital computation and feedback in a meaningful and
comprehensive manner (Ishii, 2008). Along the same lines, Hurtienne and Israel (2007) propose to
focus on subjectively meaningful patterns to create intuitive design for tangible interfaces.
TI systems are considered ‘strong specific’ interfaces rather than ‘weak general’ interfaces. They
are developed for a specific purpose, and therefore their appearance and interaction can be tailored
towards a specific function. This allows TI to convey meaningful information about the control and
representation of digital data.
According to Djajadiningrat et al. meaningfulness in interactive products should be provided in
form as well as action. By designing action and appearance concurrently with function, a meaningful
relationship between appearance, action and function can be created. On the other hand, for simple
products with limited action and function options, such as taps or lights it may be sufficient to provide
solely information about the effective action through formgiving (i.e. affordances) (Djajadiningrat et
al., 2004).
Metaphors are a possible way to provide meaningfulness in form and interaction. According to
Fishkin metaphors in the context of TI imply that the system effect of a user action is analogous to the
real-world effect of similar actions (Fishkin, 2004). Hurtienne and Israel (2007) claim that metaphors
do not imply mimicking the real world as closely as possible. Instead, they use the term metaphor to
indicate the transfer of schematic structures of real world phenomena to abstract concepts in thought
(i.e. cognitive phenomena). Lakoff and Johnson (1980), state that metaphors allow us to understand
things in terms of another, and that this is fundamental to our entire understanding of the world. We
believe that metaphors can be used to understand the interaction, but that this does not necessarily
imply that a product resembles another product. A possible way to achieve this is through image
schemas. Image schemas are abstract representations of recurring dynamic patterns of bodily
interactions that structure our understanding of the world (Hurtienne & Israel, 2007).
Hurtienne and Israel (2007), state that an interface is intuitive if pre-existing knowledge of the user
leads to effective interaction. The pre-existing knowledge may originate from different sources, and
may be classified along a continuum from innate knowledge, perceptual-motor knowledge, knowledge
specific to one’s culture, to expertise knowledge. Lakoff and Johnson (1980), state that “no metaphor
can ever be comprehended or adequately represented independently of its experiential basis”. The
meaningfulness of interactive products is thus a result of previous experiences that generated
knowledge at different levels.
Digital Object Indentifier 10.1109/MC.2012.360 0018-9162/$26.00 2012 IEEE
This article has been accepted for publication in Computer but has not yet been fully edited.Some content may change prior to final publication.
11
The Tangible Interaction quality meaningful control and representation of digital data is illustrated
by two interaction examples in Table 1. Physical props is meaningful as it allows users to see different
digital section cuts, based on the physical action of making a section cut using an object and a section
plane. (Hinckley et al., 1994). Graffity Wall (http://tangibleinteraction.com/chanel-dgw) leverages
meaningfulness by giving the controller the shape of a spray can, co-locating the controller input and
output and mimicking the action of actual spraying.
Summarizing, meaningful interaction with TI systems is an important quality as it provides the user
with an understanding of the interaction. Concepts aiding this meaningfulness include affordances,
metaphors and image schemas.
Discussion
The field of Tangible Interaction is concerned with the interaction between human beings and
computational systems using physical objects that function as representation and control of digital data
(Ullmer & Ishii, 2000). The foundations and qualities described in this paper provide a handle for
defining and designing Tangible Interaction. We believe that developers and designers who are aware
of the TI foundations and subsequently apply the qualities in a system design will contribute to the
tangible experience of interactive systems.
We believe the HCI direction of Tangible Interaction will have an impact on computer science in
the near future in several ways. The computer will in many cases no longer be perceived as one single
device and its controls, but a collection of smaller devices that combine input and output and are
spread around spaces and bodies. There will be more freedom in designing these devices and therefore
the form factor will be more flexible and might distract from the computing functionality despite the
embedded computation. The trend of adding intelligence to everyday objects will contribute to this
development. Computer science will become even more important, however not in the traditional form
of keyboard, mouse and screen.
Using the foundations and qualities of TI in actual systems, requires a close collaboration between
the experts in the fields related to all TI foundations. The expertise of ICT professionals, psychologists
and product designers is crucial when developing systems that aim to benefit from Tangible
Interaction. Therefore developing computing systems will no longer be a task that can be done by
engineers alone, which will inherently change development processes and approaches.
Although the use of TI foundations and qualities is considered a positive contribution to the design;
implementing them is not a guarantee for a good system. Depending on the application and its use,
some qualities may be more desirable than others. Therefore, not all qualities have to be equally
present in a TI system, instead they should be selected consciously in order for the interaction to profit
optimally.
To date, not many TI systems with a high degree of complexity have successfully made it to the
market. We believe the reason for this to be that the focus has been too much on designing complete TI
systems, rather than exploiting TI benefits. We believe designing such systems may benefit from an
Digital Object Indentifier 10.1109/MC.2012.360 0018-9162/$26.00 2012 IEEE
This article has been accepted for publication in Computer but has not yet been fully edited.Some content may change prior to final publication.
12
approach that allows designers to create hybrid systems where the user interaction is in part tangible.
Interface elements that for example target novice users, have a high frequency of use, or high degree of
urgency may benefit more from TI qualities. As many everyday systems require a certain complexity,
we should not be afraid to use the best of both worlds.
Conclusions
This paper identifies a set of three foundations of tangible interaction (TI) from an analysis of the
literature: computation, human skills, and the physical world. The combination of these three
foundations in design provides TI systems with the following qualities: direct, integrated, and
meaningful control and representation of digital data. The identification of these foundations and
qualities of TI serves two purposes:
1) Analysis of the extent to which a system exploits the advantages of TI. The foundations and
qualities of TI presented in this paper can be used to compare and characterize different TI systems.
2) Development and design of TI systems for everyday use. We believe that being more aware of
the foundations and qualities will help TI to transfer from the labs into the real world. The focus
should be on usefully combining the three foundations of TI and creating the qualities of TI in order to
exploit the advantages of TI systems, instead of merely making a digital system tangible.
By exploiting the advantages of computation, human skills, and the physical world and making use
of the qualities direct, integrated, and meaningful control and representation of digital data new TI
systems can bring even more benefits to the user than with existing TI systems.
Acknowledgements
This work was funded by the Dutch Government via grant no. KWR 09072 of Agentschap NL. We
also like to thank the reviewers and guest editors who provided helpful comments on previous versions
of this document.
References
Bakker, S., Hoven, E. van den and Eggen, B. Design for the Periphery. Proc. Eurohaptics 2010,
(2010), 71-80.
Bellotti, V., Back, M., Edwards, W.K., Grinter, R.E., Henderson, A., and Lopes, C. Making sense
of sensing systems: five questions for designers and researchers. Proc. CHI’02, (2002), 415-422.
Blackwell, A.F., Fitzmaurice, G., Holmquist, L.E., Ishii, H. and Ullmer, B. ‘Tangible user
interfaces in context and theory’, Proc. CHI’07 Extended Abstracts, (2007), 2817–2820.
Djajadiningrat, T., Wensveen, S., Frens, J., and Overbeeke, K. Tangible products: redressing the
balance between appearance and action. Personal Ubiquitous Computing, 8, 5 (2004), 294-309.
Edge, D. and Blackwell, A.F. Peripheral tangible interaction by analytic design. Proc. TEI’09,
(2009), 69-76.
Digital Object Indentifier 10.1109/MC.2012.360 0018-9162/$26.00 2012 IEEE
This article has been accepted for publication in Computer but has not yet been fully edited.Some content may change prior to final publication.
13
Fischer, P.T., Zöllner, C., Hoffmann, T., and Piatza, S. VR/Urban: SMSlingshot. Proc. TEI’10,
(2010), 381-382.
Fishkin, K.P. A taxonomy for and analysis of tangible interfaces. Personal and Ubiquitous
Computing, 8, 5 (2004), 347-358.
Fitzmaurice, G.W., Ishii, H., and Buxton, W.A.S. Bricks: laying the foundations for graspable user
interfaces. Proc. CHI’95, (1995), 442-449.
Frens, J.W. A rich user interface for a digital camera. Personal and Ubiquitous Computing, 10(2-3),
(2006), 177-180.
Gibson, J.J. The Ecological Approach To Visual Perception. Psychology Press (1986).
Hinckley, K., Pausch, R., Goble, J.C., & Kassell, N.F. (1994). A survey of design issues in spatial
input. Proc. UIST ’94, (1994), 213–222.
Hornecker, E. Beyond affordance: tangibles' hybrid nature. Proc. TEI’12, (2012), 175-182.
Hurtienne, J. and Israel, J.H. Image schemas and their metaphorical extensions: intuitive patterns
for tangible interaction. Proc. TEI’07, (2007), 127-134.
Ishii, H. and Ullmer, B. Tangible bits: towards seamless interfaces between people, bits and atoms.
Proc. CHI’97, (1997), 234-241.
Ishii, H. Tangible bits: beyond pixels. Proc. TEI’08, (2008), xv-xxv.
Jordà, S., Geiger, G., Alonso, M., and Kaltenbrunner, M. The reacTable: exploring the synergy
between live music performance and tabletop tangible interfaces. Proc. TEI’07, (2007), 139-146.
Lakoff, G. and Johnson, M. Metaphors We Live by. University Of Chicago Press, (1980).
Mazalek, A. and van den Hoven, E. Framing tangible interaction frameworks. Artif. Intell. Eng.
Des. Anal. Manuf. 23, 3 (2009), 225-235.
Price, S., Falcão, T.P., Sheridan, J.G., and Roussos, G. The effect of representation location on
interaction in a tangible learning environment. Proc. TEI’09, (2009), 85-92.
Shaer, O. and Hornecker, E. Tangible User Interfaces: Past, Present, and Future Directions.
Foundations and Trends in Human-Computer Interaction 3, (2010), 1–137.
Shneiderman, B. Direct manipulation for comprehensible, predictable and controllable user
interfaces. Proc. IUI’97, (1997), 33-39.
Ullmer, B. and Ishii, H. Emerging frameworks for tangible user interfaces. IBM Syst. J. 39, 3-4
(2000), 915-931.
Underkoffler, J. and Ishii, H. Urp: a luminous-tangible workbench for urban planning and design.
Proc. CHI’99, (1999), 386-393.
Zigelbaum, J., Chang, A., Gouldstone, J., Monzen, J.J., and Ishii, H. SpeakCup: simplicity, BABL,
and shape change. Proc. TEI’08, (2008), 145-146.
Digital Object Indentifier 10.1109/MC.2012.360 0018-9162/$26.00 2012 IEEE
This article has been accepted for publication in Computer but has not yet been fully edited.Some content may change prior to final publication.