What  Is  Interaction?  Kasper Hornbæk

University of Copenhagen

Antti Oulasvirta Aalto University

ABSTRACT  The term interaction is field-defining, yet surprisingly con-fused. This essay discusses what interaction is. We first ar-gue that only few attempts to directly define interaction ex-ist. Nevertheless, we extract from the literature distinct and highly developed concepts, for instance viewing interaction as dialogue, transmission, optimal behavior, embodiment, and tool use. Importantly, these concepts are associated with different scopes and ways of construing the causal re-lationships between the human and the computer. This af-fects their ability to inform empirical studies and design. Based on this discussion, we list desiderata for future work on interaction, emphasizing the need to improve scope and specificity, to better account for the effects and agency that computers have in interaction, and to generate strong prop-ositions about interaction.

Keywords  Interaction; human-computer interaction; scientific pro-gress; concepts; models; theories.

ACM  Classification  Keywords  H.5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous.

INTRODUCTION  The term interaction is a workhorse for our field. It is an el-ement of the names Human-Computer Interaction and In-teraction Design, as well as a multiplicity of topics and the-ories, such as reality-based interaction [45], seven stages of interaction [63], and embodied interaction [25]. In the CHI 2016 proceedings, the word occurs more than 4 500 times. The folk notion behind such uses seems to be that of two entities (computer, human) engaged in an interplay of sorts.

For such a field-defining concept, however, it is underde-fined. Open a textbook on HCI and you will likely look in vain for a definition. In Designing the User Interface, 4th ed. [82], entries on “interaction” concern interaction devic-es, styles, and systems; the first chapter of the textbook, In-teraction Design: Beyond Human-computer Interaction [73], is eloquent about interaction design but silent about

interaction per se. Turning to research papers, Beaudouin-Lafon [8] argued that HCI is “far from having solid (and falsifiable) theories of interaction” and Payne [67] lamented that “despite its name, the field of HCI has not devoted much research attention to the nature of interaction”. Re-cently, it has been suggested that interaction “hinges on an outmoded notion of technology in use” [85, p. 50] and that we need to move “beyond interaction” [88]. The confusion is complete. Perhaps this has contributed to HCI’s claimed lack of “motor themes, mainstream topics, and schools of thought” [53] and “low problem-solving capacity” [66].

What is interaction? This essay analyzes the concept from a pragmatic perspective of problem-solving [66]: Instead of asking what ‘the right’ definition of interaction is, our ulti-mate aim is to understand what kinds of theories and con-cepts of interaction are out there, and how they might be formulated so as to be useful for HCI. We ask how they construe interaction in terms of relations among humans and computers, and what this construal implies for their ca-pacity to inform decisions in design.

Why bother? We see three benefits. First and foremost, moving from everyday concepts to sharper, scientific con-cepts is key to all scholarship; we should be able to better define, measure, reason about, and predict interaction. Se-cond, how you understand interaction impacts your work by affecting your notion of what makes interaction good; it works as a thinking tool. However, we know of no previous account of the assumptions, uses, and limitations of this tool. Third, independently of the confusion around current views of interaction, we see it as the glue that ties our field together. Any clarity might help bring together the disparate disciplines and belief systems in HCI.

MISSING  IN  ACTION:  DEFINITIONS  OF  INTERACTION  We first argue that attempts to directly define interaction are either absent from the literature or stuck with folk no-tions. To do so, we survey some influential textbooks and handbooks. We then cover some old and a few newer dis-cussions of interaction. Finally, we discuss what is missing.

Folk  Notions  of  Interaction  Most textbooks (e.g., [73,77,82]) and handbooks (e.g., [39,44]) provide a sparse coverage of “interaction”. The books referenced have many entries about interaction, such as “interaction devices”, “interactive systems”, and “inter-action design”. For instance, The Handbook on Human-Computer Interaction [44] lists entries on interaction, span-ning modalities of interaction, over interaction design and children to interaction with networked systems. But no en-try provides a definition or high-level discussion of interac-tion. Interaction Design: Beyond Human-computer Interac-

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy oth-erwise, or republish, to post on servers or to redistribute to lists, re-quires prior specific permission and/or a fee. Request permissions from Permissions@acm.org. CHI 2017, May 06-11, 2017, Denver, CO, USA © 2017 ACM. ISBN 978-1-4503-4655-9/17/05…$15.00 DOI: http://dx.doi.org/10.1145/3025453.3025765

tion [73] contains one chapter on interaction, viewing it as instructing or conversation, but lacks a direct definition. Saffer [77] offers a more detailed definition than most:

“An interaction, grossly speaking, is a transaction be-tween two entities, typically an exchange of information, but it can also be an exchange of goods or services. […] It is this sort of exchange that interaction designers try to engender in their work. Interaction designers design for the possibility of interaction. The interaction itself takes place between people, machines, and systems, in a variety of combinations.” (p.4).

This definition is a step forward but raises a myriad of questions. Can only information be transmitted? What is the difference between a narrow (machines) and a broad (sys-tems) view of that we interact with? What about the experi-ence of transacting? What happens over time? To sum up, most of the field’s go-to books provide either folk notions of interaction or discuss theories and models without relat-ing them to a defined concept.

Ongoing  Discussions  of  Interaction  and  Interactivity  Several academics have echoed this sentiment. In a 1990 keynote at CHI, Winograd [90] asked, “What do we mean by human-computer interaction?” and continued, “Webster defines ‘interaction’ as ‘mutual or reciprocal action or in-fluence.’ Clearly, humans act on computers and computers influence humans. But how? In what dimensions?” (p. 444-445). Others argue similarly [67].

A few papers have provided in-depth discussions of interac-tion. Bødker and Kammersgaard distinguished four types of interaction—system, tool, dialogue partner, and media (cit-ed in [69]; see also [49])—and discussed the actors and ide-als involved in those types. Pike et al. [70] discussed the prospects and contents of a science of interaction, focusing on visualizations. Svanæs [84] reviewed the philosophy of interaction, contrasting the cognitive science view with those of Heidegger and Merleau-Ponty.

Instead of interaction, some work has discussed interactivi-ty [33,47,48]; this is a prominent theme in communication and advertisement research. In HCI, Janlert and Stolterman [47] reviewed the literature, concluding that agency, pre-dictability, and pace are important characteristics. They did not attempt to define interaction in such a way that it was compatible with these characteristics. Kirsh [52] also dis-cussed the concept of interactivity. He noted that many basic psychological processes are interactional and used this view to critique of plan-driven models of interaction, in particular of Norman’s seven-stage model. Overall, howev-er, most definitions of interactivity treat it as a quality of an interface and says little about the interplay with the human.

At the same time, the notion of interaction itself has been criticized. Taylor wrote that interaction “hinges on an out-moded notion of technology in use” [85, p. 50]. He argued that the boundary between things supposed to interact is too hard to locate and that interaction design is about world

making (and not just the interplay between people and ma-chines). Others have voiced similar concerns [88].

What  Is  Missing?  The work surveyed above shows some progress in critiqu-ing and refining the dictionary definition and folk notions of interaction. Yet, a lot is missing. First, as we argue in the following, we find that HCI already has rich views of inter-action, but they are tacit and intertwined in the multiple theories of HCI. How do they view and define interaction? We find it valuable to extract those views and make their content and assumptions scrutinizable, as the first step in moving toward scientific concepts of interaction. Second, with the exception of Bødker and Kammersgaard, the dis-cussions have been about the merits of one view of interac-tion. There is little comparison of the relative merits and shortcomings of the views. Third, it is not clear what work a view of interaction should do. Extracting and discussing ex-isting concepts of interaction might help make this clear.

CONCEPTS  OF  INTERACTION  IN  HCI’S  THEORIES  Although attempts to define interaction are somewhat con-fused, a main take-away of this paper is that more devel-oped concepts do exist, yet they have been embedded in some of the well-known theories of human-computer inter-action. To show this, this section extracts seven concepts of interaction by examining the literature of HCI. The views were selected because they are much cited, cover very dif-ferent notions of interaction, and are discussed frequently (at least individually) in textbooks; one view, interaction-as-control, was included because it views interaction mainly as statistical determination (we later explain what this is and why it is important). We make no claim to cover all concepts of interaction in the history of HCI or in its nu-merous theories. We recognize missing accounts, such as those about symbolic value, behavior change, neural sys-tems, integration, and collaboration. Yet, the ones we iden-tify seem so central to HCI’s intellectual history that any discussion of “What is interaction?” must deal with them.

Table 1 summarizes the concepts. The three right-hand col-umns were developed as a synthesis of the key differences among views. Note that they are synthesized over many pa-pers and may not directly map to a particular school or the-ory or model. Also, have patience with the length of the synthesis; it is necessary to get clarity on the concepts’ sta-tus before discussing their differences.

In the following, we build on the work of Dubin [26]. He considers a proposition a theoretical statement that links constructs (or “units” in Dubin’s terms; these could be hu-man, coupling, experience, computer) with how they play together (which he calls “interacts”), the boundaries of the proposition, and the overall state of the system being con-sidered. Propositions can be both about processes and about outcomes (corresponding roughly to understanding and predicting). We are looking for propositions about the link or interplay or type of action between humans and comput-ers; we call those concepts of interaction.

Interaction  as  Dialogue  This concept sees interaction as a cycle of communication acts channeled through input/output from the machine per-spective, or perception/action from the human perspective. Interaction as dialogue has been prominent from the early history of HCI. Nickerson et al. [61] suggested that “the thing that, above all others, makes the man-computer inter-action different from the interaction that occurs in other man-machine systems is the fact the former has the nature of a dialogue” (p.129). The concept of dialogue was also used by Card et al. [14, p. 4] as background for GOMS.

Dialogue is composed of stages or turns. Norman’s seven-stage model [62] is one detailed account of such stages; others talk about turns [67]. Norman’s stages go from users formulating their goal, over specifying and executing the actions needed to move that goal forward, to perceiving the resulting system state and relating that to the goal. Under-standing interaction-as-dialogue stresses the need for users’ acts to be understood by the computer and for users to un-derstand the computer. For Norman, this means that map-ping and feedback become crucial concepts for appreciating user interfaces. Mapping requires the user to figure out how to achieve an intention with an interface or the task where “the user must translate the psychological goals and inten-tions into the desired system state, then determine what set-tings of the control mechanisms will yield that state, and then determine what physical manipulations of the mecha-nism are required”, [62, p. 37]. These in turn are related to the concepts of gulf-of-execution and gulf-of-evaluation, which describe breakdowns when users seek to express their intentions and interpret feedback from the system.

Further, this concept highlights understanding, directness, and simplicity. Successful interfaces should “provide a strong sense of understanding and control” [62, p. 49] and “The easier and more direct these two mappings, the easier and more pleasant the learning and use of the interface” (p. 53). Interaction-as-dialogue speaks of directness to clarify

what makes mappings easy: “The ‘naturalness’ of a map-ping is related to the directness of the mapping, where di-rectness can be measured by the complexity of the relation-ship between representation and value, measure by the length of the description of the mapping”. [64, p. 28-29].

This concept has inspired several forms of support for eval-uation and construction of interaction. In evaluation, the view is used in the User Action Framework [2] and by some as the foundation for cognitive walkthrough [74]; guessability studies are one way of evaluating good interac-tion from this view [92]. The seven stages of action can also be used as design aids [63, pp. 52-53]). Mixed-initiative de-sign guidelines also use the concept of stages [42].

Interaction  as  Transmission  (of  Information)  This concept sees interaction as transmission of information between the computer and the user. The rate of transmission can be used as a criterion for what makes a good interface, and the rate of transmission from computer to human and vice versa can be evaluated and improved through design. The focus is not on turn-taking or communicative acts (as in dialogue), but on passing of messages with information content over a channel with limited capacity and noise.

More formally, transmission occurs when a message is se-lected from a set of possible messages and transferred to the receiver over a noisy channel. The size of the set of possible messages determines the information content of the trans-fer. In HCI, this concept is based on Shannon’s theory of communication [27,58,81,89]. This concept operationalizes the rate of message passing as throughput.   This concept of interaction sees good interaction as being about the rate with which bits can be correctly communi-cated. For instance, in discrete aimed movements, a user in-terface is organized into targets with spatially defined areas. The user communicates by aiming and moving an end-effector within one of them. The number of messages is de-fined as the number of areas that could have been selected.

Concept View of interaction Key phenomena and con-

structs Good interaction Example support for

evaluation and design Dialogue a cyclic process of commu-

nication acts and their inter-pretations

mappings between UI and in-tentions; feedback from the UI; turn taking

understandable; simple, natural; direct

methods/concepts for guessability, feedback, mapping; walkthroughs

Transmis-sion

a sender sending a message over a noisy channel

messages (bits); sender and receiver; noisy channels

maximum throughput of in-formation

metrics and models of user performance

Tool use a human that uses tools to manipulate and act in the world

mediation by tools; directness of acting in the world; activity as a unit of analysis

useful and transparent tools; amplification of hu-man capabilities

compatibility in instrumental interaction; break down analysis

Optimal behavior

adapting behavior to goals, task, UI, and capabilities

rationality; constraints; prefer-ences; utility; strategies

improves or reaches max-imum or satisfactory utility

models of choice, foraging, and adaptation

Embodi-ment

acting and being in situations of a material and social world

intentionality; context; coupling provides resources for and supports fluent participa-tion in the world

studies in the wild; thick description

Experience an ongoing stream of expec-tations, feelings, memories

non-utilitarian quality; expecta-tions; emotion

satisfies psychological needs; motivating

metrics of user experience; experience design methods

Control interactive minimization of error against some reference

feedforward; feedback; refer-ence; system; dynamics

rapid and stable conver-gence to target state

executable simulations of interactive control tasks

Table 1. Overview of some key concepts of interaction in HCI literature. The columns sum up the core view of what interaction is, the key phenomena that the view has helped see and their associated constructs, the notion of good interaction that follows from the

view, and the key techniques and methods to help evaluating and designing user interfaces.

In addition to aiming and choice, this concept is also appli-cable to computer output. This has happened less formally (for instance in Tufte’s data-ink ratio [87], which requires visualizations to communicate the most information with the least effort) or formally, as in visual encoding [19]. Goodness is thus on the one hand determined by the com-plexity of messages an agent can pass. The more possible messages that could be communicated, the better. On the other hand, goodness is also determined by how much time must be spent to complete messaging.

In both cases the concept of interaction as transmission may drive design. Beyond offering measurable constructs, its understanding of interaction can be taken as an objective for design. Balakrishnan [4], for instance, synthesized re-search on pointing by its attempt to beat Fitts’s Law, that is, to increase throughput by redesigning the factors that limit transmission. Work on smoothing input for gesture recog-nizers also attempt to deal with noisy channels [17,93].

Interaction  as  Tool  Use  This concept sees interaction as tool use, not different from using a hammer to drive nails or a mental rule to calculate algebra in one’s head [3]. This concept is pervasive in HCI in different forms, and central in (for instance) activity the-ory [10,55]. The computer, when viewed as a tool, is some-thing manipulated by users to affect change and to extend themselves. Interaction, then, is about manipulation (use) of technology for some aims beyond the tool itself.

Three implications arise. First, this concept suggests that in-teraction (and the associated tools) shapes us and how we act. Activity theory [10,55] captures this in the notion of mediation, taken to mean that “tool use influences the na-ture of external behavior and also the mental functioning of individuals”. Beaudoin-Lafon [7] has developed this into a notion of interaction instruments, the parts of the computer that mediates between the user and the domain objects of interest. In general, this concept claims that the tools and the tasks people do with them mutually influence each oth-er; this is also emphasized in the task-artifact cycle [16].

Second, interaction-as-tool-use emphasizes the mediating role of the tool; the point is to “act through the interface” [10]. Therefore, the degree to which the interface interferes with doing this is key; this has been captured in notions of breakdowns [11] or, positively phrased, in ideals of direct manipulation [43], transparent, or invisible UIs [65].

Third, interaction-as-tool-use naturally emphasizes use: “strictly speaking, nothing is a tool except during use”. Thereby, interaction becomes tied to use and to tasks; this is emphasized in what has been called HCI’s turn to practice [56]. Seeing interaction as tool use assumes a sort of unity of the artefact and the purpose it is used for [3].

This view emphasizes usefulness and utility as key evalua-tion criteria of good interaction. Good interaction is defined via the significance of the functions the tool can be used to perform [1,2]. It can succeed or fail to extend our reach,

amplify force, or more generally amplify our capabilities, be they physical, virtual, or cognitive. Moreover, using the tool with a feel of direct engagement with the object of in-terest is preferable to focusing on the tool. Thus, directness [43] is key, sometimes also described as transparent, disap-pearing, or invisible interfaces.

This concept has inspired several forms of support for eval-uation and construction of interaction. Beaudoin-Lafon suggested that the degree of compatibility [7] is key. The latter, for instance, concerns “the similarity between the physical actions of the users on the instrument and the re-sponse of the object” (p. 450). Although Beaudoin-Lafon does not consider this an evaluation dimension, higher compatibility seems better. Analysis of breakdowns is an activity theory approach to evaluation [11]

Interaction  as  Optimal  Behavior  This concept sees interaction as adaptive behavior maxim-izing utility under constraints posed by task, own capabili-ties, and the UI. That is, interaction is joint computer-user behavior where a user pursues some goal or utility to the best of his/her capability within constraints posed by the UI, environment, and the task. Interaction emerges in a system consisting of rewards and costs (or utilities), actions, and constraints (e.g., structure of environment). Adaptation is exhibited in different strategies for using a computer.

The concept rests on the idea of rationality: over time, peo-ple approximate optimal adaptation to constraints. There is abundance of evidence on this idea in motor control, learn-ing, perception, and complex cognitive activities [28]. No-table theories explaining adaptation are information forag-ing, adaptive interaction, economic models, rational analy-sis, and, more recently, computational rationality. For in-stance, foraging theory assumes that an adaptive animal changes the patch as soon as the gain in calories decreases to a level at which it would make more sense to move somewhere else. Information foraging theory [71] uses an analogue where “prey” is information and the environment is a digital environment. As in food foraging, information ecologies are patchy: information is unevenly distributed in regions, or patches, with different contents and expected gains. These properties form constraints and costs that the agent must take into account. From what is locally visible, the informavore must infer what the distant environment may carry. To understand the informavore’s behavior, one needs to understand decision-making under uncertainty.

Good interaction is thus defined via proximity of the joint computer-plus-user performance to the maximum level it might attain. However, the “optimality” of behavior should not be mistaken for the pursuit of high performance in par-ticular. One insight from research on interactive behavior is that even user behavior that appears lazy or ignorant can be optimal for the organism to the extent it prefers saving ef-fort over maximizing performance [68]. Similarly, novice users may not be performing optimally but, given their level of knowledge or the design, are simply not able to do better.

The main contribution to design and evaluation is the anal-ysis it offers for understanding how the structure of the UI—temporal, spatial, and statistical—affects behavior. Theories like information foraging, adaptive information framework, and economic models of interaction have been used to understand adaptation of behavior, choice, and strategies as a function of UI design. And empirical studies have adopted these notions, reporting a surprisingly vast scope of strategic adaptation across domains of interaction [51], from pointing with mouse [29], through learning to use a complex UI [9], to the use of the external world as a memory [30]. Such observations are supported by the con-cept of adaptive behavior, but not so much by, for example, interaction-as-transmission nor interaction-as-dialogue.

Interaction  as  Embodied  Action  This concept sees interaction as embodiment: being and par-ticipating in the world [25,83,91]. Dourish [25] and Wino-grad and Flores [91] root this concept in the ecological psy-chology of Gibson and the hermeneutic phenomenology of Heidegger. Other notable work include Suchman [83], who emphasized situatedness in technology use. Unlike the in-teraction-as-tool-use concept, which is essentially a third-person view, embodiment and situatedness emphasize the first-person view: “doing being a user”.

Three essential concepts for understanding interaction are intentionality (the aboutness of action), coupling, and con-text [25]. Instead of analyzing how the motion of a user’s arm moves a cursor toward a target on a display, interac-tion-as-embodiment highlights the intention of the user (e.g., browsing social media) and how his/her actions are coupled with multiple contexts (sitting in a chair, working, being tired). For the user, the mouse is ready-to-hand and used for “browsing a social media” or “doing my work”, and only when something unexpected occurs, it may be-come present-at-hand. Interaction-as-embodied-action fo-cuses on the lived experience, and it relies less on invariant structures in explanation, such as mental representations. It rather emphasizes and expounds the nuanced and multifac-eted effects that context has in interaction. What we view as interaction emerges in a situation and cannot be reduced back to either the human or the world. This view also em-phasizes how our bodies shape everything we think and do.

Since the computer can be involved as an instrument in ac-tion, as well as being its object, there is no single way to de-fine good interaction except via a negation: a well-defined artefact does not disrupt human pursuits. In good interac-tion, by contrast, the technology offers resources for the human to better participate and be in the world, to “be more human”. In reality-based interaction [45], for example, good interaction taps and respects skill relating to body and social awareness. Other important concepts in this view are present-at-hand and ready-to-hand [84,91], seamfulness [18], and place [32]. Proponents of these concepts empha-size a fluent relationship among intentionality, context, and coupling in action, and discuss their breakdowns.

This concept also has been used to suggest support for evaluation and construction of interaction. Because of the assumed nature of context and situatedness, unlike in the previous views, researchers never developed a “model” or “metric” of embodied interaction with clear-cut concepts and a reasoning apparatus. Rather, focus shifted to method-ology, a departure from the lab-driven approach, rooted in the cognitive paradigm [22]. The new analytical lens neces-sitated an “in the wild” approach to data collection and “thick description” of interaction as it emerges.

Interaction  as  Experience  Interaction is here described as shaped by the users’ expec-tations, momentary unfolding of experience, and recounting of episodes of interaction. While experience may be thought to be an epiphenomenon, a side effect of sorts, this concept heralds its role in shaping how we use computers. The concept is about interaction because people appraise, attribute, and realize their experiences to and with the user interface; experience is also a constantly ongoing part of in-teracting. This view is anchored in research on user experi-ence [38], and well exemplified by work from McCarthy and Wright [94] and Hassenzahl [34].

The key proposition of this view is that human experience is a key factor in how interaction unfolds. For Hassenzahl, experience is a “momentary, primarily evaluative feeling (good-bad) while interacting with a product or service” [35, p. 12]. This feeling is part of all interaction, and is shaped by and shaping interaction. For example, expectations of in-teraction shape how people interact and how they assess the goodness of an interface [94]. Moreover, similar interac-tions are assessed quite differently, depending on their or-der (for instance, the most recent and most intense moments matter differently [20]). The varieties of experience during interaction are as wide-ranging as human experience. Alt-hough experience can be considered an evaluative feeling, research has in particular concerned aesthetics [86], emo-tion [59], and need fulfillment [37], but it can also concern feelings of surprise or stimulation. Although experience can be about usefulness, the key focus is on non-utilitarian as-pects of experience (e.g., those relating to pleasure and meaning, [60]). This view also argues that experiences can-not simply be mapped to interface features or steps in the interaction. This has led to a study of “experiential qualities of technology use rather than product qualities” [37, p. 353]. The underlying idea is that experiences of interaction are the result of appraisal [78], that is, of our cognitive evaluations of events. Moreover, people feel that experi-ences are caused by the interaction to different degrees; this is often called attribution in studies of user experience [37].

Interaction-as-experience has refined our view of good in-teraction. In particular, it has helped emphasize non-instrumental qualities (by definition different from those re-lating to the tool-use-view because they are not about utili-ty); research uses such measures extensively [6]. Further-more, this concept has helped identify the influence of ex-

pectations on experience (e.g., [94]) and the many ways ex-perience shows how we recall episodes of interaction.

This concept has several implications for design, such as positive design [23] and patterns of associations between design and experiences (e.g., Experience Design [36]).

Interaction  as  Control  Finally, control theory sees interaction as interactive mini-mization of error against some reference or goal state of the user. Although not commonly recognized as part of HCI lit-erature, control theory precedes HCI and has influenced its terminology. Yet it differs from the above concepts. Some notions of control theory are widely used in the HCI litera-ture, such as feedback, feedforward, control, input, output, state, and system [46]. However, only a handful of papers actually deploys it in HCI [31,40,54,72].

In control theory, interaction is defined via a system aiming to change a control signal to a desired level (called the ref-erence) and updating its behavior according to feedback. In human use of a computer, the user receives feedback about the state of the computer and produces output (feedforward) to change it toward the reference. Thus, human-computer interaction is a system of goals, signals (inputs and outputs), feedback, feedforward, and states. A user is a goal-directed controller acting with reference to some feedback about the state of the system. States store energy (or information), represented mathematically as integrators. Control theory assumes that all interaction is continuous, or analogous, alt-hough a computer may register it as discrete events.

Its potential value lies in how it makes multiple aspects of ‘goodness’ of interaction explicit and actionable, for exam-ple, notions like distance to goal (error), speed of conver-gence, and stability in the face of noise or uncertainty. It permits a multi-level analysis that can expose lower-level causes of emergent system-level issues such as sluggish performance or errors. Consider pointing for example. Un-like statistical models such as Fitts’ Law, control theory can predict velocity and acceleration profiles and allow inspect-ing overshoot, settling time, peak time, and rise time during pointing. For an analyst, it offers tools like block diagrams and step-wise simulations to model complex interactive be-havior. This could support design and evaluation by antici-pating some of the dynamic ways system-level behavior emerges without expensive studies. By manipulating a model, a researcher can investigate what would happen if, for example, some aspect of the user (e.g., biomechanics) or the device (e.g., transfer function, precision) changed.

WHAT  WORK  DO  THE  CONCEPTS  DO?  Next we discuss the work these concepts can do for indi-vidual HCI researchers and for the field, based on Table 1.

Drawing  Boundaries  Around  Interaction  The concepts of interaction vary significantly in how they portray boundaries of the computer and the human, and the extent to which interaction requires us to consider them as one system (see Table 1, column View of interaction). For in-

stance, does it involve the interface only or also that on which the interface acts or even non-computing parts essen-tial to the interaction? What is the time span of interaction?

For instance, the view of the computer differs. Interaction-as-transmission looks at the efficiency with which an in-tended message can be communicated and correctly regis-tered, which focuses a researcher’s attention on the in-put/output interface. In contrast, interaction-as-embodied situates technology in a praxis and is therefore unwilling to clearly delineate computer artefacts and separate them from non-computing artefacts (see [79]). Some critiques of inter-action, for instance that of Taylor [85], miss the observation that different concepts of interaction come with different scopes; that is, some focus on interaction, others on context. Likewise, the view of the human differs. Some concepts excel by relating the human to its social and physical con-text (e.g., interaction-as-embodiment); others focus narrow-ly on humans as senders, noisy channels, and receivers (e.g., interaction-as-transmission). How one thinks about in-teraction thus relates to key debates in HCI. The third wave debate [12], for instance, is about extending the scope of the human, but at least in some interpretations it throws out views of interaction with narrower scope that are complete-ly useful (and at times necessary).

Our second observation about scope is that some concepts of interaction see the boundary around the entire computer-human system; because interaction links human and com-puter, it makes little sense to analyze them separately. This is true for tool use and optimal behavior; the earlier quote by Butler (“nothing is a tool except during use”) highlight-ed this. By contrast, interaction-as-experience subscribes to no such link. Interaction as an ongoing stream of expecta-tions, feelings, and memories does not depend on a tool. This raises an important point for user experience research. While it describes what makes up experience, much of it is about downstream consequences of interaction [37] or about its emotional quality [59]. Theories of user experi-ence say little about how the human-computer relationship is construed such that both entities affect each other.

A third observation is that the views differ in the time-scales at which they consider interaction (see also [71]). Some operate at the level of seconds (e.g., interaction-as- control, interaction-as-transmission), others at the level of Newell’s unit task, and still others consider both expecta-tions and memories of interaction (e.g., interaction-as-experience). The time-scale is related to whether concepts emphasize expert or novice users. Interaction-as-transmission often focuses on expert performance and con-cepts like mediation (how tools shape our thinking about the world) and the task-artefact cycle, both related to tool-use, require long periods of use and proficient users.

Vocabulary  for  Phenomena  in  Interaction  The concepts of interaction each identify distinct phenome-na in interaction (see Table 1, column Key phenomena and constructs). These are important for problem-solving in

HCI [66] in that they create or elaborate “descriptions of re-al-world phenomena related to human use of computing” (p. 4958). This is important, because concepts not only sen-sitize a reader to certain aspects of interaction but also pro-vide vocabulary and a reasoning apparatus to work with them. The more general consideration relating to these is their ability to transfer; that is, how well they “transfer to neighboring problems or other instances of the problem” [66, p. 4959]. Concepts that are transferable equip a re-searcher with a general tool to work with interaction.

For example, the concepts of mapping and feedback (inter-action-as-tool-use) are useful because they explain break-downs in interaction known to most users of everyday arte-facts and computers; they are further transferable across a range of contexts and technologies. The concept of map-ping has been used in as diverse fields as mid-air-gestures and augmented reality. The existence of such transferable concepts relating to interaction helps HCI.

A  Yard  Stick  for  “Good  Interaction”  Table 1 (column Good interaction) further suggests that con-cepts of interaction come with some assumed notion of when interaction is good or successful. Concepts of interac-tion, as concepts often are, are value-laden. This is notewor-thy because it means that one’s notion of interaction pre-scribes a particular aim for design. For tool-use it is primar-ily about usefulness; for dialogue it is ease of translation or mapping (naturalness); and for embodiment it is coupling. It also means that measures of interaction are interpreted differently. For instance, the role of time in “goodness” is interpreted quite differently in the various concepts [41]: For transfer and tool-use, time should be minimized, for embodied action it becomes relevant only upon disruption.

Taking  New  Perspectives    Based on Table 1 and discussion above you might wonder: “Given the views of interaction, which is right?”. We find this question misguided. Whether or not they are true is less important than whether they are useful for understanding phenomena in human use of computing or interactive tech-nology. The second reason is pragmatic. We argue that knowing many concepts better help a researcher see new opportunities in research and change perspectives when reaching an impasse. Or, when reaching an impasse, per-haps a new concept can be invoked. Perspective can be changed to rethink scope, constructs, or criteria for good-ness. Compared to earlier discussions of interaction [70] and theories of HCI [75], we believe the present set of con-cepts on interaction can be more directly used for this.

How might perspective-taking be useful in everyday HCI research? Let us take work on electric muscle stimulation (EMS) as an example. EMS in HCI has been used to gener-ate prototypes that can move users’ hands and arms and serve as new ways of body-based output. EMS work seems to conceive of interaction mostly as transmission of infor-mation, in particular, how one can control the limbs of the users in the most accurate way. In essence, this is about

dealing with the noise channel of electrically stimulating a user’s arm to produce movement. And it is about through-put of that channel, for instance, as modelled, as done by (for instance) Kaul et al. [50], or accuracy of recognition, as done by (for instance) Lopes and Baudisch [57]. We argue that the concepts of interaction may be used to think through alternative directions for EMS in HCI. Take inter-action-as-tool-use. This concept emphasizes mediation by the EMS system and the disappearance of tools; both of which are unexplored in EMS work. Or take interaction-as-embodied-action and its emphasis on how the user being situated in a social world changes the way an interface is being used: how do onlookers perceive motion by EMS and what is the experience of interacting with other people, say, giving a handshake, through muscle stimulation?

CONSTRUING  THE  HUMAN-­COMPUTER  RELATIONSHIP  The previous discussion made several observations about differences in the scope and the work that concepts of inter-action do. However, this discussion has mainly identified dimensions but not gone deeply into the ways in which the concepts help us move from a folk notion of interaction to a notion that really explains interaction.

Let us illustrate the concern by comparing two views, transmission and embodiment. Consider Fitts’ Law (a key technique in interaction-as-transmission). Its core assump-tion is that the properties of the motor system and those of an input device place an upper boundary to pointing per-formance. Yet this boundary is only statistical: it applies to aggregates of multiple trials. In contrast, compare interac-tion-as-embodiment. In this view, “input” gains meaning only as part of engagements such as communicating with other people. Those elements, in context, shape what “do-ing” and “being” a user means. The raw performance of an input device is largely irrelevant in that context unless a dis-ruption that brings it to the fore. This concept qualitatively examines a complex web of effects, factors, and phenomena to describe an interaction, prioritizing descriptive thickness.

Our idea for a starting a serious discussion of such differ-ences more generally is based on Bunge’s typology of causal explanations [13]. This provides a thinking tool to answer Bannon’s call for reconsidering the human-technology relationships [5].

Reasoning  About  Interaction  According  to  Bunge  According to Bunge [13] causal determination is just one of the multiple forms of relationships we employ in modern scientific explanation. Bunge uses the notion determination, an umbrella term for the various ways in which two things can be connected and influence each other. He develops a rich typology for determination. In addition to causal de-termination, the types of determination include teleological, mechanical, statistical, structural, dialectical, interaction (which for Bunge is reciprocal causation such as in gravity), and quantitative self-determination.

With Bunge, human-computer interaction is about the de-termination that both the computer and the human have on events that unfold in computer use. Events in computer use can be attributed solely neither to the human nor to the computer. The two must be considered together. Some con-cepts appear to embrace this better than others. For exam-ple, we argued earlier how interaction-as-experience is only starting to discuss mutual (e.g., appraisal) instead of one-way determination, whereas interaction-as-tool-use gives the tool its “toolness” only in and via some human activity. Let us try to see if clarity ensues from relating Bunge’s types of determination to concepts of interaction. Note that some concepts deploy more than one type of determination.

First off, we note that all concepts we have discussed above use teleological determination. That is, human intentions postulate a norm or goal of interaction. Even the view of in-teraction-as-transmission assumes that the human sender has a message in mind. Some views, such as interaction-as-embodied-action and interaction-as-dialogue, explicitly mention intentions. In interaction-as-optimal-behavior, we have intentions defined precisely as utilities.

Mechanical determination refers to explanation of conse-quence by the antecedent; for instance, human sends input, which is processed, which changes the computer's state, which responds, which human registers, etc. Some models related to interaction-as-dialogue, say GOMS and Key-stroke-Level Modeling (KLM) [14], are deterministic in this way. In KLM and GOMS, interaction is described as sequences of deterministic processing steps with defined costs and outputs. Other models such as ACT-R [1] have relaxed the mechanistic assumption and added stochastic components (e.g., when modeling learning and memory).

Statistical determination characterizes the end results of interaction as being jointly determined, stochastically, by some efficient causes and mutual forces. The point is not to describe in exact and exhaustive terms how they affect each other, but to capture statistically their effect on antecedents of interaction. Fitts’ Law and information foraging are ex-amples following two different interaction concepts, yet both engage in statistical reasoning about the joint effects.

Structural determination refers to explanation of parts by the whole. Information-as-optimal-behavior engages in this form systematically [68]. Information foraging theory, for example, assumes that certain exploration behavior emerges as a function of how the environment is arranged. Another well-known example is related to tool-use, name task anal-ysis. It is the structure of the task that affects what acts the user performs and in what order. Interaction-as-embodiment employs this too, as does, for example, the theory of situated action. They reject the mechanistic cau-sality of mental representations and claim that users’ plans are only weakly related to how they actually behave, which instead emerges situationally in an unpredictable and un-planned manner in a complex web of phenomena.

We could not identify examples of dialectical determinism nor Bunge’s interaction, which he exemplifies with gravity and the interactive effect that planetary bodies have on each other. However, we noted that control-theoretical models subscribe to Bunge’s quantitative self-determination: similar to thermodynamics, they describe interaction as continuous unfolding of states that differ from one another in quantitative respects.

THREE  BLIND  SPOTS  IN  CONCEPTS  OF  INTERACTION  Above we have argued that theorizing related to the differ-ent concepts differ (1) in scope, and (2) in the apparatus used for causal reasoning. This is important because con-struals of the human-computer relationship implicitly con-strain how one can or cannot work with phenomena in in-teraction. Let us discuss three such limitations.

First, while all concepts subscribe to teleological reason-ing, most say little about how intentions are formed or af-fected by interaction. All concepts we reviewed commit to some causal role of intentions (e.g., goals)—even the low-er-level control and transmission views. However, inten-tions are taken-for-granted and precede interaction. Even Norman’s “gulfs” say nothing about how they change via interaction. An exception is the interaction-as-embodiment view, which speaks about intentions and agency, but which still is silent about how intentions evolve. Thus, HCI, via its concepts, has had an overwhelming tendency to understand interaction as one-sided—as channeling and realization of human intentions through a computer, furthermore assum-ing that these intentions are outside the realm of interaction itself. This has created a ‘blind spot’, with implications for our ability to address some important contemporary topics. For designers, this means that they are not able to anticipate how user experience changes over the course of interaction, as users discover new opportunities for action. And it means that we have no good models for multi-party interac-tion (e.g., among multiple users and computer agents), such as in persuasive computing or human-robot interaction.

Second, none of the concepts of interaction offers anything in terms of a workable concept about what a computer or computing is in interaction, as opposed to any human-made artefact or natural object. Compare this to the amount of at-tention to models of users in HCI. The seven concepts of interaction appear to apply to anything from interaction with doorknobs to that with Turing machines. Whether some piece of technology is, internally, a door handle or a Turing machine is a non-issue, although questions like this are at the core of computer science. There is little “C” in HCI. The point that HCI research is not about (Turing-strong) computers but about technology, systems, media-tion, or experience, has been made by many others before us (e.g., [5,15,24,76,80]), but not by these grounds.

Third, in our attempts to extract concepts of interaction, we used the work of Dubin [26] and tried to extract from the literature propositions about the link or interplay or type of action between humans and computers. Somewhat surpris-

ingly, finding such propositions is hard, even for well-established theories, not to mention critical discussions or empirical tests of their validity. For instance, consider as an example Norman’s seven-stage model. It is present in one form or another in most HCI textbooks as well as in a lot of papers in HCI. Yet its propositions about the interplay are oddly limited. It says that interaction proceeds in stages, but even Norman disagrees with that [62, pp. 41-42]. It says that interaction is a dialogue, but ignores the longer-term goals and consequences of that dialogue. Whereas Nor-man’s model provides important concepts, it has limited empirical support; we cannot think of a single empirical challenge of the model. Our work using Bunge also suggest that extracting propositions about the specifics of determi-nation from the HCI literature is a very hard task.

UNDERSTANDING  LIMITS  OF  CONSTRUCTIVE  POWER    The observant reader will have noted that we did not dis-cuss in depth, so far, the column about evaluation and de-sign in Table 1 (Example Support for evaluation and design). This is rooted in an unease with the extent to which views of interaction support thinking about construction, an essen-tial part of problem-solving in HCI [66]. For the present paper, this has to do with how concepts of interaction can inform design. If they cannot, the link between theorizing and construction would be broken [53,66]. While plenty of attention has been devoted to how empirical studies can in-form design, we know of no attempt to systematically look at HCI’s concepts and theories for this end.

To address this, we first need to understand what it means for concepts of interaction (or any construct in HCI) to in-form design. Intuitively, to inform design means that a de-signer is better able to achieve some desired ends in interac-tion given the concept. Users fare better when something has been designed using a concept than when it has not. Thus, concepts must inform design towards better choices. To then understand what actionable information may look like, we need the notion of counterfactual reasoning. Coun-terfactual reasoning allows us to anticipate what happens in some speculated conditions of interaction. If some condi-tion of interaction were (hypothetically) to change, it would tell what consequences to process or what outcomes are likely. Fitts’ Law is a straightforward example: A change in the spatial properties of a target (distance or width, D or W) affects average pointing performance (outcome) in a statis-tically reliable manner. Thus, the constructive power of an HCI construct (concept, theory, model) is the scope, validi-ty, and practical relevance of the counterfactual reasoning it permits that links the conditions of interaction and its events (process and outcomes).

Let us use Fitts’ Law to illustrate this. Scope refers to the number of conditions and events, and level of detail therein, that a construct applies to. Fitts’ Law’s scope is narrow in this sense. It maps two variables (D and W) to one variable (MT). To increase its scope, it should be extended to talk about other design choices, such as colors or input modali-

ty. Validity refers to the construct’s ability to produce em-pirically testable and accurate predictions. Fitts’ Law pre-dicts statistical averages of pointing performance, but it is mute about how an individual might perform in an individ-ual act of pointing. Finally, practical relevance refers to how important these design decisions and events are to de-signers. Fitts’ Law becomes relevant when improving a widely-used interface for a mass of experienced users. Hence, it is relevant in some but not most design projects.

With this definition, we can start dividing constructs into classes according to what they offer in design:

Design-agnostic constructs do not support counterfactual reasoning linking conditions of interaction to events. The theory of computer self-efficacy [21], related to interaction-as-experience, for example, does not link perceived self-efficacy to conditions influenced by designers, but talks about a user’s expectations and perceptions. Similarly, a claim about two types of experience (viz., utilitarian, he-donic) is sometimes made without linking their emergence to conditions of interaction. The issue is that constructs in this category can inform design only after the fact. They are weak in their ability to inform design. At their best, they can point out objectives to a designer and offer empirical methodology and measurement instruments. This might still be valuable for understanding and theorizing about phe-nomena, but less so for designing.

Design-sensitizing constructs, in contrast, point toward opportunities in design (conditions) but do not tell how changes in those conditions affect interaction. They are valuable to the extent that such opportunity might be oth-erwise overlooked. Consider seamful interaction [18]. It tells us that one should make technological enablers visible to users when their breakdowns and disfluencies affect them. However, it does not tell how to design seams for un-derstandability or to avoid them becoming disfluencies themselves. Generally, constructs in this category provide ideas and inspiration to design, but do not strongly guide our choices. Direct manipulation (interaction-as-tool-use) suggest that command languages are worse because humans are better at recognition than at recall. But it does not say how features of an interface are linked to recognition.

We note that some of the theories involved in interaction-as-embodied-action and interaction-as-tool-use are “super-frameworks” with multiple design-sensitizing constructs. Their value is in that they are frameworks that link multiple phenomena and factors, sensitizing a reader to them. Yet they do not explicate how their relation is shaped in some particular interaction.

Design-determining constructs, finally, comprise the class of constructs that link design variables to events in interac-tion in some determinate way. This means that they can in-form a particular or probably good decision, such as what modality to use or which type of feedback motivates users to stay in a service. As discussed, there are multiple types

of determinacy, from statistical to structural to teleological, so this determinacy can take different forms. Our view is that the theories that design-determining constructs HCI has mostly come from behavioral and social sciences, includ-ing, among many others, such theories as information for-aging theory, theories of color, task analysis, or cognitive simulations (e.g., EPIC).

A  Dilemma  for  Concepts  of  Interaction  With these three groups in mind, we return to point out a di-lemma in HCI’s ability to support design: While we have concepts of interaction that talk about design (many, in fact), we have failed to produce theories and concepts that have both high determinacy and adequate scope. In other words, we cannot find concepts of interaction (or, general HCI constructs or theories, for that matter) that deal with a large number of phenomena at the level of specificity that is needed to inform design decisions.

Instead, those concepts fall into two types, both unsatisfac-tory: (1) high determinacy but inadequate scope and (2) large scope but low determinacy. Examples of the former are classic mathematical and simulation models of the user subscribing to interaction-as-dialogue, interaction-as-transmission, interaction-as-control, and interaction-as-optimal-behavior. Examples of the latter can be found from theories around interaction-as-embodiment, interaction-as-tool-use, interaction-as-experience, such as theories like sit-uated action, distributed cognition, embodied interaction, and so on. While they encompass several phenomena relat-ed to design, they offer very little direct guidance to con-crete decisions in design. This is related to the difference between understanding and prediction, discussed in depth by Dubin [26]; here, however, the tension seems to be be-tween understanding and constructing.

To conclude, design and engineering in HCI involve com-plex decisions and complex phenomena, but HCI has only offered either high-determinacy models with limited scope, implying that one has to use many of such models, or broadly scoped theories with low determinacy, implying that one must carry out empirical studies to understand the case at hand. Although frowned upon by some HCI schol-ars, compare this situation to that in areas of engineering that routinely use theories such as mechanics or dynamics to inform complex projects such as in civil engineering. We find it important to work toward higher determinacy and wider scope in HCI theories.

INTERACTION,  REVISITED  After all this, what is interaction? Let us first answer this in the negative: Interaction is not the idea promoted and re-peated in folk notions that a computer and a human are en-gaged in some interplay. HCI researchers should not be sat-isfied with definitions that are simply listings of dimensions of this everyday concept.

What is it, then? Our extraction of concepts from HCI theo-ries gives a preliminary idea. Following Bunge, interaction

concerns two entities that determine each other’s behavior over time. In HCI, the entities are computers (ranging from input devices to systems) and humans (ranging from end-effectors to tool users). Their mutual determination can be of many types, including statistical, mechanical, and struc-tural. But their causal relationship is teleologically deter-mined: Users, with their goals and pursuits, are the ultimate metric of interaction. From here on, how researchers con-strue that determination influences the phenomena they can attend to, what they think good interaction is, and what tools they have to offer evaluation and design. We showed that there are at least seven endpoints to such construals, each with distinguishable boundaries and scopes, determi-nations, and valuations. The fact that the relation between human and computer can be construed in so many ways has (thus far) made the concept so elusive. We have shown that interaction, as a concept, can do much more work in HCI than it has done thus far [85,88]. This observation calls for applications, expansions, and critiques of these concepts, rather than for throwing them away. It is clear that our se-lection of views was selective; we look forward to system-atic, large-scale reviews of concepts of interaction in HCI.

We can now offer an explanation to Kostakos’ contention of “a big hole in HCI research” [53]. Conceptions of inter-actions have been relatively long-lasting, and they—and the key findings associated with them (Table 1)—form a com-mon core of HCI research, but this core has not been as recognized and developed as it could have been. Better concepts and theories of interaction is the way of getting away from “the hole” and aiming towards HCI research that is relatively independent of developments in technologies and use situations. Interaction, simply, is the core of HCI.

Looking forward, HCI research must work on core concepts of interaction that can inform its “point studies and point designs”. To reach this stage, we need to ask what interac-tion could be? Based on the preceding discussion, the con-cepts of interaction should have much more to say about design and should be much better able to describe mutual dependencies specific to HCI. We must focus much more on propositions [26] about interaction: theoretic statements that link the determination between humans and computers, given certain starting conditions and boundary conditions. Those can be quantitative or qualitative; about prediction or understanding. Those conditions link theories to design and equip researchers/designers with some foresight that has been missing. We hope that this essay encourages the de-velopment of a richer and more useful science of interac-tion.

ACKNOWLEDGEMENTS  This project has received funding from the European Re-search Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreements 648785 and 637991). We thank several colleagues for dis-cussions, including Barry Brown, Susanne Bødker, Morten Hertzum, Elisa Mekler, and Stuart Reeves.

REFERENCES  1. John R. Anderson. 1996. ACT: A simple theory of com-

plex cognition. American Psychologist 51, 4: 355. 2. Terence S. Andre, H. Rex Hartson, Steven M. Belz, and

Faith A. McCreary. 2001. The user action framework: a reliable foundation for usability engineering support tools. International Journal of Human-Computer Studies 54, 1: 107–136.

3. Christopher Baber. 2003. Cognition and tool use: Forms of engagement in human and animal use of tools. CRC Press.

4. Ravin Balakrishnan. 2004. “Beating” Fitts’ Law: Virtual Enhancements for Pointing Facilitation. Int. J. Hum.-Comput. Stud. 61, 6: 857–874.

5. Liam Bannon. 2011. Reimagining HCI: toward a more human-centered perspective. interactions 18, 4: 50–57.

6. Javier A. Bargas-Avila and Kasper Hornbæk. 2011. Old wine in new bottles or novel challenges: a critical analy-sis of empirical studies of user experience. In Proceed-ings of the 2011 annual conference on Human factors in computing systems, 2689–2698.

7. Michel Beaudouin-Lafon. 2000. Instrumental interac-tion: an interaction model for designing post-WIMP user interfaces. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems, 446–453.

8. Michel Beaudouin-Lafon. 2004. Designing interaction, not interfaces. In Proceedings of the working conference on Advanced visual interfaces, 15–22.

9. Suresh K. Bhavnani and Bonnie E. John. 2000. The stra-tegic use of complex computer systems. Human-computer interaction 15, 2: 107–137.

10. Susanne Bødker. 1987. Through the interface-A human activity approach to user interface design. DAIMI Report Series 16, 224.

11. Susanne Bødker. 1995. Applying activity theory to vid-eo analysis: how to make sense of video data in human-computer interaction. In Context and consciousness, 147–174.

12. Susanne Bødker. 2006. When Second Wave HCI Meets Third Wave Challenges. In Proceedings of the 4th Nor-dic Conference on Human-computer Interaction: Changing Roles (NordiCHI ’06), 1–8.

13. Mario Augusto Bunge. 1979. Causality and Modern Science. New York: Dover Publications.

14. Stuart K. Card, Allen Newell, and Thomas P. Moran. 1983. The psychology of human-computer interaction.

15. John M. Carroll. 1997. Human-computer interaction: psychology as a science of design. Annual review of psychology 48, 1: 61–83.

16. John M. Carroll, Wendy A. Kellogg, and Mary Beth Rosson. 1991. The Task-artefact cycle. In Designing In-teraction, John M. Carroll (ed.). Cambridge University Press, New York, NY, USA, 74–102.

17. Géry Casiez, Nicolas Roussel, and Daniel Vogel. 2012. 1 € Filter: A Simple Speed-based Low-pass Filter for Noisy Input in Interactive Systems. In Proceedings of

the SIGCHI Conference on Human Factors in Compu-ting Systems (CHI ’12), 2527–2530.

18. Matthew Chalmers and Areti Galani. 2004. Seamful in-terweaving: heterogeneity in the theory and design of in-teractive systems. In Proceedings of the 5th conference on Designing interactive systems: processes, practices, methods, and techniques, 243–252.

19. William S. Cleveland and Robert McGill. 1984. Graph-ical perception: Theory, experimentation, and applica-tion to the development of graphical methods. Journal of the American Statistical Association 79, 387: 531–554.

20. Andy Cockburn, Philip Quinn, and Carl Gutwin. 2015. Examining the peak-end effects of subjective experi-ence. In Proceedings of the 33rd annual ACM confer-ence on human factors in computing systems, 357–366.

21. Deborah R. Compeau and Christopher A. Higgins. 1995. Computer self-efficacy: Development of a measure and initial test. MIS quarterly: 189–211.

22. Andy Crabtree, Alan Chamberlain, Rebecca E. Grinter, Matt Jones, Tom Rodden, and Yvonne Rogers. 2013. In-troduction to the special issue of “The Turn to The Wild.” ACM Transactions on Computer-Human Interac-tion (TOCHI) 20, 3: 13.

23. Pieter MA Desmet and Anna E. Pohlmeyer. 2013. Posi-tive design: An introduction to design for subjective well-being. International Journal of Design, 7 (3), 2013.

24. Alan Dix. 2016. Human–computer interaction, founda-tions and new paradigms. Journal of Visual Languages & Computing.

25. Paul Dourish. 2004. Where the action is: the foundations of embodied interaction. MIT press.

26. Robert Dubin. 1969. Theory building. The Free Press, New York.

27. Paul M. Fitts. 1954. The information capacity of the human motor system in controlling the amplitude of movement. Journal of experimental psychology 47, 6: 381.

28. Samuel J. Gershman, Eric J. Horvitz, and Joshua B. Tenenbaum. 2015. Computational rationality: A con-verging paradigm for intelligence in brains, minds, and machines. Science 349, 6245: 273–278.

29. Wayne D. Gray and Deborah A. Boehm-Davis. 2000. Milliseconds matter: An introduction to microstrategies and to their use in describing and predicting interactive behavior. Journal of Experimental Psychology: Applied 6, 4: 322.

30. Wayne D. Gray and Wai-Tat Fu. 2001. Ignoring perfect knowledge in-the-world for imperfect knowledge in-the-head. In Proceedings of the SIGCHI conference on Hu-man factors in computing systems, 112–119.

31. Peter A. Hancock, Richard J. Jagacinski, Raja Par-asuraman, Christopher D. Wickens, Glenn F. Wilson, and David B. Kaber. 2013. Human-automation interac-tion research past, present, and future. Ergonomics in Design: The Quarterly of Human Factors Applications 21, 2: 9–14.

32. Steve Harrison and Paul Dourish. 1996. Re-place-ing space: the roles of place and space in collaborative sys-tems. In Proceedings of the 1996 ACM conference on Computer supported cooperative work, 67–76.

33. Jennefer Hart, Alistair G. Sutcliffe, and Antonella De Angeli. 2013. Love It or Hate It!: Interactivity and User Types. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’13), 2059–2068.

34. Marc Hassenzahl. 2004. The interplay of beauty, good-ness, and usability in interactive products. Human-Computer Interaction 19, 4: 319–349.

35. Marc Hassenzahl. 2008. User experience (UX): towards an experiential perspective on product quality. In Pro-ceedings of the 20th Conference on l’Interaction Homme-Machine, 11–15.

36. Marc Hassenzahl. 2010. Experience design: Technology for all the right reasons. Synthesis Lectures on Human-Centered Informatics 3, 1: 1–95.

37. Marc Hassenzahl, Sarah Diefenbach, and Anja Göritz. 2010. Needs, affect, and interactive products – Facets of user experience. Interacting with Computers 22, 5: 353–362.

38. Marc Hassenzahl and Noam Tractinsky. 2006. User ex-perience - a research agenda. Behaviour & Information Technology 25, 2: 91–97.

39. Martin G. Helander, Thomas K. Landauer, and Prasad V. Prabhu. 1997. Handbook of human-computer interac-tion. Elsevier.

40. Jesse Hoey, Tobias Schroder, and Areej Alhothali. 2013. Bayesian affect control theory. In Affective Computing and Intelligent Interaction (ACII), 2013 Humaine Asso-ciation Conference on, 166–172.

41. Kasper Hornbæk. 2006. Current practice in measuring usability: Challenges to usability studies and research. International journal of human-computer studies 64, 2: 79–102.

42. Eric Horvitz. 1999. Principles of mixed-initiative user interfaces. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems, 159–166.

43. Edwin L. Hutchins, James D. Hollan, and Donald A. Norman. 1985. Direct manipulation interfaces. Human–Computer Interaction 1, 4: 311–338.

44. Jacko, J. A. 2012. The Human-Computer Interaction Handbook. CRC Press.

45. Robert J.K. Jacob, Audrey Girouard, Leanne M. Hirsh-field, et al. 2008. Reality-based interaction: a framework for post-WIMP interfaces. In Proceeding of the twenty-sixth annual CHI conference on Human factors in com-puting systems - CHI ’08.

46. Richard J. Jagacinski and John M. Flach. 2003. Control theory for humans: Quantitative approaches to model-ing performance. CRC Press.

47. Lars-Erik Janlert and Erik Stolterman. 2016. The Mean-ing of Interactivity—Some Proposals for Definitions and Measures. Human-Computer Interaction.

48. Grace J. Johnson, Gordon C. Bruner II, and Anand Ku-mar. 2006. Interactivity and its facets revisited: Theory and empirical test. Journal of Advertising 35, 4: 35–52.

49. John Kammersgaard. 1988. Four different perspectives on human–computer interaction. International Journal of Man-Machine Studies 28, 4: 343–362.

50. Oliver Beren Kaul, Max Pfeiffer, and Michael Rohs. 2016. Follow the Force: Steering the Index Finger To-wards Targets Using EMS. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA ’16), 2526–2532.

51. Alex Kirlik. 2006. Adaptive perspectives on human-technology interaction: Methods and models for cogni-tive engineering and human-computer interaction. Ox-ford University Press.

52. David Kirsh. 1997. Interactivity and multimedia inter-faces. Instructional Science 25, 2: 79–96.

53. Vassilis Kostakos. 2015. The Big Hole in HCI Research. interactions 22, 2: 48–51.

54. Per Ola Kristensson, Xiaojun Bi, Andrew Howes, et al. 2015. Principles, techniques and perspectives on optimi-zation and HCI. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems, 2441–2444.

55. Kari Kuutti. 1996. Activity theory as a potential frame-work for human-computer interaction research. Context and consciousness: Activity theory and human-computer interaction: 17–44.

56. Kari Kuutti and Liam J. Bannon. 2014. The Turn to Practice in HCI: Towards a Research Agenda. In Pro-ceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’14), 3543–3552.

57. Pedro Lopes, Patrik Jonell, and Patrick Baudisch. 2015. Affordance++: Allowing Objects to Communicate Dy-namic Use. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI ’15), 2515–2524.

58. I. Scott MacKenzie. 1992. Fitts’ law as a research and design tool in human-computer interaction. Hum.-Comput. Interact. 7, 1: 91–139.

59. Sascha Mahlke and Manfred Thüring. 2007. Studying antecedents of emotional experiences in interactive con-texts. In Proceedings of the SIGCHI conference on Hu-man factors in computing systems, 915–918.

60. Elisa D. Mekler and Kasper Hornbæk. 2016. Momentary Pleasure or Lasting Meaning?: Distinguishing Eudai-monic and Hedonic User Experiences. In Proceedings of the 2016 CHI Conference on Human Factors in Compu-ting Systems (CHI ’16), 4509–4520.

61. Raymond S. Nickerson, Jerome I. Elkind, and Jaime R. Carbonell. 1968. Human Factors and the Design of Time Sharing Computer Systems. Human Factors: The Jour-nal of the Human Factors and Ergonomics Society 10, 2: 127–133.

62. Donald A. Norman. 1986. Cognitive engineering. User centered system design: New perspectives on human-computer interaction 3161.

63. Donald A. Norman. 1988. The psychology of everyday things. Basic books.

64. Donald A. Norman. 1991. Cognitive artifacts. In De-signing Interaction. 17–38.

65. Donald A. Norman. 1999. The invisible computer: Why good products fail, the personal computer is so complex, and information appliances are the solution. MIT press.

66. Antti Oulasvirta and Kasper Hornbæk. 2016. HCI Re-search as Problem-Solving. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Sys-tems, 4956–4967.

67. Stephen J. Payne. 1990. Looking HCI in the I. In Pro-ceedings of the IFIP TC13 Third Interational Confer-ence on Human-Computer Interaction, 185–191.

68. Stephen J. Payne and Andrew Howes. 2013. Adaptive interaction: A utility maximization approach to under-standing human interaction with technology. Synthesis Lectures on Human-Centered Informatics 6, 1: 1–111.

69. Marianne Graves Petersen, Ole Sejer Iversen, Peter Gall Krogh, and Martin Ludvigsen. 2004. Aesthetic Interac-tion: A Pragmatist’s Aesthetics of Interactive Systems. In Proceedings of the 5th Conference on Designing In-teractive Systems: Processes, Practices, Methods, and Techniques (DIS ’04), 269–276.

70. William A. Pike, John Stasko, Remco Chang, and The-resa A. O’Connell. 2009. The science of interaction. In-formation Visualization 8, 4: 263–274.

71. Peter Pirolli and Stuart Card. 1999. Information forag-ing. Psychological review 106, 4: 643.

72. Henning Pohl and Roderick Murray-Smith. 2013. Fo-cused and casual interactions: allowing users to vary their level of engagement. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2223–2232.

73. J Preece, Yvonne Rogers, and Helen Sharp. 2015. Inter-action Design: Beyond Human-computer Interaction. Wiley.

74. Antonio Rizzo, Enrica Marchigiani, and Alessandro An-dreadis. 1997. The AVANTI project: prototyping and evaluation with a cognitive walkthrough based on the Norman’s model of action. In Proceedings of the 2nd conference on Designing interactive systems: processes, practices, methods, and techniques, 305–309.

75. Yvonne Rogers. 2012. HCI theory: classical, modern, and contemporary. Synthesis Lectures on Human-Centered Informatics 5, 2: 1–129.

76. Pertti Saariluoma. 2005. The challenges and opportuni-ties of human technology. Human Technology 1, 1: 1–4.

77. Dan Saffer. 2010. Designing for interaction: creating innovative applications and devices. New Riders.

78. Klaus R. Scherer, Angela Schorr, and Tom Johnstone. 2001. Appraisal processes in emotion: Theory, methods, research. Oxford University Press.

79. Kjeld Schmidt and Jørgen Bansler. 2016. Computational Artifacts: Interactive and Collaborative Computing as an Integral Feature of Work Practice. In COOP 2016: Pro-ceedings of the 12th International Conference on the

Design of Cooperative Systems, 23-27 May 2016, Tren-to, Italy, Antonella De Angeli, Liam Bannon, Patrizia Marti and Silvia Bordin (eds.). Springer International Publishing, 21–38.

80. Abigail Sellen, Yvonne Rogers, Richard Harper, and Tom Rodden. 2009. Reflecting human values in the digital age. Communications of the ACM 52, 3: 58–66.

81. Steven C. Seow. 2005. Information theoretic models of HCI: a comparison of the Hick-Hyman law and Fitts’ law. Human-Computer Interaction 20, 3: 315–352.

82. Shneiderman Ben Shneiderman and Catherine Plaisant. 2005. Designing the user interface 4 th edition. ed: Pearson Addison Wesley, USA.

83. Lucy A. Suchman. 1987. Plans and situated actions: The problem of human-machine communication. Cam-bridge university press.

84. Dag Svanæs. 2013. Philosophy of interaction. The Ency-clopedia of Human-Computer Interaction, 2nd Ed.

85. Alex Taylor. 2015. After Interaction. interactions 22, 5: 48–53.

86. N Tractinsky, A. S Katz, and D Ikar. 2000. What is beautiful is usable. Interacting with Computers 13, 2: 127–145.

87. Edward R. Tufte. 1983. The visual display of quantita-tive information. Graphics press Cheshire, CT.

88. Peter-Paul Verbeek. 2015. Beyond interaction: a short introduction to mediation theory. interactions 22, 3: 26–31.

89. Christopher D. Wickens, Justin G. Hollands, Simon Banbury, and Raja Parasuraman. 2015. Engineering psychology & human performance. Psychology Press.

90. Terry Winograd. 1990. What can we teach about hu-man-computer interaction? (plenary address). In CHI ’90 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.

91. Terry Winograd and Fernando Flores. 1986. Under-standing computers and cognition: A new foundation for design. Intellect Books.

92. Jacob O. Wobbrock, Meredith Ringel Morris, and An-drew D. Wilson. 2009. User-defined gestures for surface computing. In Proceedings of the 27th international conference on Human factors in computing systems, 1083–1092.

93. Jacob O. Wobbrock, Andrew D. Wilson, and Yang Li. 2007. Gestures Without Libraries, Toolkits or Training: A $1 Recognizer for User Interface Prototypes. In Pro-ceedings of the 20th Annual ACM Symposium on User Interface Software and Technology (UIST ’07), 159–168.

94. Peter Wright and John McCarthy. 2004. Technology as experience. MIT Press Cambridge, MA.