Interaction Models I
Marti Hearst (UCB SIMS)SIMS 213, UI Design &
DevelopmentMarch 9, 1999
Last Time: Heuristic Evaluation
A “discount” usability testing method UI experts inspect an interface
– prototype initially– later, the full system
Check the design against a list of design guidelines or heuristics
Present a list of problems to the UI designers and developers, ranked by severity
See homework assignment (Due Mar 18)
Outline
Windowing systems Events overview Event dispatching and handling For more information, see Olsen 98, Developing User
Interfaces
Windowing Systems
Provide infrastructure to support common UI-related services
Provide functionality for– Input/Output device handling– Which window gets which
input/output Window manager
»create and organize windows» implement interaction in those windows
Adapted from slide by James Landay
Windows Top level windows known as root
windows– provide an abstraction to separate
applications– windowing system arbitrates resources
among these Each root window belongs to an app.
– all descendant windows belong to same application
– not used by OLE (ActiveX)
Adapted from slide by James Landay
Networked Windowing Systems
XWindows designed to allow applications to run on remote machines
Uses client-server model– (X reverses standand usage of client/server terminology)
X Serverstd system software
Clientapp software
Network
User
Adapted from slide by James Landay
XWindows Note backwards terminology
– User is on “server” not “client” X Server
– interprets X commands and can send events– determines which window receives events
and forwards over network to proper client X Client
– software interface to X (Xlib)– assembles the output from Xlib routines into
packets for transmission to server
Adapted from slide by James Landay
Events An event is something “interesting” that
happens in the system– Mouse button goes down– Item is being dragged– Keyboard button was pressed
Adapted from slide by James Landay
Window Events User interacts with input device
– action translated into software events– must be distributed to the
appropriate window Events contain information on
– type– mouse position or character key– the window the event is directed to
Adapted from slide by James Landay
Input Events
Mouse button events– depress/release mouse key– modifier (shift keys, etc.)– double click (X doesn’t have this fakes it)
Mouse movement events– implement painting with mouse– mouse drag
Mouse enter and exit events– e.g. if you entered / exited a button region
Adapted from slide by James Landay
Button Events
Button
Button
mouseenter
mouseexit
(But using mouse move events within a buttonis unnecessary, due to the semantics of buttons.)
Adapted from slide by James Landay
Events (cont.)
Keyboard events– translate raw “scan codes” into ASCII
Adapted from slide by James Landay
Events (cont.) Windowing events on window
– creation / destruction– opening / closing– iconifying / deiconifying– selection / deselection– resize– redraw
» (window manager tells the application to redraw within a certain region; the application does the actual redrawing)
Adapted from slide by James Landay
Widgets that can register specific kinds of eventsclose box
title bar
folder
scroll bar
size control
Adapted from slide by James Landay
Major Issues
How to decompose the UI into interactive objects (widgets)?
How to distribute inputs to the interactive objects?
Contrast– Sequential (standard) program flow– Event-driven program flow
Sequential Programs
Program takes control, prompts for input
Examples include – command-line prompts (DOS, UNIX)– LISP interpreter
The user waits on the program– Program tells user it’s ready for more
input– User enters more input
Adapted from slide by James Landay
Sequential Programs (cont.) The user waits on the program
– 1. Program tells user it’s ready for more input
– 2. User enters more input
– 3. Program responds and goes back to 1. But how to model the many
actions a user can take?– For example, a word processor?– Need to do editing, inserting, etc.
Adapted from slide by James Landay
Event-Driven Programming Instead of the user waiting on program,
have the program wait on the user All communication from user to
computer is done via “events” An event is something “interesting” that
happens in the system– Mouse button goes down– Item is being dragged– Keyboard button was pressed
Adapted from slide by James Landay
Interactor Tree Decompose interactive objects into a tree
– based on screen geometry of objects– use nested rectangles
Used for dispatching events– Events are dispatched (sent) based on code
associated with the widget– The code responds appropriately to the event
Variety of methods for dispatching events– Return to this later
Adapted from slide by James Landay
Interactor Tree
Display Screen
“F:\cs160\Public” window Inner Window title bar
horizontal scroll bar contents area
“CDJukebox” folder“Home Ent…” folder…
size control …
“Web Newspaper” window…
Adapted from slide by James Landay
Interactor Tree
Display Screen
Outer Win [white]
7 8 9
4 5 6
0 + -
1 2 3
=
93.54
ENT
?????
Adapted from slide by James Landay
Interactor Tree
Display Screen
Outer Win [white]
Result Win [tan]Result String
Inner Win [green]
Keypad [grey]
- button+ button0 button
= button
7 8 9
4 5 6
0 + -
1 2 3
=
93.54
ENT
Adapted from slide by James Landay
Main Event Loop
Main event loopInitialization
While (not time to quit) {
Get next event E
Dispatch event E
}
The meat of the program is in the code that handles the “dispatch”
Adapted from slide by James Landay
Event Queues
Input events are placed in a queue– Ensures events are processed in order
Main event loop removes them from the queue (get-next-event) & dispatches for processing
Mouse move (22, 33)Mouse move (40, 30)Mouse down left (45, 34)Mouse up left (46, 35)
Adapted from slide by James Landay
Event Queues (cont.)
Can use ignore unwanted events– e.g., ignore mouse moves in a forms-
based program» just get enter/exit events
– however, do want to track mouse moves in a drawing program
Adapted from slide by James Landay
Event Dispatch
Dispatch (event E) { switch (E.window) { ... case FIVE-KEY:
if (E.type == left-down){ cur = 5 + 10*cur; display (cur); last-op = NUMBER; } ...
7 8 9
4 5 6
0 + -
1 2 3
=
0
ENT
Hit the ‘5’ key
Adapted from slide by James Landay
Event Dispatch
Dispatch (event E) { switch (E.window) { ... case TWO-KEY:
if (E.type == left-down) { cur = 2 + 10*cur; display (cur); last-op = NUMBER; } ...
7 8 9
4 5 6
0 + -
1 2 3
=
5
ENTHit the ‘2’ key
Adapted from slide by James Landay
Event Dispatch
Dispatch (event E) { switch (E.window) { ... case ENTER-KEY: if (E.type == left-down){
push (cur); cur = 0; last-op = COM; }
...
7 8 9
4 5 6
0 + -
1 2 3
=
52
ENT
Hit the ‘enter’ key
Adapted from slide by James Landay
Event Dispatch
Dispatch (event E) { switch (E.window) { ... case SIX-KEY:
if (E.type == left-down) { cur = 6 + 10*cur; display (cur); last-op = NUMBER; } ...
7 8 9
4 5 6
0 + -
1 2 3
=
0
ENT
Hit the ‘6’ key
Adapted from slide by James Landay
Event Dispatch
Dispatch (event E) { switch (E.window) {
...case PLUS-WIN: if (E.type == left-down){
if (last-op == NUMBER)push (cur);
result = pop() + pop();push (result);
display (result);cur = 0;last-op = COM;
}
7 8 9
4 5 6
0 + -
1 2 3
=
6
ENTHit the ‘+’ key
Adapted from slide by James Landay
Dispatching Events
If user scrolls the text, the software must:– direct the mouse events to the scroll bar– update the scroll bar display during the drag– notify the text editing window it needs to
scroll itself so that the text appears to have moved
Adapted from slide by James Landay
Dispatching Events (cont.) Algorithm selects the bottom-most, front-
most region in the interactor tree Called bottom-first event dispatching
– scroll bar or contents over outerwin (bottom-most)– scroll bar over contents (front-most)– Advantage:
» each window need only consider its own events» simple
– disadvantage:» difficult to impose a high level of control» inflexible
Adapted from slide by James Landay
Dispatching Events (cont.) Top-down event dispatching
– events passed to top-most, front-most window
– it dispatches to one or more of its children
Adapted from slide by James Landay
Event Focus Where should keyboard events go?
– mouse-based»attach mouse position to all key events
and dispatch events in the same way as mouse events
– click-to-type »send all keyboard events to last window
where mouse-down occurred
Adapted from slide by James Landay
Event Focus Mouse focus
– scrollbar example: retain the focus on the scrollbar widget until the mouse button is released
– compensates for difficulty of keeping the mouse within the narrow scrollbar
Adapted from slide by James Landay
Event Handling Event tables (in the early days…)
– indexed by event types (integer from 0 - 255)– hold pointers to functions that handle each event– one table per / window– lots of things to maintain when attached to a
widget that you want to make reusable Callbacks
– each kind of widget defines a set of named callbacks which will be run when the widget receives an appropriate event
Adapted from slide by James Landay
Callback Example How do we notify text window to
scroll when the scroll bar is moved?– create a vertical scroll bar widget– write a callback procedure which has code
to notify text windows of their new position– register the callback as being the program
to invoke when the scroll bar is moved– register the text window as the data for the
callback» so the system knows which window to scroll
Summary
Windowing systems– software to manage and arrange windows
» keeps track of current focus
– software to support handling of user-created events
» complex GUIS are built up of hierarchically nested windows
Events– associated with various types of input
devices and actions– are handled one by one from a queue