University of Washington HCDE 518 & INDE 545 Design Research HCDE 518 & INDE 545 Winter 2012 With credit to Jake Wobbrock, Dave Hendry, Andy Ko, Jennifer

University of Washington HCDE 518 & INDE 545

Design Research

HCDE 518 & INDE 545Winter 2012

With credit to Jake Wobbrock, Dave Hendry, Andy Ko, Jennifer Turns, & Mark Zachry


Agenda

Announcements Class Activity – Heuristic Evaluation Lecture – Design Research Break – 5 mins Lecture & Discussion – A research project Discussion Questions – Ben & Sarah Next Class


Announcements

R9, A3 due next Monday P4 (demo & report) and P5 (presentation) on

Wednesday

Questions?


Class Activity: Heuristic Evaluation

Electronic voting machine Download prototype:

http://courses.washington.edu/hcde518/lectures/12-AccuvoteWithPrinter.swf

Download form: http://courses.washington.edu/hcde518/lectures/12-HeuristicEvalForm.xlsx

Nielsen’s heuristics: http://www.useit.com/papers/heuristic/heuristic_list.html

Use form and Nielsen's heuristics to evaluate the voting interface in groups of 2-3


Design research vs. Design practice

Credit to Andy Ko, who also created the sketches


What have we been doing so far?

Mostly design practice, with some support from research on design methods

Learning to think like a designer


What is design thinking?

There are at least seven things…1.explicit problem reformulation2.divergent/convergent thinking 3.exploitation of failure4.externalization of ideas5.emotional distance from ideas6.group critique of ideas7.articulation of rationale


Problem Formation & Perspective Taking

Design is inherently about changing existing situations to preferred ones.

But how one formulates this preference affects the solutions we consider. For example, consider these… How can we make lectures more efficient? How can we spend less time in class? How can we class time more productive?


Convergent & Divergent Thinking Our natural tendency is to judge; we see

something wrong and we point it out (or we think it, at the very least)

A critical skill is being able to defer judgment, and leave yourself space to consider alternatives that may appear unworkable at first glance

When divergent thinking is interrupted by critique, we stop seeking relationships between ideas and cease combining ideas


Acceptance & Exploitation of Failure

By generating hundreds of possible solutions to a problem, even solutions that are risky or unfavorable, one can learn a great deal about what makes one particular idea effective generate alternatives expect failure utilize failure


Externalization of Ideas

When we keep ideas in our minds, we are inherently limited by our capacity to imagine details

Ideas can be externalized as writing, drawings, or other forms

The sketching of ideas facilitates a dialog between designers and their decisions by forcing them to express details, often revealing which parts of an idea are still ill- or undefined


Emotional Distance from Ideas

It’s easy to fall in love with our own ideas: we invest our time and our passions into them and become invested

But there’s no greater hindrance to quality than emotional attachment to an idea

It prevents us from seeking feedback, from listening to feedback, and from exploring superior alternatives


Group Critique of Ideas Ideas need diverse perspectives to evolve One way to obtain these perspectives is

through group critique, where designers present externalizations of their ideas and solicit feedback.

Group critiques help designers question their assumptions, but they also help synthesize feedback, by allowing people to build on each others’ critiques and to learn from each other’s struggles


Articulation of Rationale

While externalizing one’s ideas creates the opportunity for critique, it does not guarantee it

A central part of design thinking is justifying one’s design choices

The act of explaining decisions forces you to evaluate your decisions and identify what evidence, principles, or knowledge to used to motivate the choice


What ties these all together?

Theory of Mind

“Theory of mind is the ability to attribute mental states—beliefs, intents, desires, pretending, knowledge, etc.—to oneself and others and to understand that others have beliefs, desires and intentions that are different from one's own” [Premack, D. G. & Woodruff, G. 1978]

Related to empathy


Theory of Mind + Design Thinking to formulate a problem, you have to view the world from several people’s

perspectives to defer judgment, you have to entertain multiple perspectives on a

problem to exploit failure, you have to assess your ideas independent from you to externalize ideas, you have to imagine them from someone’s

perspective to keep distant from your ideas, you have to discount your perspectives

on it in favor of users to use critiques, you have to view opinions from other people’s

perspectives to articulate rationale, you have to imagine details from other people’s

perspectives


Implications

What can you do to be more empathetic? learn to listen expose yourself to other cultures judge yourself more than you do others reflect on your work as you work

The more you do these things, the better able you’ll be to solve other people’s problems


If that’s design practice, what’s design research?


What’s the difference between...

Bank of America designing a new mobile banking application designs it builds it deploys it maintains it

Intent is to provide a reliable, usable, desirable service (and make $$$)

A Ph.D. student designing a new mobile application designs it builds it deploys it ?

why would a Ph.D. student do this?


To learn something about human behavior

Designs can facilitate the answering of research questions that were previously unanswerable how do people manage their money? what does daily knowledge of net assets do to spending

behavior? how do different visualizations of spending behavior affect

anxiety about retirement? how does a banking application that criticizes people’s

spending affect spending behavior? how do software developers manage coordinate their

work relative to deadlines


Example: ShutEye How can we get people to think more about the things that

impact their sleep? We designed a simple mobile app that ran on the background of

people’s phones Studied system use for 4 weeks, interviewed people before and after

to assess their understanding of sleep factors Also did assessments of whether people’s sleep improved


To learn something about designing systems

Attempting to design something that has never been built before can result in generalizable knowledge about how systems can be designed a banking application that predicts spending in a novel way might teach us how

to predict other types of behaviors a mobile app that is has a customizable user interface may teach us how to

make other kinds of application’s user interfaces customizable

a mobile app with novel security features might

result in security features applicable to other

applications


Example: Lullaby How can we design a system that might help correlate

people’s sleep environment to their sleep quality? My PhD student Matt Kay and I designed and built an entire

application to answer this question The application itself had to coordinate multiple sensor streams, figure

out how to mitigate privacy concerns, not disrupt people’s sleep We are finding that people are identifying aspects of their sleep they

hadn’t previously realized


Research = generalizability?

For design to be research, perhaps the activity must have implications beyond the design itself new knowledge new possibilities


Research = reporting?

What if you never describe it to anyone? There could be 1,000’s of new discoveries

bout humans and technology lurking inside Google

If they never report on these, are they research?


Research = reflecting?

What if you describe your design, but you don’t analyze it? Google engineers could explain the design behind

Google Books, but what would we know about it, other than how to recreate it?

What do we know about its merits and limitations relative to other media?


The design in design research is secondary

The point of designing something in a research context is rarely to create an actual thing the goal is to produce knowledge about how to create similar things or to enable the investigation of questions about people and the natural world However, designing things in research often has a the nice

benefit of creating useful and/or novel things People can actually use Lullaby or ShutEye to improve their sleep Both of these could be turned into commercial products and make

money But it’s usually not researchers who do it


Three ways of knowing (of many) qualitative methods help us observe causality through

rigorous, holistic, subjectivity the observer is the instrument, designing the process of observation

quantitative methods help us test causality through rigorous, narrow, objectivity We design objective measurements to quantify

design methods help us manipulate causality through rigorous, specific interventions the object is a probe, changing behavior, enabling us to observe or

measure change


What would make your designs research?

Did your designs reveal new knowledge? Did they teach you how to design similar

things? Does this knowledge generalize? Did you report this knowledge? Did you reflect on how this changes the

knowledge we had previously?


BREAK – 5 MINUTES


Discussion Questions - Ben 1. Is it allowed for a design team to make up their own Heuristics? If so

what are the benefits? Draw backs? (In other words ask users to look for specific issues that they anticipate or recognize)

2. When following a heuristic, people can say if it breaks it. When is the best time to ask them for advice on how to make it better, inline or at the end?

3. What other applications see such a drastic law of diminishing returns (after 5 people...) it seems like a pretty logarithmic decline.

4. How do you handle disagreements between evaluators? 5. Nielson indicates that a good evaluator – someone who easily

identifies user issues – is just as likely to find a hard to find “hard to find” issues as a “poor” evaluator, why do you think this is?


Discussion Questions - Ben 6. Of the 10 Heuristics listed by Nielsen what do you think major

companies do the least, why? 7. As an engineer the notion that performing the same experiment twice

and not drawing the same conclusion being a good thing (according to Forlizzzi) is difficult to swallow. Do you agree that a good HCI study should concentrate on relevance rather that validity?

8. Where do you think a traditional designer is more useful in HCI in the front end development or back end usage?

9. When performing a Heuristic evaluation how do you focus a participant away from creeping featurism and toward the design as it is. Especially in the hi fidelity proto-type stage.

10. Is it better to ask your users to categorize their comments according to Nielsons severity ratings, or based on your interpretation of the extent of the issue they highlighted to bin them yourself (as the designer)?


Discussion Question - Sarah Nielsen provides severity ratings for usability problems. Often

these are subjective based on the evaluator. Nielsen recommends using multiple evaluators to avoid this. Can you think of a way to make this less subjective?

Are there any gaps in the model provided by Forlizzi et al? Nielsen provides the ten usability heuristics. These were

developed before touch screens. Can you think of any guidelines that need to be added in order to effectively evaluate technology on touch screens?

According to Forlizzi et al. interaction design research in HCI is changing. How will this affect the field of HCI?


Next Class Topics

Monday, March 5th

Trends in UCD Discussants: Stephan & Sarah

Wednesday, March 7th

Final project presentations & demos Course evals

Upcoming Work R9, A3


Group Project time

Documents

University of Washington HCDE 518 & INDE 545 Design Research HCDE 518 & INDE 545 Winter 2012 With credit to Jake Wobbrock, Dave Hendry, Andy Ko, Jennifer