1

Closing the gap between research and practice

World Usability Day - November 11, 2010 - Dayton (area)

Keith Instone

keith <year> @ instone.org@keithinstone

linkedin.com/in/keithinstonefacebook.com/keithinstone

instone.org

“development”“design”“user

research”“scientificresearch”

2

#UXRPI

3

Welcome to World Usability Day 2010!

Theme: Communication

4

2009

5

2008

6

2007

7

2006

8

2005

9

2005 web site

10

About me: Professional background

• BGSU > Computer science > Research associate > HCI > Hypertext > Web usability > Information architecture > User experience (practitioner)

11

About me: IBM

• IBM > Transformation > CIO > Workforce & Web > Enterprise Solutions > User experience

12

Last time in this building

13

Get on with it!http://www.youtube.com/watch?v=l1YmS_VDvMY

14

My current “community service” focus…

Researcher – Practitioner Interaction

with purposeful over-use of “interaction”

…instead of UXnet, conference organizing, local groups, …

15

(My) goals for this session

• I am by no means an expert– I suspect many people in this room have been

addressing this for a lot longer than me

• Facilitator – get you talking about it, then hopefully doing something about it

• And I am hear to learn from you!

16Did you do your homework?http://instone.org/wud-uxrpi-prework

If you are a user experience practitioner, what types of challenges do you face often that you wish you had a "scientific" answer to?

• Have you tried to find answers in the research literature? • What roadblocks did you encounter when looking for

answers? • What successes have you had in taking research

findings and improving your practice?

17Did you do your homework?http://instone.org/wud-uxrpi-prework

If you are a researcher, what is the value in engaging with practitioners? What is in it for you?

• Do you have any examples of success stories, where your research got better because of interactions you had with practitioners?

18Did you do your homework?http://instone.org/wud-uxrpi-prework

What should students of HCI, interaction design and other user experience disciplines be taught about research to better prepare themselves for the practitioner world?

19

Trend? Fad? Good time to revisit?

• Start: How do we make the CHI 2010 conference better for user experience practitioners?

• CHI conference bug / feature: the HCI research part• Answer: Organize a workshop CHI 2010 workshop

• At same time: IA Summit session• Since then: IUE “panel”, Norman in interactions• Demarcating UX (Dagstahl)

• How do they all fit together? Dunno. So I volunteered to give a talk to force me to figure it out.

20

CHI 2010 workshop (SIG)

• This workshop explores whether problems exist between HCI researchers and practitioners who are consumers of research– Yes!

• Articulate factors that may render the research literature inaccessible or irrelevant to practitioners and to suggest potential improvements and approaches – Some

• Learn from researchers how their research could benefit from practitioner input – Not so sure yet

21

Workshop questions

• How can the usefulness of research papers be improved to suit varied audiences?

• How should research be disseminated to different audiences, including practitioners?

• What are the barriers that discourage practitioners from adopting research findings?

• How can collaboration between the two sub-communities be increased for future CHI conferences?

• What should students of HCI and interaction design be taught about research, to prepare them for the practitioner world?

22Snippets from CHI position papers

(or, what people who are smarter than me say)

• Arnie Lund– Biz sharing with research: “This is too secret for MSR to see!”– Research collaborating with biz: “Who would get credit for idea?”

• John Karat– Research papers are for the research community to evaluate

(not really for practitioners anyway)

• Kath Straub– HFI’s Research Update Newsletter: Yes, practitioners

understand that recent research can speak to relevant issues

• Nigel Bevan– Trying: UPA Usability Body of Knowledge– Works: Research-Based Web Design & Usability Guidelines

(produced with US Government funding)

23

Research culture“Publish or perish”

Answers narrow questionsOpen sharing

Experimentation

Corporate culture“Produce or perish”

Wants broad answersStrategic advantage

Fear of failure

HCI research culturePublish for researchers,

not for practitionersExpanding field

Status within academia

UX practice cultureNo time for “research”:

good enoughRapidly evolving practiceStatus within corporations

Immovable objects

24

Research culture“Publish or perish”

Answers narrow questionsOpen sharing

Experimentation

Corporate culture“Produce or perish”

Wants broad answersStrategic advantage

Fear of failure

HCI research culturePublish for researchers,

not for practitionersExpanding field

Status within academia

UX practice cultureNo time for “research”:

good enoughRapidly evolving practiceStatus within corporations

EducationHard to teach students when practice evolving quickly

Amateur practitionersEconomic model for training

Need more research into how to practice

KnowledgeNo shared knowledge base

Hard to organize research for practical useMulti/inter-disciplinary research & practice

CommunicationLittle shared language

Speed-of-operation differencesFinding each other

Fragmented professional organizationsMapping “research answers” and “practical questions”

Bridges: the challenges

WUD 2010 theme!

25

26

27

Information architecture: RPI challenges

• Research and practice in IA are fractured– With most resources devoted to practice– One-way ride from university to practice– You cannot live in both worlds

• Need an IA research agenda…• …Or, there is no such thing as IA research by itself:

there is HCI, IS, or LIS research that IAs care about• Many people still confuse “project research” with

“scientific research”• Can find existing research, hard to make sense of it

28

If Don Norman is talking about it, …

• “The research-practice gap: The Need for Translational Developers” - interactions, July/August 2010

• See also Kolko (Sep-Oct) “On academic knowledge production”

29

A few “problem statement” quotes from Don

• Immense gap between research and practice– Gap is fundamental– Knowledge and skill sets required of each group differ

• Science: hypotheses, conclusions, and evidence

• Practices (of most professions): scientific links are tenuous at best– Reliance upon “best practice”

30

Pasteur: fundamental research aimed at solving important applied problems

Researchers most often play in the fun quadrant, finding lovely problems to work on without regard for whether anyone cares outside of their fellow research in-group

Inventors such as Thomas Edison fit the quadrant of searching for relevant knowledge to solve an applied problem, but without any attempt to expand our general understanding of phenomena

A third quadrant is filled with tinkerers who produce inventions that neither add to fundamental understanding nor have any use

Appl

icab

ility

Aimed atsomepracticalproblem

Pure science. No application in mind

Knowledge

Search for new understanding Apply existing knowledge

31A new kind of practitioner (new discipline):

the translational developer

– Translate between the abstractions of research and the practicalities of practice– Translating research findings into the language of practical development and

business – Translating the needs of business into issues that researchers can address

Abstractions of research

Research findings

Issues to address with research

Practicalities of practice

Language of development and business

Needs of business

“stop pretending that researchers and practitioners speak the same language” (Nov / Dec recap)

32

Mark Newman comments

• Translational skills– What are the translation skills, who has them, how to

motivate them? Lucrative publishing/consulting career a la Nielsen?

• Translational research (at NIH)– Big pots of money DO motivate researchers to shift

their focus. Would it work for HCI?

33

Susan Weinschenk

What Makes Them Click

Applying Psychology to Understand How People Think, Work, and Relate

34

Danielle Cooley (a practitioner)

To contrast with this, Mark Ackerman (U of M SI) drew:

“one head, two bodies” (researcher body, teacher body)

35

Derek Poppink

36

Why this is important (Susan Weinschenk)

• Research is power in UX practice – Too many "urban legends" that people base their

designs on– Rather than say, "I think that" about a design question

you can say, "A study by xxx showed that..., and therefore we should do ....."

• There is a lot of misunderstanding by practitioners about research– "I need research on XYZ. Why aren't you talking

about that?" (There isn't any research to talk about!)– "That research is from 2003. It's too old". (If it's valid

research it doesn't necessarily lose truth with age).

37

Why this is hard (Susan, Mark)

• Research papers are hard to read and interpret – Often poorly written, not have results that are practical – Results cannot be or should not be generalized

• It takes a special (weird?) person– To be able to read and understand the papers and

even more special to LIKE doing it• The best papers may be outside of your field: and each

has its own research style– “Scientific" model is only one; qualitative social

science is another– Even researchers have a hard time with this stuff!!

• Research may not give you answers– Best research results in more questions than answers

38Demarcating User eXperience (workshop)

September 15-18

• UX is seen as a holistic concept covering all aspects of experiencing a phenomenon, but we are facing the point where UX has become a concept too broad to be useful in practice.

• Practitioners have difficulties to understand the concept and to improve UX in their work, and researchers rather use some other term to make their research scope clear.

• Net problem: Do not even have a clear core concept to bridge the gap between research & practice

• Goal: Write a “white paper” to define UX for a certain set of cases that make its scope clear

39White paper so far…

http://www.dagstuhl.de/Materials/index.en.phtml?10373

• (35 definitions of “user experience”….)• Describes the core concepts of user experience and

distinguishes UX from related concepts• Rooted in use• Subjective• Dynamic (changes over time)• Designing for a desired experience (and evaluation to

see how close we are getting)

• Elizabeth Buie: "If the practitioners can't use it [the white paper], it doesn't work!"

40

PAUSE

41

42

Ideas for solutions

• <fill in this page>

43

CHI 2011

• Editing “benefits” statements for the program– Add “practitioner take-aways” to the program

• Awards: best case study, impact for research paper

• Community leadership– Design– Engineering– Management– User Experience– Health– Games and entertainment– Sustainability– Child-computer interaction

44

Research culture?

Corporate cultureImprove UX status

HCI research cultureGrand challenges

UX practice culture?

EducationTraining for managers

KnowledgeSIGCHI collaborate on UPA BoK

CommunicationBrokerage system to connect R & P with

common interestsTranslators between R & P

Face-to-face meetings (to start buildinglong-term relationships)

Social media

Ideas / Solutions(very incomplete)

45

46

47

(New) Usability BoK

• Living reference that represents the collective knowledge of the usability profession

• Derived from published literature, conference proceedings, and the experiences of practitioners (guide)

• Some goals (related to UXRPI):– Define the knowledge underlying the usability

profession (methods, knowledge, and skills)– Facilitate professional development, curriculum

development • Users: practitioners, educators & researchers, …

48

Next steps?

• Comment on Norman’s interactions article• Tweet something with #uxrpi• Draw / post sketch to Flickr – tag uxrpi• Volunteer @ usabilitybok.org• More details / links at instone.org

• Pick an idea from today’s discussion: let’s try it!

• Contact me: keith <year> @ instone.org - @keithinstone