This is the second part of my fourth lecture at HITLab, Canterbury University, Christchurch, New Zealand about user research. I am presenting the three levels of understanding user needs and the methods that correspond with investigating these needs. The idea is to show how different methods enable a designer to dig for different insights and how to conduct exemplary studies for each type of the method.
Text of User research - different approaches and methods
user research: dierent approaches and methods aga
szstek(at)gmail.com
what people level of knowledge research methods Fraukje
Sleesvijk Visser
what people level of knowledge tell think explicit interviews
research methods Fraukje Sleesvijk Visser
what people level of knowledge tell think do act explicit
observational interviews observations research methods Fraukje
Sleesvijk Visser
what people level of knowledge tell think do act know feel
dream explicit observational subconscious latent deep interviews
observations generative techniques research methods Fraukje
Sleesvijk Visser
before discussing the dierent methods it is crucial to know
about the concurrent and retrospective experiences
- when people report on their current feelings the feelings
themselves are accessible to introspection allowing for accurate
reports - aective experiences are eeting and become not available
once the feeling is gone - once the feeling disappears the aective
experiences need to be reconstructed
- when people report on their current feelings the feelings
themselves are accessible to introspection allowing for accurate
reports - aective experiences are eeting and become not available
once the feeling is gone - once the feeling disappears the aective
experiences need to be reconstructed - reports of past feelings are
based on semantic knowledge - when asked how theyusuallyfeel during
a particular activity, people draw on their general beliefs about
that activity - the actual experience does not gure prominently in
these reports because the experience itself is no longer accessible
- the same knowledge is a basis for predicting future feelings, for
which episodic information is not available to begin with
so, lets discuss the methods
what people level of knowledge tell think explicit interviews
research methods
introspective versus retrospective methods
introspective methods Analysis of present behaviour and
gathering of preferences by combining observations and interviews
in context: - cognitive walkthorugh: heuristic identication of
problems connected to the current ways of behaviour and interaction
- think aloud: heuristic identication of problems arising from the
current state of interaction through verbal articulation of user
observations while interacting with the solution - co-discovery: a
method where two users collaborate in order to resolve the problems
found in the process of interaction with the current solution -
Contextual Inquiry: gathering of detailed information about
behaviour and motivations of users while the user is actually
performing a given task
retrospective methods Support for retrospective assessment of
user preferences towards a given solution: - interviews: a
discussion with the user aiming to collect rich material about his
/ her interactions and attitudes - focus groups: group discussion
about a given problem or solution - retrospective analysis of
behaviour: and interview supported by a qualitative recoding of
user behaviour (e.g., video or audio recordings)
when to apply? - exploration phase - to get a better
understanding - to nd inspiration for further research -
requirements validation - to check whether study assumptions are
correct - evaluation - to discuss and verify design decisions
types of studies
- fully structured: the entire study takes place according to
the earlier prepared scenario - good for comparison across
individuals - more extensive answers comparing to written surveys -
inability ask unplanned questions
- fully structured: the entire study takes place according to
the earlier prepared scenario - good for comparison across
individuals - more extensive answers comparing to written surveys -
inability ask unplanned questions - semi-structured: there are
certain guidelines to the elements of the study but the order is
unimportant - pre-specied questions - questions as a starting point
for discussion - digression from the script is acceptable
- fully structured: the entire study takes place according to
the earlier prepared scenario - good for comparison across
individuals - more extensive answers comparing to written surveys -
inability ask unplanned questions - semi-structured: there are
certain guidelines to the elements of the study but the order is
unimportant - pre-specied questions - questions as a starting point
for discussion - digression from the script is acceptable -
unstructured: going with the ow, only the initial question and a
possible list of topics is dened - good for open ended exploration
- dicult to conduct and analyse
study questions - close-ended - simple questions get simple
answers - On a scale of 1-10, 10 being best, how did you like the
prototype? - Did you like the prototype? - easy to analyse, but may
not be that informative - open-ended - invite elaboration and
discussion - build conversation and stories - What did you think
about the prototype? - avoid - long questions, jargon, leading
questions, unconscious biases, negative questions
- ask why? - even when you think you know the answer, ask
people why they do or say things - encourage stories - whether or
not the stories people tell are true, they reveal how they think
about the world - look for inconsistencies - sometimes what people
say and what they do are dierent. These inconsistencies often hide
interesting insights - observe nonverbal cues - be aware of body
language and emotions. - do not be afraid of silence - if you allow
for silence, a person can reect on what theyve just said and may
reveal something deeper
preparing the study - brainstorm questions - identify and order
themes - pilot the study - write an interview guide - prepare for
capturing the interview - prepare logistic backup
conducting the study - you are the host - be friendly,
respectful and non-judgmental - do not bias through body language -
be exible (unless fully structured) but keep things on track - ask
for clarications if something is unclear - listen (dont talk too
much) and read between the lines - be consistent - outline - briey
introduce testing goals - complete paperwork (informed consent) -
simple questions rst, hard questions later
capturing the study - have someone to help taking notes or do
the audio or video recording - capture insights, non-verbal
responses, etc. - balance between detail and conciseness -
summarize immediately after the interview - do transcriptions -
respect privacy and anonymity
debrieng - ask for any nal comments - provide more detail about
research goals - oer a brief summary of ndings - turn o recording
devices - saythanks! - reect and summarize notes immediately
what people level of knowledge tell think do act explicit
observational interviews observations research methods
- roots in ethnographic research - observing people in action -
recording what is being observed - helping the researcher learn
perspectives held by participants - often used in conjunction with
other methods (e.g. interviews, focus groups, content
analysis)
- observations: observing and recording of user behaviour
without a direct contact between the user and the researcher -
shadowing: direct observation of user behaviour by the researcher
without intervening with his / her actions - interaction logging:
logging previously dened behaviours and system states that pertain
to the specied user behaviour - diaries: systematic reporting by
the user his / her interactions in a form of a diary or a blog -
experience sampling: a systematic way of having participants
provide samples of their ongoing behavior when participants'
reports are dependent on either a signal, pre-established intervals
or the occurrence of some event - day reconstruction method:
capturing daily experiences of the users at the end of each day
where participants are asked to listed all activities of the day
that somehow related to the product or a situation that is under
investigation
data collection - eld notes - video - audio - transcripts -
behaviour matrix - situational drawings
pros and cons - exible and open-ended - directly measures
behavior - gives the researcher insider view - allows for morphing
of the study - hard to be an insider - invasive and intrusive -
dicult to record - subjective - Hawthorne eect - time consuming -
not generalizable - does not measure cognitive or aective
aspects
phases of observational research - phase 1: develop positive
relationships with participants, gatekeeper, etc. - phase 2:
improve design, refocus, redene questions after initial
observations (may be ongoing) - phase 3: select additional
participants as necessary - phase 4: follow up and probe
deeper
ethical considerations - how much do you disclose? - make your
intentions clear - get an informed consent - develop a code of
practice - maintain condentiality
(Mack, et al., 2005)
being an eective observer - know what you are researching -
check in with your lens, biases, experiences and expectations --
have a blank mind - go where people are engaged in their daily
lives and nd the right informants - understand the culture -
rehearse how you will explain the purpose of your study - decide
how you will do the observation and collect the notes - document
what you observe without expectations - summarize and expand on eld
notes as soon as possible - be open to the research problem
reformulation
what people level of knowledge tell think do act know feel
dream explicit observational subconscious latent deep interviews
observations generative techniques research methods
using design as a research tool
cultural probes
- tools for better understanding of peoples latent needs and
dreams - small craft objects used in empathic engagements with
individuals around issues centered on self- identity and personal
signicance - their form should reect the research question and its
context - asking questions by applying a delicate, provoking and
creative means - and oering intriguing ways to consider the asked
questions and to look for surprising answers
understanding what people do with their invoices
AgaSzstek,MarcinPiotrowski,
what is agood and a bad invoice?
what happens to an invoice once it arrives?
what is the best way to pay an invoice?
how should the email accompanying an invoice look like?
which elements should actually stay on an invoice?
what should happen if you forgot to pay?
technology probes
- simple, exible, adaptable interactive installations -
combining collecting data about the use of the technology in a
real-world setting, testing the technology and inspiring users (and
designers) to think of new kinds of technology - installed in user
environment with the goal to observe them over a period of time -
helping to identify design partners determining which kinds of
technologies would be interesting to pursue - open-ended, inspiring
new activities and encouraging experimentation
managing interruptions | project supervised by Aga Szstek
dealing with stress at work
design probes
- collaboration with a singular person based on a very deep
engagement in the topic that is personally signicant - designing
objects, which reect individual experiences and their meaning for
the user - objects that support reection and auto-reection - such
objects often become a part of an intimate relationship between the
designer and the user
Top Trumps probe cards to mark objects which are perceived to
have magical power
helping to keep a straight posture | Magda Rydygier project
supervised by Aga Szstek
preparing the study - brainstorm the context to the given
problem (e.g. through contextmapping exercises) - brainstorm on
user expectations and the ideal experience - prepare a list of
questions regarding the current experience - decide on the format
of the cultural probe (digital elements, physical element and
intangible elements) - select the criteria for choosing
participants and prepare awards - develop the probes and the
instruction - organize a session with participants to explain to
them the elements of the probe and the study procedure - provide
informed consent forms - personally give the probes to the
participants
running the study - the study typically lasts no longer that
one week (but this is very individual to the research problem) -
usually the study happens without an active involvement of the
researcher - if the study involves an online diary check the
entries every day and comment on them - give participants a phone
number and an email address where they can contact you in the case
of any problems - at the end of the study invite all participates
either to a joint session (a Focus Group or a creative workshop) or
for individual interviews - record the sessions for later
reference
summary - the research method should reect the information you
are trying to obtain - dont be afraid to combine the methods and
experiment with them - think of creating visual materials for your
studies they engage the users bringing you more insight - user
research is a moment when you approach users with open mind try not
to bias them and put your ideas into their mouth - the quality of
the data you collect will likely be directly reected in the ideas
you will create later on
references Gaver, William W., et al. "Cultural probes and the
value of uncertainty." interactions 11.5 (2004): 53-56. Gaver,
Bill, Tony Dunne, and Elena Pacenti. "Design: cultural probes."
interactions 6.1 (1999): 21-29. Mattelmki, Tuuli. Design probes.
Aalto University, 2006. Hutchinson, Hilary, et al. "Technology
probes: inspiring design for and with families." Proceedings of the
SIGCHI conference on Human factors in computing systems. ACM, 2003.
Buijs, Jan. The Delft innovation method: A design thinker's guide
to innovation. Eleven International Publishing, 2012.
