User research - different approaches and methods

  • Published on
    11-Aug-2014

  • View
    1.878

  • Download
    18

Embed Size (px)

DESCRIPTION

This is the second part of my fourth lecture at HITLab, Canterbury University, Christchurch, New Zealand about user research. I am presenting the three levels of understanding user needs and the methods that correspond with investigating these needs. The idea is to show how different methods enable a designer to dig for different insights and how to conduct exemplary studies for each type of the method.

Transcript

  • user research: dierent approaches and methods aga szstek(at)gmail.com
  • what people level of knowledge research methods Fraukje Sleesvijk Visser
  • what people level of knowledge tell think explicit interviews research methods Fraukje Sleesvijk Visser
  • what people level of knowledge tell think do act explicit observational interviews observations research methods Fraukje Sleesvijk Visser
  • what people level of knowledge tell think do act know feel dream explicit observational subconscious latent deep interviews observations generative techniques research methods Fraukje Sleesvijk Visser
  • before discussing the dierent methods it is crucial to know about the concurrent and retrospective experiences
  • - when people report on their current feelings the feelings themselves are accessible to introspection allowing for accurate reports - aective experiences are eeting and become not available once the feeling is gone - once the feeling disappears the aective experiences need to be reconstructed
  • - when people report on their current feelings the feelings themselves are accessible to introspection allowing for accurate reports - aective experiences are eeting and become not available once the feeling is gone - once the feeling disappears the aective experiences need to be reconstructed - reports of past feelings are based on semantic knowledge - when asked how theyusuallyfeel during a particular activity, people draw on their general beliefs about that activity - the actual experience does not gure prominently in these reports because the experience itself is no longer accessible - the same knowledge is a basis for predicting future feelings, for which episodic information is not available to begin with
  • so, lets discuss the methods
  • what people level of knowledge tell think explicit interviews research methods
  • introspective versus retrospective methods
  • introspective methods Analysis of present behaviour and gathering of preferences by combining observations and interviews in context: - cognitive walkthorugh: heuristic identication of problems connected to the current ways of behaviour and interaction - think aloud: heuristic identication of problems arising from the current state of interaction through verbal articulation of user observations while interacting with the solution - co-discovery: a method where two users collaborate in order to resolve the problems found in the process of interaction with the current solution - Contextual Inquiry: gathering of detailed information about behaviour and motivations of users while the user is actually performing a given task
  • retrospective methods Support for retrospective assessment of user preferences towards a given solution: - interviews: a discussion with the user aiming to collect rich material about his / her interactions and attitudes - focus groups: group discussion about a given problem or solution - retrospective analysis of behaviour: and interview supported by a qualitative recoding of user behaviour (e.g., video or audio recordings)
  • when to apply? - exploration phase - to get a better understanding - to nd inspiration for further research - requirements validation - to check whether study assumptions are correct - evaluation - to discuss and verify design decisions
  • types of studies
  • - fully structured: the entire study takes place according to the earlier prepared scenario - good for comparison across individuals - more extensive answers comparing to written surveys - inability ask unplanned questions
  • - fully structured: the entire study takes place according to the earlier prepared scenario - good for comparison across individuals - more extensive answers comparing to written surveys - inability ask unplanned questions - semi-structured: there are certain guidelines to the elements of the study but the order is unimportant - pre-specied questions - questions as a starting point for discussion - digression from the script is acceptable
  • - fully structured: the entire study takes place according to the earlier prepared scenario - good for comparison across individuals - more extensive answers comparing to written surveys - inability ask unplanned questions - semi-structured: there are certain guidelines to the elements of the study but the order is unimportant - pre-specied questions - questions as a starting point for discussion - digression from the script is acceptable - unstructured: going with the ow, only the initial question and a possible list of topics is dened - good for open ended exploration - dicult to conduct and analyse
  • study questions - close-ended - simple questions get simple answers - On a scale of 1-10, 10 being best, how did you like the prototype? - Did you like the prototype? - easy to analyse, but may not be that informative - open-ended - invite elaboration and discussion - build conversation and stories - What did you think about the prototype? - avoid - long questions, jargon, leading questions, unconscious biases, negative questions
  • - ask why? - even when you think you know the answer, ask people why they do or say things - encourage stories - whether or not the stories people tell are true, they reveal how they think about the world - look for inconsistencies - sometimes what people say and what they do are dierent. These inconsistencies often hide interesting insights - observe nonverbal cues - be aware of body language and emotions. - do not be afraid of silence - if you allow for silence, a person can reect on what theyve just said and may reveal something deeper
  • preparing the study - brainstorm questions - identify and order themes - pilot the study - write an interview guide - prepare for capturing the interview - prepare logistic backup
  • conducting the study - you are the host - be friendly, respectful and non-judgmental - do not bias through body language - be exible (unless fully structured) but keep things on track - ask for clarications if something is unclear - listen (dont talk too much) and read between the lines - be consistent - outline - briey introduce testing goals - complete paperwork (informed consent) - simple questions rst, hard questions later
  • capturing the study - have someone to help taking notes or do the audio or video recording - capture insights, non-verbal responses, etc. - balance between detail and conciseness - summarize immediately after the interview - do transcriptions - respect privacy and anonymity
  • debrieng - ask for any nal comments - provide more detail about research goals - oer a brief summary of ndings - turn o recording devices - saythanks! - reect and summarize notes immediately
  • what people level of knowledge tell think do act explicit observational interviews observations research methods
  • - roots in ethnographic research - observing people in action - recording what is being observed - helping the researcher learn perspectives held by participants - often used in conjunction with other methods (e.g. interviews, focus groups, content analysis)
  • - observations: observing and recording of user behaviour without a direct contact between the user and the researcher - shadowing: direct observation of user behaviour by the researcher without intervening with his / her actions - interaction logging: logging previously dened behaviours and system states that pertain to the specied user behaviour - diaries: systematic reporting by the user his / her interactions in a form of a diary or a blog - experience sampling: a systematic way of having participants provide samples of their ongoing behavior when participants' reports are dependent on either a signal, pre-established intervals or the occurrence of some event - day reconstruction method: capturing daily experiences of the users at the end of each day where participants are asked to listed all activities of the day that somehow related to the product or a situation that is under investigation
  • data collection - eld notes - video - audio - transcripts - behaviour matrix - situational drawings
  • pros and cons - exible and open-ended - directly measures behavior - gives the researcher insider view - allows for morphing of the study - hard to be an insider - invasive and intrusive - dicult to record - subjective - Hawthorne eect - time consuming - not generalizable - does not measure cognitive or aective aspects
  • phases of observational research - phase 1: develop positive relationships with participants, gatekeeper, etc. - phase 2: improve design, refocus, redene questions after initial observations (may be ongoing) - phase 3: select additional participants as necessary - phase 4: follow up and probe deeper
  • ethical considerations - how much do you disclose? - make your intentions clear - get an informed consent - develop a code of practice - maintain condentiality
  • (Mack, et al., 2005)
  • being an eective observer - know what you are researching - check in with your lens, biases, experiences and expectations -- have a blank mind - go where people are engaged in their daily lives and nd the right informants - understand the culture - rehearse how you will explain the purpose of your study - decide how you will do the observation and collect the notes - document what you observe without expectations - summarize and expand on eld notes as soon as possible - be open to the research problem reformulation
  • what people level of knowledge tell think do act know feel dream explicit observational subconscious latent deep interviews observations generative techniques research methods
  • using design as a research tool
  • cultural probes
  • - tools for better understanding of peoples latent needs and dreams - small craft objects used in empathic engagements with individuals around issues centered on self- identity and personal signicance - their form should reect the research question and its context - asking questions by applying a delicate, provoking and creative means - and oering intriguing ways to consider the asked questions and to look for surprising answers
  • understanding what people do with their invoices AgaSzstek,MarcinPiotrowski,
  • what is agood and a bad invoice?
  • what happens to an invoice once it arrives?
  • what is the best way to pay an invoice?
  • how should the email accompanying an invoice look like?
  • which elements should actually stay on an invoice?
  • what should happen if you forgot to pay?
  • technology probes
  • - simple, exible, adaptable interactive installations - combining collecting data about the use of the technology in a real-world setting, testing the technology and inspiring users (and designers) to think of new kinds of technology - installed in user environment with the goal to observe them over a period of time - helping to identify design partners determining which kinds of technologies would be interesting to pursue - open-ended, inspiring new activities and encouraging experimentation
  • managing interruptions | project supervised by Aga Szstek
  • dealing with stress at work
  • design probes
  • - collaboration with a singular person based on a very deep engagement in the topic that is personally signicant - designing objects, which reect individual experiences and their meaning for the user - objects that support reection and auto-reection - such objects often become a part of an intimate relationship between the designer and the user
  • Top Trumps probe cards to mark objects which are perceived to have magical power
  • helping to keep a straight posture | Magda Rydygier project supervised by Aga Szstek
  • preparing the study - brainstorm the context to the given problem (e.g. through contextmapping exercises) - brainstorm on user expectations and the ideal experience - prepare a list of questions regarding the current experience - decide on the format of the cultural probe (digital elements, physical element and...