User research - different approaches and methods

<ul><li> user research: dierent approaches and methods aga szstek(at)gmail.com </li> <li> what people level of knowledge research methods Fraukje Sleesvijk Visser </li> <li> what people level of knowledge tell think explicit interviews research methods Fraukje Sleesvijk Visser </li> <li> what people level of knowledge tell think do act explicit observational interviews observations research methods Fraukje Sleesvijk Visser </li> <li> what people level of knowledge tell think do act know feel dream explicit observational subconscious latent deep interviews observations generative techniques research methods Fraukje Sleesvijk Visser </li> <li> before discussing the dierent methods it is crucial to know about the concurrent and retrospective experiences </li> <li> - when people report on their current feelings the feelings themselves are accessible to introspection allowing for accurate reports - aective experiences are eeting and become not available once the feeling is gone - once the feeling disappears the aective experiences need to be reconstructed </li> <li> - when people report on their current feelings the feelings themselves are accessible to introspection allowing for accurate reports - aective experiences are eeting and become not available once the feeling is gone - once the feeling disappears the aective experiences need to be reconstructed - reports of past feelings are based on semantic knowledge - when asked how theyusuallyfeel during a particular activity, people draw on their general beliefs about that activity - the actual experience does not gure prominently in these reports because the experience itself is no longer accessible - the same knowledge is a basis for predicting future feelings, for which episodic information is not available to begin with </li> <li> so, lets discuss the methods </li> <li> what people level of knowledge tell think explicit interviews research methods </li> <li> introspective versus retrospective methods </li> <li> introspective methods Analysis of present behaviour and gathering of preferences by combining observations and interviews in context: - cognitive walkthorugh: heuristic identication of problems connected to the current ways of behaviour and interaction - think aloud: heuristic identication of problems arising from the current state of interaction through verbal articulation of user observations while interacting with the solution - co-discovery: a method where two users collaborate in order to resolve the problems found in the process of interaction with the current solution - Contextual Inquiry: gathering of detailed information about behaviour and motivations of users while the user is actually performing a given task </li> <li> retrospective methods Support for retrospective assessment of user preferences towards a given solution: - interviews: a discussion with the user aiming to collect rich material about his / her interactions and attitudes - focus groups: group discussion about a given problem or solution - retrospective analysis of behaviour: and interview supported by a qualitative recoding of user behaviour (e.g., video or audio recordings) </li> <li> when to apply? - exploration phase - to get a better understanding - to nd inspiration for further research - requirements validation - to check whether study assumptions are correct - evaluation - to discuss and verify design decisions </li> <li> types of studies </li> <li> - fully structured: the entire study takes place according to the earlier prepared scenario - good for comparison across individuals - more extensive answers comparing to written surveys - inability ask unplanned questions </li> <li> - fully structured: the entire study takes place according to the earlier prepared scenario - good for comparison across individuals - more extensive answers comparing to written surveys - inability ask unplanned questions - semi-structured: there are certain guidelines to the elements of the study but the order is unimportant - pre-specied questions - questions as a starting point for discussion - digression from the script is acceptable </li> <li> - fully structured: the entire study takes place according to the earlier prepared scenario - good for comparison across individuals - more extensive answers comparing to written surveys - inability ask unplanned questions - semi-structured: there are certain guidelines to the elements of the study but the order is unimportant - pre-specied questions - questions as a starting point for discussion - digression from the script is acceptable - unstructured: going with the ow, only the initial question and a possible list of topics is dened - good for open ended exploration - dicult to conduct and analyse </li> <li> study questions - close-ended - simple questions get simple answers - On a scale of 1-10, 10 being best, how did you like the prototype? - Did you like the prototype? - easy to analyse, but may not be that informative - open-ended - invite elaboration and discussion - build conversation and stories - What did you think about the prototype? - avoid - long questions, jargon, leading questions, unconscious biases, negative questions </li> <li> - ask why? - even when you think you know the answer, ask people why they do or say things - encourage stories - whether or not the stories people tell are true, they reveal how they think about the world - look for inconsistencies - sometimes what people say and what they do are dierent. These inconsistencies often hide interesting insights - observe nonverbal cues - be aware of body language and emotions. - do not be afraid of silence - if you allow for silence, a person can reect on what theyve just said and may reveal something deeper </li> <li> preparing the study - brainstorm questions - identify and order themes - pilot the study - write an interview guide - prepare for capturing the interview - prepare logistic backup </li> <li> conducting the study - you are the host - be friendly, respectful and non-judgmental - do not bias through body language - be exible (unless fully structured) but keep things on track - ask for clarications if something is unclear - listen (dont talk too much) and read between the lines - be consistent - outline - briey introduce testing goals - complete paperwork (informed consent) - simple questions rst, hard questions later </li> <li> capturing the study - have someone to help taking notes or do the audio or video recording - capture insights, non-verbal responses, etc. - balance between detail and conciseness - summarize immediately after the interview - do transcriptions - respect privacy and anonymity </li> <li> debrieng - ask for any nal comments - provide more detail about research goals - oer a brief summary of ndings - turn o recording devices - saythanks! - reect and summarize notes immediately </li> <li> what people level of knowledge tell think do act explicit observational interviews observations research methods </li> <li> - roots in ethnographic research - observing people in action - recording what is being observed - helping the researcher learn perspectives held by participants - often used in conjunction with other methods (e.g. interviews, focus groups, content analysis) </li> <li> - observations: observing and recording of user behaviour without a direct contact between the user and the researcher - shadowing: direct observation of user behaviour by the researcher without intervening with his / her actions - interaction logging: logging previously dened behaviours and system states that pertain to the specied user behaviour - diaries: systematic reporting by the user his / her interactions in a form of a diary or a blog - experience sampling: a systematic way of having participants provide samples of their ongoing behavior when participants' reports are dependent on either a signal, pre-established intervals or the occurrence of some event - day reconstruction method: capturing daily experiences of the users at the end of each day where participants are asked to listed all activities of the day that somehow related to the product or a situation that is under investigation </li> <li> data collection - eld notes - video - audio - transcripts - behaviour matrix - situational drawings </li> <li> pros and cons - exible and open-ended - directly measures behavior - gives the researcher insider view - allows for morphing of the study - hard to be an insider - invasive and intrusive - dicult to record - subjective - Hawthorne eect - time consuming - not generalizable - does not measure cognitive or aective aspects </li> <li> phases of observational research - phase 1: develop positive relationships with participants, gatekeeper, etc. - phase 2: improve design, refocus, redene questions after initial observations (may be ongoing) - phase 3: select additional participants as necessary - phase 4: follow up and probe deeper </li> <li> ethical considerations - how much do you disclose? - make your intentions clear - get an informed consent - develop a code of practice - maintain condentiality </li> <li> (Mack, et al., 2005) </li> <li> being an eective observer - know what you are researching - check in with your lens, biases, experiences and expectations -- have a blank mind - go where people are engaged in their daily lives and nd the right informants - understand the culture - rehearse how you will explain the purpose of your study - decide how you will do the observation and collect the notes - document what you observe without expectations - summarize and expand on eld notes as soon as possible - be open to the research problem reformulation </li> <li> what people level of knowledge tell think do act know feel dream explicit observational subconscious latent deep interviews observations generative techniques research methods </li> <li> using design as a research tool </li> <li> cultural probes </li> <li> - tools for better understanding of peoples latent needs and dreams - small craft objects used in empathic engagements with individuals around issues centered on self- identity and personal signicance - their form should reect the research question and its context - asking questions by applying a delicate, provoking and creative means - and oering intriguing ways to consider the asked questions and to look for surprising answers </li> <li> understanding what people do with their invoices AgaSzstek,MarcinPiotrowski, </li> <li> what is agood and a bad invoice? </li> <li> what happens to an invoice once it arrives? </li> <li> what is the best way to pay an invoice? </li> <li> how should the email accompanying an invoice look like? </li> <li> which elements should actually stay on an invoice? </li> <li> what should happen if you forgot to pay? </li> <li> technology probes </li> <li> - simple, exible, adaptable interactive installations - combining collecting data about the use of the technology in a real-world setting, testing the technology and inspiring users (and designers) to think of new kinds of technology - installed in user environment with the goal to observe them over a period of time - helping to identify design partners determining which kinds of technologies would be interesting to pursue - open-ended, inspiring new activities and encouraging experimentation </li> <li> managing interruptions | project supervised by Aga Szstek </li> <li> dealing with stress at work </li> <li> design probes </li> <li> - collaboration with a singular person based on a very deep engagement in the topic that is personally signicant - designing objects, which reect individual experiences and their meaning for the user - objects that support reection and auto-reection - such objects often become a part of an intimate relationship between the designer and the user </li> <li> Top Trumps probe cards to mark objects which are perceived to have magical power </li> <li> helping to keep a straight posture | Magda Rydygier project supervised by Aga Szstek </li> <li> preparing the study - brainstorm the context to the given problem (e.g. through contextmapping exercises) - brainstorm on user expectations and the ideal experience - prepare a list of questions regarding the current experience - decide on the format of the cultural probe (digital elements, physical element and...</li></ul>