Workshop: Mindful Moderating

Moderating Qualitative ResearchFebruary 2, 2012

AgendaDay One 9:00-9:30 Intro to Qualitative Research 9:30-10:00 Screener Creation

Discussion 10:00-10:15 BREAK 10:15-12:00 Creating Protocols

Workshop 12:00-1:00 Lunch 1:00-2:30 Questioning and Probing 2:30-3:00: BREAK 3:00-4:30: Interview Workshop using

Protocols 4:30-5:00 Discussion

2

What is Qualitative Research?

Explore issues Understand

phenomena Answer questions

3

The Basics

4

Developed in the social and human sciences

Typically involves gathering empirical materials using some form of observation or interviewing method.

Why Qualitative Research Works

5

More engaging than quantitative research.

More answers through probing ("Help me understand why you feel that way") to reach beyond initial responses and rationales.

The opportunity to observe, record and interpret non-verbal communication (i.e., body language, voice intonation) as part of a respondent’s feedback, which is valuable during interviews or discussions, and during analysis.

The opportunity to engage respondents in "play" such as projective techniques and exercises, overcoming the self-consciousness that can inhibit spontaneous reactions and comments.

In groups, building on (each other’s) comments and ideas.

Qualitative vs. Quantitative

6

The aim is a complete, detailed description.

Researcher may only know roughly in advance what he/she is looking for.

Recommended during earlier phases of research projects.

The design emerges as the study unfolds.

Researcher is the data gathering instrument.

Data is in the form of words, pictures or objects.

Subjective – individuals’ interpretation of events is important, e.g., uses participant observation, in-depth interviews etc.

Data is more 'rich', time consuming, and less able to be generalized.

Researcher tends to become subjectively immersed in the subject matter.

The aim is to classify features, count them, and construct statistical models in an attempt to explain what is observed.

Researcher knows clearly in advance what he/she is looking for.

Recommended during latter phases of research projects.

All aspects of the study are carefully designed before data is collected.

Researcher uses tools, such as questionnaires or equipment to collect numerical data.

Data is in the form of numbers and statistics.

Objectively seeks precise measurement & analysis of target concepts, e.g., uses surveys, questionnaires etc.

Quantitative data is more efficient, able to test hypotheses, but may miss contextual detail.

Researcher tends to remain objectively separated from the subject matter.

Qualitative Quantitative

Qualitative and Quantitative

7

Qualitative research involves analysis of data such as words (e.g., from interviews), pictures (e.g., video), or objects (e.g., an artifact).

Quantitative research involves analysis of numerical data.

The strengths and weaknesses of qualitative and quantitative research are a hot debate.

The personality / thinking style of the researcher and/or the culture of the organization influences choice.

Focus on how the techniques can be integrated.

So, what’s your role?

8

Some people differentiate…

9

Goal: Keep the information and communication flow clear and accessible to all who participate.

Role: Information Manager

Keep the session on track, assure all questions/activities are completed and making sure everyone shares their perspectives.

In an online environment, s/he monitors the communication flow, makes summaries and digests, approves participants’ requests and posts, and even maintains the online environment.

Quite invisible.

Goal: steers the communication flow and keeps it on track.

Role: Leader of Discussion

S/he focuses on including all participants in the discussion, and using their responses to move the discussion forward. S/he keeps it vibrant, interesting and useful to those who participate.

The facilitator enables a comfortable and inclusive environment of openness and trust for those who participate.

Visible and active.

Moderating Facilitating

This is your role.

10

The qualitative researcher is a detective in search of clues as to motivations, desires, beliefs, ways of thinking and words for description.

A qualitative researcher must look for absence as well as presence - what are and where are the gaps, difference as well as similarity - what types of people are more in tune with this idea, paradoxes as well as consistencies - what explains why people say one thing and do another.

The aim is to draw out the vital clues that explain behavior and attitudes by developing a discussion - getting people to talk through in their own words how they see a particular idea or subject or product area.

But it’s not easy.

11

The researcher has to be aware of his or her role in the discussion.

The researcher can turn and bias the discussion with just one word or introducing one idea, but ignore clues and key words and the conversation is just a regurgitation of other people's marketing messages.

The researcher must be sharp enough to probe beneath the surface of a subject and to challenge and tease out the contradictions. Is this really what you believe or are you just mouthing someone else's words?

But that’s why you are here.

12

Every company is doing qualitative research.

Your job is to have better questions and techniques, which gives us better insight.

Think of it this way…

13

A good qualitative researcher is someone who listens, then thinks, then asks the probing question to get to the next level of the conversation.

It's no good to have the inspiration for the perfect question back in the office once the interview or group is complete.

Is this personality related? Is this a social barrier?

Are there underlying issues about image? Or profit? Or guilt?

Is this an emotional decision or is it purely rational?

Does it involve other people and in what way?

Does the message communicate and at which levels?

Is this an instinctive response or a learnt response?





14

It all starts with talking to the right people.

15

16

Market Segmentation

17

Market Segmentation

18

Isolate traits that distinguish a group from the overall market and that could impact behavior

Geographic

Urban vs. suburban

US or International

Releavance (Home Depot in city, lawncare in city, taxis in suburbia)

Demographic

Gender, Age, Ethnicity, Income, Occupation, Education, Household size, stage in the family life cycle.

Psychographic

Values, lifestyles, way of thinking (needs, motives, perceptions, attitudes)

http://www.strategicbusinessinsights.com/vals/presurvey.shtml

Product-related



Psychographic Profiles

19


20


21

Screener Discussion

22

Do’s and Don’ts

How to word questions

Questions to avoid

Questions to include

Where screeners are going next?





23

Welcome Back.http://www.youtube.com/watch?v=3SuNx0UrnEo

24





25

Creating Protocols• Start from Scratch• Create a storyboard for the flow of the conversation• Know/write down the goal of the question• Pilot and revise

26

What are you looking for?

27

How people conceptualize a market/product

The factors (conscious and unconscious) that influence decisions

How people feel about concepts or products

WHY?

How can you find it?

28

Ethnography (Contextual Inquiry, Shop Along)

Focus Groups (Dyad and Triads)

Interviews (1:1, phone, onsite, lab)

Immersive Research (Diary studies)

https://revelationwebinar.webex.com/ec0605ld/eventcenter/recording/recordAction.do;jsessionid=25J8PJLWG4hn0YZHQwht5Mxxp09JdsQ71whWcDGQ7Gw1hq39zKX1!-99200423?theAction=poprecord&actname=/eventcenter/frame/g.do&actappname=ec0605ld&renewticket=0&renewticket=0&apiname=lsr.php&entappname=url0107ld&needFilter=false&&isurlact=true&rID=4295507&entactname=/nbrRecordingURL.do&rKey=8243ac5f247aa3db&recordID=4295507&siteurl=revelationwebinar&rnd=0436604936&SP=EC&AT=pb&format=short

Types of Qualitative Research

29

Observational & Ethnographic Research Observe

Natural environment

Over time


30

Focus Groups An interactive group discussion led by a moderator

Unstructured (or loosely structured) discussion where the moderator encourages the free flow of ideas

Usually 8 to 12 members in the group who fit the profile of the target group or consumer but may consist of two interviewees (a dyad) or three interviewees (a triad) or a lesser number of participants (known as a mini-group)

Usually last for 1 to 2 hours

Can use computer and internet technology for on-line focus groups

Respondents feel a group pressure to conform

Group dynamics is useful in developing new streams of thought and covering an issue thoroughly


31

In-Depth Interviews Interview is conducted one-on-one, and lasts between 30 and 60 minutes

Best method for in-depth probing of personal opinions, beliefs, and values

Very rich depth of information

Very flexible

Probing is very useful at uncovering hidden issues

They are unstructured (or loosely structured)- this differentiates them from survey interviews in which the same questions are asked to all respondents

Can be time consuming and responses can be difficult to interpret

Requires skilled interviewers - expensive - interviewer bias can easily be introduced

There is no social pressure on respondents to conform and no group dynamics

Start with general questions and rapport establishing questions, then proceed to more purposive questions

Laddering to start with questions about external objects and external social phenomena, then proceed to internal attitudes and feelings

Hidden issue questioning is a technique used by depth interviewers in which they concentrate on deeply felt personal concerns and pet peeves

Symbolic analysis is a technique used by depth interviewers in which deeper symbolic meanings are probed by asking questions about their opposites


32

Diary Studies Users record their experiences

Can be offline or online

Responses are natural

Probing is retrospective

Case Study 48 total people

24 moms of toddler girls 24 moms of toddler boys Equally split as Carter’s and OshKosh

Mix of ethnicity, definitely include Hispanic Mix of Expectant Mom (first time mom) vs. experienced moms Online shoppers (younger moms – 25-34) Brand fans (multi channel, multibrand, buy 4 or more times a year) Prospects (Sarah’s, Tina’s, Paula’s, Brittany’s who are not our

customers but should be)

33

Goals of studies Consumer tastes – what is their style sense / this is best captured by having moms

post photos of their child’s closet and post comments about why certain items have high repeat “wearage” vs. low repeat “wearage”

What are moms’ pain points about the daily routine – getting child dressed in the morning (easiest/hardest outfits), potty training, getting child into PJs, making the getting ready process easier as the child grows, building the child’s interest in their clothes –and how can we improve garment functionality and style to make her life easier

What they like and dislike about different types of apparel in Carter’s and OshKosh – PJs, body suits, jeans, overalls, dresses, T’s, outerwear, accessories

How they would compare our merchandise vs. the competition (this could best be explored by having them compartmentalize their closet by brand and talk about “best in class” types of apparel and where they bought it (a video diary or posting snapshots and comments could work here)

For ongoing new product development testing – garment features, color palette, design aesthetic, garment functionality –

34





35

Welcome Back. Again.http://www.youtube.com/watch?v=Gv8pmIr3a7k&feature=related

36





37

The most important things you do as a qualitative researcher is probe and ask questions to get insights.

38

Techniques

39

Questions

Sorting things into categories

Creating collages

Drawing

Writing/Diaries

Show and tell

Personification

Observations

Free/Word association

Sentence Completion

Techniques

40

Story Completion

Cartoon bubble Completion

Thematic Apperception (Story Creation)

Third-person technique

Rating Scales

Moderator’s Accomplice (in groups) as Devil’s advocate

Role playing

Why…

41

Figuring out Why is why we do research. Sometimes, it’s easy. Not everything is subterranean and difficult to extract.

Price Availability Status

Sometimes beliefs are so engrained that more determined, assertive probing is needed to expose how participants really behave and why.

Question Types

42

A closed question usually receives a single word or very short, factual answer. For example, "Are you thirsty?" The answer is "Yes" or "No"; "Where do you live?" The answer is generally the name of your town or your address.

Open questions elicit longer answers. They usually begin with what, why, how. An open question asks the respondent for his or her knowledge, opinion or feelings. "Tell me" and "describe" can also be used in the same way as open questions. Here are some examples:

What happened at the meeting?

Why did he react that way?

How was the party?

Tell me what happened next.

Describe the circumstances in more detail.

Open questions are good for:

Developing an open conversation: "What did you get up to on vacation?"

Finding our more detail: "What else do we need to do to make this a success?"

Finding out the other person's opinion or issues: "What do you think about those changes?"

Question Types

43

Closed questions are good for:

Testing your understanding, or the other person's: "So, if I get this qualification, I will get a raise?"

Concluding a discussion or making a decision: "Now we know the facts, are we all agreed this is the right course of action?"

Frame setting: "Are you happy with the service from your bank?” Direct questions: “Have you ever received money for good grades? When you mention competition, do you then think of a sportsmanlike or a destructive competition?”

Indirect questions: Projective questions such as ‘How do you believe other pupils regard the competition of grades?”

Structuring questions: indicating when a theme is exhausted by breaking off long irrelevant answers: “I would now like to introduce another topic:…”

Introducing questions: “Can you tell me about….?”, “Do you remember an occasion when…?” “What happened in the episode mentioned?”,…

Specifying questions: “What did you think then?” What did you actually do when you felt a mounting anxiety?”, “How did your body react?”,…

Question Types

44

Free Association

Ok, so when I say Cadillac, what does that mean to you?

What’s the first word (or 3 words) that come to mind when you see this?

Asking for an example or interpretation

“Can you tell me about the last time you suffered from pain? What did you do? How did you manage it?”

“You then mean that….?” “Is it correct that you feel that…?”Does the expression…. Cover what you have just expressed?”

Funnel Questions

This technique involves starting with general questions, and then homing in on a point in each answer, and asking more and more detail at each level.

When using funnel questioning, start with closed questions. As you progress through the funnel, start using more open questions.

Question Types

45

Leading Questions

Leading questions try to lead the respondent to your way of thinking. They can do this in several ways:

With an assumption: "How late do you think that the project will deliver?". This assumes that the project will certainly not be completed on time.

By adding a personal appeal to agree at the end: "Lori's very efficient, don't you think?" or "Option 2 is better, isn't it?"

Phrasing the question so that the "easiest" response is "yes" (our natural tendency to prefer to say "yes" than "no" plays an important part in the phrasing of referendum questions): "Shall we all approve Option 2?" is more likely to get a positive response than "Do you want to approve option 2 or not?". A good way of doing this is to make it personal. For example, "Would you like me to go ahead with Option 2?" rather than "Shall I choose Option 2?".

Giving people a choice between two options, both of which you would be happy with, rather than the choice of one option or not doing anything at all. Strictly speaking, the choice of "neither" is still available when you ask "Which would you prefer of A or B", but most people will be caught up in deciding between your two preferences.

Note that leading questions tend to be closed.

Question Types

46

Retrospective

Come back to it later (and get rephrased feelings)

“So earlier someone mentioned they had visited Yahoo in the past. Let’s talk about that again. What were you after again? Why were you annoyed again?”

Probing Questions

Asking probing questions is another strategy for finding out more detail. Sometimes it's as simple as asking your respondent for an example, to help you understand a statement they have made. At other times, you need additional information for clarification, "When do you need this report by, and do you want to see a draft before I give you my final version?", or to investigate whether there is proof for what has been said, "How do you know that the new database can't be used by the sales force?"

An effective way of probing is to use the 5 Whys method, which can help you quickly get to the root of a problem.

Use questions that include the word "exactly" to probe further: "What exactly do you mean by fast-track?", "Who, exactly, wanted this report?

Probing Techniques

47

Repeating they key word(s) as a question (nondirective probing)

“Cadillac and Mercedes?”

“Sad?”

“That makes you feel sad?”

Asking for help:

“I don’t understand…”

“Can you help me understand…”

“I’m sorry, can you repeat that?”

Being direct

What do you mean by XXX?

You seem troubled. Why?

Probing Techniques

48

Pushing them slightly:

“Would you really?”

“Are you sure that is what you would do?”

“Is that all that matters?”

“I want to raise a question about something that has me puzzled…I’m not trying to persuade you, but several of you said you buy ovens to cook turkeys, but no one here has cooked a turkey within the last two years. Why is that important then?”

Using their words to show a theme/discrepancy

“Is that a change from what you said earlier?”

“I heard you say Cadillac and Mercedes. Why those/Are there others/Why not another?”

Providing an opinion they can agree or refute:

“That seems like a change from what you said earlier.”

“We’ve heard some others say the opposite. What do you think about that?”

Probing Techniques

49

Acknowledge and move beyond

Imagine we are in “super secure world” where security is not an issue. Then, where and when do you think you would use your iPad?

Silence

3 seconds

With or without eye contact

Make it concrete

Ok so when was the last time you did XXX

What if the patient was 25? What about 60?

Deprivation

If you were skiing and the resort only stocked one kind of beer, what would it be

If you could only use one search engine on your phone, which one would you choose?

(But remember, this isn’t realistic or that accurate of the marketplace)

Common Probes

50

Facial Expression

Eyebrow raise (Surprised)

Furrow eyebrows (Confused)

Nod slowly (Understand but maybe agree)

Be careful not to communicate surprise, rejection or shock

When to Probe

51

Almost always!

When attitudes are not fully formed

When they are providing trite/superficial responses

If you don’t fully understand their opinion

Watch outs for participants

52

Behaviors are driven by unconscious, so questions aren’t enough?

Why do you drink Coke?

People are really good at inventing reasons, especially to mask laziness, greed, ignorance, the importance of appearance and image, group affiliation, etc.

Watch outs for you

53

Leading questions

Wouldn’t you use the product if it was proven to be effective?

It’s a nice looking site… “You mean the color?” (vs. what do you mean by nice looking?)

Long questions

If you take a breath, it’s too long

Complex questions

Ask one at a time

Ask 2-3 then break them down into each and re-ask

Be specific

How interested are you in this product vs.

Would you buy it?

Watch outs for you

54

In groups: let others comment before probing if at all possible (following up to each comment ruins the group dynamic as they talk to you, not each other)

Try not to ask why, seek why

“What are the reasons for…?”

“I don’t understand…help me out.”

“What made you do that?”

“What really happened?”

“Because…”

“Could you please explain more?”

The Moderator

55

What matters?

Age?

Gender?

Race?

Personality?

Authoritarian Seductive Entertaining Too personal Intimidating

What makes a great moderator?

56

If the moderator can keep the discussion on topic with all panelists contributing more or less equally, that would make the moderator a pretty good moderator, but not a great one. A great moderator introduces tension into the discussion – tension in the form of controversies, contradictions, and conflicts.

Controversies. Almost all topics have their controversies, and a good and easy way to create tension is for the moderator to ask the panelists on their view of the latest controversies. “Would if be a good idea for Singapore’s ruling political party to blog?” I might ask a panel discussing blogging in Singapore. The best controversies will bring out different views from different panelists.

Contradictions. It takes a sharp moderator to catch the contradictions made by the panelists. “You mentioned X just now, but now you seem to be saying Y. Could you clarify this?” or, “you once mentioned X in your blog, but now you’re saying Y…” The panelist who self-contradicts is not giving the audience enough respect, especially if that panelist hopes to get away it. The audience will appreciate the moderator who catches this, and soon the moderator will be seen as the star of the show. Wait, did I just contradict myself about being the star?

Conflicts. It’s hard to catch panelists contradicting themselves, but it’s much easier to catch them contradicting one another, giving the moderator a chance to create conflict. “Panelist A mentioned X just now. What is your view on X?” Good panelists (and dominant ones as well) will create conflict on their own, but quieter panelists would need a little more prodding. Again, conflict helps to add more views to the discussion, besides creating tension. Of course, the amount of conflict should be controlled, even though an escalation to physical conflict may make the panel a lot more memorable for the audience. So far, I’ve never encountered very serious (or exciting) conflicts during panel discussions. Unless you count the part when the audience gets to ask questions.


57

Sensitivity to the feelings of individuals: Creating and maintaining an atmosphere of trust and respect requires an awareness of how people are responding to both the topics under discussion and the opinions and reactions of others. Most people will not articulate their discomfort, hurt feelings, or even anger; instead they silently withdraw from the discussion and often from the group. Sensing how people are feeling and understanding how to respond to a particular situation is a critical skill of facilitation.

Sensitivity to the feeling of the group: In any group, the whole is greater than the sum of the parts, and group "chemistry" generally reflects shared feeling: eager, restless, angry, bored, enthusiastic, suspicious, or even silly. Perceiving and responding to the group's dynamic is essential to skillful facilitation.

Ability to listen: One way the facilitator learns to sense the feelings of individuals and the group is by acute listening, both to the explicit meaning of words and also to their tone and implicit meaning. In fact, facilitators generally speak less than anyone in the group. And often the facilitator's comments repeat, sum up, or respond directly to what others have said.

Tact: Sometimes the facilitator must take uncomfortable actions or say awkward things for the good of the group. The ability to do so carefully and kindly is critical. Furthermore the subject matter of human rights can evoke strong feelings and painful memories. The facilitator needs particular tact in dealing with emotional situations respectfully and sometimes also firmly.


58

A sense of timing: The facilitator needs to develop a "sixth sense" for time: when to bring a discussion to a close, when to change the topic, when to cut off someone who has talked too long, when to let the discussion run over the allotted time, and when to let the silence continue a little longer.

Flexibility: Facilitators must plan, but they must also be willing to jettison those plans in response to the situation. Often the group will take a session in an unforeseen direction or may demand more time to explore a particular topic. The facilitator needs to be able to evaluate the group's needs and determine how to respond to it. Although every session is important, sometimes a facilitator will decide to omit a topic in favor of giving another fuller treatment.

Resourcefulness and creativity: Each group is as different as the people who make it up. A good facilitator needs an overall program and goals but may also adapt it to fit changing conditions and opportunities. For example, the facilitator may call on the talents and experiences of people in the group and the community, or participants may suggest resources.

How to handle participants

59

Shy

Know-it-all

Disengaged

Dominator (in groups)

Threatening/Rude/Condescending

Not the right profile





60

Hang In There & Grab a Coffee!

3.0http://www.youtube.com/watch?v=Gv8pmIr3a7k&feature=related

Masterlockhttp://www.youtube.com/watch?v=FxbV1IlvjSw

61

http://www.youtube.com/watch?v=Gv8pmIr3a7k&feature=related






Workshop 12:00-1:00 Lunch 1:00-2:30 Questioning and Probing 2:30-3:00: BREAK 3:00-4:30: Interview Workshop

using Protocols 4:30-5:00 Discussion

62





63

Documents

Workshop: Mindful Moderating