Analysing textual data from a qualitative perspectiveUSING NVIVO TO ANALYSE TEXTUAL DATA

DR NATHANIEL OWEN

18TH OCTOBER 2016

This session1. Background

2. Ongoing research agenda

3. Research Questions

4. Use of NVivo to explore the data

5. Current findings

The Busuu Project• Run by Fernando Rosell-Aguilar

• http://www.open.ac.uk/education-and-languages/main/busuu

• Focuses on how language learners use apps to support their learning experience

• Study looks at practices and beliefs of language learners who use the Busuu app

The Busuu Project: Data collection• Online survey distributed to Busuu users

• Consists of 30 questions.

• Questions 1-3 collect demographic data. Questions 4-30 focus on app use to support learning

• Self-selecting sample of 4,095 app users responded to the questionnaire

• The questionnaire was written in both Spanish and English to target users from those two countries. 55.3 per cent (n=2,265) respondents were from the UK with 44.7 percent (n=1,830) from Spain.

• Sample was even with respect to gender

The Busuu Project: Age of respondents

646

1380

854

472335

200127

0

200

400

600

800

1000

1200

1400

1600

Under 18 18-25 26-35 36-45 46-55 56-65 Over 65

The Busuu Project: Qualitative data17. What features do you like LEAST in the Busuu app exercises? (Select up to 3 answers)

• Vocabulary practice

• Grammar practice

• Reading practice

• Writing practice

• Listening practice

• Translation practice

• Feedback on your writing from other members of the Busuu community

• Correcting other members’ writing in your own language

• Other (please specify)

The Busuu Project: Qualitative data19. Do you post your writing exercises for comment

by other users?◦ Yes◦ No

• Why?

Provisional research questions for analysis of qualitative data1. What aspects of the Busuu app do users not like? ◦ 1a. To what extent is this related to age and gender?

2. Do the app users post writing exercises for other users to judge?◦ 2a. To what extent is this related to age and gender?

The Busuu Project: Qualitative data (snapshot)Respondent Q1 Q2 Q17_a

1 1 6I like the listening activity but would like a quiz for understanding rather than the fill in the blanks with words I hear.

2 0 1Some courses require premium. I believe there should be ads which can be removed after a small fee. People shouldn't have to pay for content.

3 1 0 You need premium to do most things

4 1 6 None

5 1 1 I don't know

6 1 1 Often I find it difficult to express myself with large phrases, as I am a beginner at spanish

7 1 3 Having to pay for features that are FREE on other apps

8 1 6 always asking me to upgrade

9 0 5 Navigating the site isn't always obvious

10 0 4 none

The Busuu Project: Qualitative data• Data comes ‘pre-transcribed’

• All participants who responded to Q17 and Q19 textual parts were identified (n=114). Data converted to word document with participant labels as ‘headings’.

• NVivo recognises word headings and can convert them to ‘nodes’.

• N.B. Prepare data carefully before importing to NVivo

• Text/participants imported into NVivo as an INTERNAL SOURCE

The Busuu Project: Qualitative data (snapshot)11

I like the listening activity but would like a quiz for understanding rather than the fill in the blanks with words I hear.

21

Some courses require premium. I believe there should be ads which can be removed after a small fee. People shouldn't

have to pay for content.

29

You need premium to do most things

37

I don't know

70

Having to pay for features that are FREE on other apps

71

always asking me to upgrade

81

Navigating the site isn't always obvious

Coding qualitative data with quantitative categorical variables• CLASSIFICATIONS in NVivo

allows you to apply characteristics to participants.

• Create and define characteristics manually:

• Import your data form the external spreadsheet you are working from: EXTERNAL DATA > CLASSIFICATION SHEETS

NVivo text searchWord Count

Weighted Percentage (%)

like 25 1.91app 20 1.53get 17 1.30

yet 17 1.30know 15 1.15premium 14 1.07feedback 13 0.99learn 13 0.99want 13 0.99nothing 12 0.92busuu 11 0.84exercises 11 0.84just 11 0.84language 11 0.84practice 11 0.84native 10 0.76pay 10 0.76

NVivo text searches• Limited data per participant; utility of text searches

for this type of data

• QUERY > TEXT SEARCH

• Q17 – what do you like least about the app?• Pay OR paid OR upgrade OR premium OR free

• These can then be manually coded into a new node

Some findings for Q17…Variable Concerned about paid content

Gender Male 12

Female 14

Age Under 18 8

18-25 6

26-35 2

36-45 5

46-55 3

56-65 2

65+ 1

Example comments“Some courses require premium. I believe there should be ads which can be removed after a small fee. People shouldn't have to pay for content”

“Having to pay for features that are FREE on other apps”

“always asking me to upgrade”

“The fact that they claim it is free but you need premium to learn things as basic as ‘estar’. I would pay a one off fee but never a subscription”

Some findings for Q19

Reasons given Male Female

No Negative statements 12 9

Unreadiness 8 3

Yes Positive statements 17 17

Feedback 10 10

Interaction with native speakers 3 5

Do you post your writing exercises for comment by other users• Yes• NoWhy?

Example comments (negative)“Because I have trouble spelling.”

“I'm not ready for that yet.”

“Haven't gotten to the point where I am doing the writing exercises yet.”

“I'm not good with criticism/correction.”

“I am not sure why. Non one ever writes anything.”

“I like studying by myself.”

“Didn't know you could.”

Example comments (positive)“To attain feedback from native speakers which helps with pronunciation and writing style.”

“The integration of human interaction is good and spurs social and educational relationships.”

“Rarely, but for personal practice.”

“In the hope that a native speaker will give feedback.”

“Interesting to learn what comments different native language speakers have.”

“I really appreciate the feedback. Also it forces me to practice.”

I don't have any friends so why why should I share?

Findings (still a work in progress)• Males/females broadly similar in their enthusiasm for the feature which allows them

to upload examples of their writing…• However, males are more likely to express worry that their language level is insufficient to take

advantage of this feature.

• Major reason given for appreciation of this feature is receiving feedback from native speakers.• Both males and females are equally enthusiastic about the possibility of receiving native

speaker feedback.

• The major feature that users did not like was the limited features on the free version of the app.• Users were put off by regular subscriptions instead of a one-off payment.

• Users wanted the quiz element of the app to be freely available.

• 18-25 age group least concerned about payment (relative to sample size).

• Under 18 group most concerned about payment (relative to sample size).

Caveats (for questionnaires)• No limits on what people can write; answers will vary

dramatically• “Because it's a very useful feature, it's more personal than a

computer, which makes it easier to reflect on your mistakes.”

• “I don't know.”

• Wide age range reflected in the comments:• “I don't have any friends so why should I share?”

• “Why should I?”

Final comments (for NVivo)• Good points…• NVivo can be used successfully for analysing questionnaire

data and for projects which have a mixed-methods design• Automated features in NVivo excellent for linking qualitative

and quantitative data

• Caveats…• Use of automated features in NVivo still requires manual

inspection of coding• Data must be carefully formatted before exporting to NVivo

Thank you!

Any questions?