User evaluation of language learning apps: a case study

CALRG 2016 conference

Fernando Rosell-Aguilar @FRosellAguilar

Senior Lecturer in Spanish / Open Media FellowSchool of Languages and Applied Linguistics, The Open University

Apps for language learning• New market since 2007 (iPhone) + iPad 2010• 1 billion apps downloaded by June 2015 on iTunes

alone. • Potential for ubiquitous learning opportunities, ease of

use, collaborative potential, content access, productivity (Murphy, 2011)

• Fits with informal and lifelong learning; learning objects; CALL; MALL theories and practices of just in time teaching.

Apps for language learning• “Super powers” (Sweeney & Moore, 2012)

– Screen size– touch screen– enhanced text entry– high-quality image, audio and video playback, recording, editing and

sharing– voice recognition– storage– Fast connectivity– Geolocation– Bluetooth connection(Goodwin-Jones, 2011)

Apps for language learning

• Criticisms– Often “designed by developers with little knowledge of

language learning” (Moore, 2011)– Often replicate what’s been done before: “pedagogically

nothing new has been done with smartphones that has not already been done with earlier mobile devices” (Burston, 2014)

– Most apps focus on cognitive processes and receptive skills (Kim & Kwon, 2012). Little socio-cognitive activity or collaborative learning.

– Have we gone 20 years back to behaviourist CALL?• This is changing.

Apps for language learning

• Research: • Wide ownership, enjoyable (Khaddage & Latemann,

2013) • Some indications of language learning improvement:

• Reading (McLanahan et al, 2012; Harmon, 2012)• Listening (Lys, 2013, Kim, 2013)• Vocabulary (Yildiz, 2012)• Verb conjugation (Castañeda & Cho, 2013)• Character writing (Shinagawa, 2014, Rosell-Aguilar & Kan, 2014)

Apps for language learningStudies into the use of apps for language learning have mostly focused on small samples of learners using a specific app rather than using the apps they have selected for themselves. More data is required to understand the ways learners engage with apps for language learning to create a realistic picture of users in their natural settings.

Research questions• Are there any differences between app users and app

non-users?Age, gender, devices owned, OS.

• What type of apps do language learners choose and use independently to aid their learning?

• How do learners use apps? How often, for how long, where, and when?

• What are the most and least appreciated features of language learning apps?

• Do learners think the use of apps contributes to their language learning?

MethodsPaper-based questionnaire.

165 adult learners of Spanish. 85 students (51.51%) returned the questionnaire, based on previous pilot study.

7 interviews (5 female and 2 male). 15 minutes long on average and audio recorded with students’ consent.

Use of apps

58 (78.38%) use apps / 16 (21.62%) do not. In pilot study 34.06% of respondents did not use apps.

Device ownership

Device owned

App users App non-users

Smartphone 19 (32.76%) 5 (31.25%)

Tablet 4 (6.90%) 2 (12.5%)

Both 35 (60.34%) 6 (37.5%)

None 0 3 (18.75)

App non-users N= 16Reasons for not using language learning apps: • Lack of awareness of apps for language learning: 37.50%• Preference for other methods (videos, podcasts, using a

laptop rather than a mobile device), or more ‘traditional’ methods, (e.g. books): 25.00%

• Lack of time: 12.50%• Screen size: 6.25%• 3 respondents (18.75%) responded that the Windows app

store lacked adequate content for language learners.

Device use:

App use:

App use:

App use:

App use:

App use:

Percentage of time spent using apps by gender.

App opinions: liked features48 respondents Most mentioned features: • the ability to focus on specific areas, such as

translation, grammar practice, or vocabulary (22 respondents).

• ease of use (12 respondents)• rapid access to information (8 respondents) • game-like elements (4 respondents)• access to authentic materials (2 respondents)• portability and feedback (1 respondent each).

App opinions: least liked features40 respondents. Most common responses:• issues about app interface and design: cluttered interface,

text size and colours that have poor contrast, too many clicks, and amount and difficulty of typing on a smartphone (8 responses)

• unreliability of content/content errors (7)• lack of grammatical explanations and usage examples (4)• software glitches / freezing (4)• advertising (2)• the decontextualised manner in which content is presented

in some apps (2)• having to pay for content (2)• requirement to be online to use certain apps (2)• poor feedback (1).

App opinions:

App value:

Conclusions:• Participant gender balance: 62.34% female / 37.66% male respondents.

Gender balance among app users: 63.79% male. Suggests men are more likely to use apps.

• Differences in age suggests younger learners are more likely to use apps.

• Decrease by over 10% in percentage of respondents who don't use apps between pilot study (2014) and this study (2015): indicator that more learners are introducing apps into their language learning.

• 80% of app non-users own devices in which apps can be used, so device ownership is not the deciding factor for non-users.

• Vocabulary and translation most popular app types: most widely available types of language apps + type of app that can be used ‘just in time’.

• Grammar apps also very popular: convenient and rewarding to use.• Location where apps are used: most popular answer was “at home” -

raises issues about how mobile the learning experience is.

Further research:• Taxonomy of apps for language learning• Criteria for evaluation of language learning apps• Large-scale study of users of busuu app: 4000+ respondents

Thank you!¡Gracias!

@FRosell-Aguilar

Fernando.Rosell-Aguilar@open.ac.uk

Rosell-Aguilar, F. (in press). User evaluation of language learning mobile applications: a case study with learners of Spanish. In A. Palalas & M. Ally (Eds), The International Handbook of Mobile-Assisted Language Learning. China Central Radio & TV University Press.