Building the future of autonomous language learning

Martyn Clark, PhD, Carrie Bonilla, PhD, Ewa Golonka, PhD, Jared Linck, PhD, Catherine Doughty, PhD

LanguageNation project update28 July 2015

University of Maryland CASL

LanguageNation Development Partnership

UMD-CASLJared Linck, PhD (Principal Investigator)Catherine Doughty, PhD (Area Director)C. Bonilla, W. Burns, M. Clark, E. Golonka, C. Green, E. Jones, M. Masters, C.A. Rytting, and M. Tare (Researchers)DoDCarol Van Ess-Dykema, PhD (USG, Chief Scientist)Tara Mechum, PhD (USG, Intern)IBMThanks to Salim Roukos, Jiri Navratil, and Todd Ward at IBM for their assistance in obtaining the data from the pilot participants described in this presentation.

2

LanguageNation Vision

• Anywhere, anytime, any device for professionals• Target users – USG learners • Not “canned” lessons, but truly adaptive learning

o Aptitude Profileo Proficiencyo Performanceo Topic of interesto Mission needs

• Autonomous learning (with a human-in-the-loop coach)

3

INTERNET

Learning Activities

Internal database (corpora)

HLT Tools

Performance Tracking

Pretesting

SLA Principles

Learner model

Learning Planner

Activity Generator

Human curators

HLT Tools

SLA Principles

HLT - Automated Materials Generation

SLA –Guided Learner Management System

Front‐end(visible to user)

5

6

Prototype Pilot Study

Proof-of-concept: activity generation for autonomous learning

7

Context of the Pilot Study Two waves of pilot participants from IBM workforce used

LanguageNation prototype between April 2014 –February 2015.

Participants(n=54) were advanced to native Spanish or English speakers with little Portuguese knowledge reported.

Three CASL researchers with advanced Spanish knowledge also piloted the system.

We analyzed data from those that spent the most time. in the system (>7 hours).

8

User time in prototype system

9

Most prototype activities targeted vocabulary

10

11

Prototype vocabulary activity

• Learners are quizzed on the target words• Options are given if the learner needs a hint• Grammatical information is provided with the right answer

This is Marivi.

é a Marivi .(Hint)

12

Prototype automatically targeted high frequency vocab

13

Log Diving

14

The benefit of online learning: Data captureNSECS CTIME UNAME PNAME ATYPE AVAL IVAL FVAL SVAL USER_ID PARAM_IDPARAM_AUSER_PARSVCNAME63535319070 2014‐05‐09‐11.44.30.620566 cbonilla@casl.umd.edu UserEvent event cbonilla@casl.umd.edu user_even 17313 11902 14 37047563535319089 2014‐05‐09‐11.44.49.732260 cbonilla@casl.umd.edu NumConceptChallenged concept makeword‐1 1 17313 22 11 435382 makeword63535319089 2014‐05‐09‐11.44.49.733199 cbonilla@casl.umd.edu NumConceptCorrect concept makeword‐1 1 17313 23 11 435384 makeword63535319089 2014‐05‐09‐11.44.49.733766 cbonilla@casl.umd.edu NumConceptHintDiscount concept makeword‐1 3.33E‐01 17313 24 11 435386 makeword63535319089 2014‐05‐09‐11.44.49.734372 cbonilla@casl.umd.edu NumMakeWordContentCompleted content ‐1@106318 1 17313 19735 12 435609 makeword63535319118 2014‐05‐09‐11.45.18.187791 cbonilla@casl.umd.edu NumConceptChallenged concept makeword‐1 1 17313 22 11 435382 makeword63535319118 2014‐05‐09‐11.45.18.188623 cbonilla@casl.umd.edu NumConceptCorrect concept makeword‐1 1 17313 23 11 435384 makeword63535319118 2014‐05‐09‐11.45.18.189230 cbonilla@casl.umd.edu NumConceptHintDiscount concept makeword‐1 0.00E+00 17313 24 11 435386 makeword63535319118 2014‐05‐09‐11.45.18.189773 cbonilla@casl.umd.edu NumMakeWordContentCompleted content ‐1@166879 1 17313 19735 12 435617 makeword63535251702 2014‐05‐08‐17.01.42.690395 cbonilla@casl.umd.edu NumLemmaShown lexicon pessoa 1 17313 15 10 40276963535251702 2014‐05‐08‐17.01.42.738753 cbonilla@casl.umd.edu NumWordPlayed lexicon pessoa 1 17313 7747 10 37972263535319135 2014‐05‐09‐11.45.35.712531 cbonilla@casl.umd.edu NumConceptChallenged concept makeword‐1 1 17313 22 11 435382 makeword63535319135 2014‐05‐09‐11.45.35.713401 cbonilla@casl.umd.edu NumConceptCorrect concept makeword‐1 1 17313 23 11 435384 makeword63535319135 2014‐05‐09‐11.45.35.713983 cbonilla@casl.umd.edu NumConceptHintDiscount concept makeword‐1 0.00E+00 17313 24 11 435386 makeword63535319135 2014‐05‐09‐11.45.35.714565 cbonilla@casl.umd.edu NumMakeWordContentCompleted content ‐1@189171 1 17313 19735 12 435638 makeword63535319141 2014‐05‐09‐11.45.41.892977 cbonilla@casl.umd.edu UserEvent event cbonilla@casl.umd.edu user_even 17313 11902 14 37047563535319145 2014‐05‐09‐11.45.45.540560 cbonilla@casl.umd.edu NumConceptChallenged concept makeword‐1 1 17313 22 11 435382 makeword63535319145 2014‐05‐09‐11.45.45.541419 cbonilla@casl.umd.edu NumConceptCorrect concept makeword‐1 1 17313 23 11 435384 makeword63535319145 2014‐05‐09‐11.45.45.541973 cbonilla@casl.umd.edu NumConceptHintDiscount concept makeword‐1 0.00E+00 17313 24 11 435386 makeword63535319145 2014‐05‐09‐11.45.45.545772 cbonilla@casl.umd.edu NumMakeWordContentCompleted content ‐1@282026 1 17313 19735 12 435662 makeword63535319145 2014‐05‐09‐11.45.45.603183 cbonilla@casl.umd.edu NumActivityCompleted content makeword 1 17313 4215 12 435425 lltp63535319146 2014‐05‐09‐11.45.46.791774 cbonilla@casl.umd.edu NumActivityCompleted content makeword 1 17313 4215 12 435425 lltp63535319147 2014‐05‐09‐11.45.47.290347 cbonilla@casl.umd.edu UserEvent event cbonilla@casl.umd.edu user_even 17313 11902 14 37047563535319150 2014‐05‐09‐11.45.50.470810 cbonilla@casl.umd.edu UserEvent event cbonilla@casl.umd.edu user_even 17313 11902 14 37047563535319152 2014‐05‐09‐11.45.52.455455 cbonilla@casl.umd.edu UserEvent event cbonilla@casl.umd.edu user_even 17313 11902 14 37047563535319156 2014‐05‐09‐11.45.56.871358 cbonilla@casl.umd.edu NumActivityCompleted content matching 1 17313 4215 12 381180 lltp63535319157 2014‐05‐09‐11.45.57.356615 cbonilla@casl.umd.edu UserEvent event cbonilla@casl.umd.edu user_even 17313 11902 14 37047563535319164 2014‐05‐09‐11.46.04.844827 cbonilla@casl.umd.edu NumFlashCardContentCompleted content 36800 1 17313 29 12 433427 flashcard63535319164 2014‐05‐09‐11.46.04.845586 cbonilla@casl.umd.edu NumLemmaShown lexicon processo 1 17313 15 10 433429 flashcard63535319164 2014‐05‐09‐11.46.04.853555 cbonilla@casl.umd.edu UserEvent event cbonilla@casl.umd.edu user_even 17313 11902 14 37047563535319164 2014‐05‐09‐11.46.04.928987 cbonilla@casl.umd.edu NumWordPlayed lexicon processo 1 17313 7747 10 433431 llaud63535319170 2014‐05‐09‐11.46.10.510097 cbonilla@casl.umd.edu NumFlashCardContentCompleted content 36800 1 17313 29 12 433427 flashcard63535319170 2014‐05‐09‐11.46.10.510986 cbonilla@casl.umd.edu NumLemmaShown lexicon processo 1 17313 15 10 433429 flashcard63535319170 2014‐05‐09‐11.46.10.522356 cbonilla@casl.umd.edu NumLemmaChallenged lexicon processo 1 17313 17 10 433584 flashcard63535319170 2014‐05‐09‐11.46.10.523104 cbonilla@casl.umd.edu NumLemmaCorrect lexicon processo 1 17313 19 10 433586 flashcard63535319170 2014‐05‐09‐11.46.10.523804 cbonilla@casl.umd.edu NumLemmaCorrectHintDiscount lexicon processo 0.00E+00 17313 21 10 433588 flashcard63535319170 2014‐05‐09‐11.46.10.587030 cbonilla@casl.umd.edu NumWordPlayed lexicon processo 1 17313 7747 10 433431 llaud63535319170 2014‐05‐09‐11.46.10.587656 cbonilla@casl.umd.edu NumWordPlayed lexicon processo 1 17313 7747 10 433431 llaud63535319187 2014‐05‐09‐11.46.27.675208 cbonilla@casl.umd.edu NumFlashCardContentCompleted content 36688 1 17313 29 12 421940 flashcard63535319187 2014‐05‐09‐11.46.27.676046 cbonilla@casl.umd.edu NumLemmaShown lexicon passeio 1 17313 15 10 421942 flashcard63535319187 2014‐05‐09‐11.46.27.735006 cbonilla@casl.umd.edu NumWordPlayed lexicon passeio 1 17313 7747 10 421944 llaud63535319192 2014‐05‐09‐11.46.32.318312 cbonilla@casl.umd.edu NumFlashCardContentCompleted content 36688 1 17313 29 12 421940 flashcard

Courtesy of IBM

15

Log files can be mined to answer research questions

16

Feedback Given Failure Success Too many attempts

Subsequent success rate

There is no such word 5 14 0 0.737

Try a different word 16 23 1 0.575

Try a word with a different form(). 6 15 1 0.682

Try a word with a different form(gender). 2 8 0 0.8

Try a word with a different form(person). 0 6 0 1.0Try a word with a different form(singular/plural). 0 1 0 1.0

Try a word with a different form. 4 10 1 0.667

Was it a typo? 11 22 1 0.629

Analysis example: Does feedback help?

17

Work underway

IntroT:1

√

Pronunciation 1T:1

√

Pronunciation vocabL:42

√

ColorsT:1

√

ArticlesT:1

√

Pronunciation 2T:1

√

seasonsT:1

√

Season quizT:1 √

Article quizL:59 √

familyT:1

√

Body partsT:1

serT:1

√

prepositionsT:1 √

Singular pluralT:1 √

Portuguese Learning Plan (partial)

Courtesy of IBM

‘Any language’

19

ILR Level ILR Function Form (Somali)

0+ Recognize and identify letters Sounds and Orthography

0+ Understand common personal and place names Common names, cities, regions, countries

0+ Understand common words and phrases Common Greetings and Benedictions

0+ Understand common words and phrases Numbers 1-20, dates, months

0+ Understand common words and phrases Numbers 20-100

1 Recognize shared vocabulary Arabic and English borrowings

1 Telling time Telling time

1 Understand descriptions Attributive adjectives

1 Understand descriptions Nouns pluralized by –o

1 Understand descriptions Agreement with definite determiners and demonstratives

1 Understand simple directions Cardinal directions and locatives

1 Understand simple directions Temporal and locative adverbs

1 Understand simple directions Basic adpositions (ku, la, ka, u)

1 Understand simple directions and statements Imperative of weak verbs

1 Understand simple statements Declarative/equative sentences using waa… … …

20

Future

‘Any device’

• Proof-of-concept to port platform to mobile device

21

Directions for future improvements

22

Enhance language contextualization Advancing intelligent feedback on error in open-ended

tasks Develop activities appropriate for learners at higher levels

of proficiency Incorporate more SLA principles in the adaptive learning

system:• Authentic and rich input (e.g., Lafford et al., 2007)• Opportunities to produce the language (e.g., Swain,

1985)• Negotiation for meaning in real-time interaction (e.g.,

Long, 1991)• Feedback on error (e.g., Doughty & Long, 2003)

For more information

ContactJared Linck, PhD (Principal Investigator)Associate Research ScientistPhone: 301-226-8879Email: jlinck@casl.umd.eduwww.casl.umd.edu

23