Language Resources and CALL

Language Resources and CALL Applications

Helmer Strik1, Jozef Colpaert2, Joost van Doremalen1, andCatia Cucchiarini1

1 Centre for Language and Speech Technology (CLST) Dept. of Linguistics, Radboud Univ. Nijmegen, The Netherlands2 Linguapolis, University of Antwerp, Antwerp, Belgium

Radboud University NijmegenLREC 2010, Malta, 22-05-2010 2

Language Resources and CALLThe current presentation:

The relation betweenlanguage resources and CALL systems

CALL: Computer Assisted Language Learning

We focus here on the project DISCO:Development and Integration of Speech technology into COurseware for language learning


Overview

A short introduction to DISCOResources used to develop a CALL systemResources obtained during development of a CALL systemResources obtained using a CALL systemConclusions

Dr. Spraak

(Dr. Speech)


A short introduction to DISCODISCO project:

develop a prototype of a CALL systemthat can give feedbackon spoken utterances

Levels:pronunciation (of sounds)grammar (syntax & morphology)




Syntax exercise


Morphology exercise


Pronunciation exercise – with feedback


Menu: conversation environment report, learner is listening to own speech in complete conversation


Menu: conversation environment report, learner is reviewing pronunciation mistakes by listening to own speech


Menu: remediation environment, overall scores for phonemes, learner can start remediation by clicking on a phoneme


Menu: remediation environment, pronunciation exercise


Menu: remediation environment, learner is reviewing progress


Characters in DISCO


ASR-based CALLASR: Automatic Speech Recognition

standard ASR: from (native) speech to words


ASR: Automatic Speech Recognition

Decoder

AcousticModels

Lexicon LanguageModel

Speech SignalInput

W1 W2 W3 W4

WordsOutput


ASR-based CALLASR: Automatic Speech Recognition

standard ASR: from (native) speech to words

ASR for CALL, 2 phases:1. content, what has been said, tolerant; recognize words despite non-native variation2. form, how has it been said, strict; error detection, find deviations from native …


Resources used to develop a CALL system (1)

More general, native resources:ASR toolkit – e.g. SPRAAK [from Stevin]Corpus with native speech –e.g. Spoken Dutch Corpus (CGN) [from TST-Centrale]Native lexicon – e.g. e-Lex [from TST-Centrale]



More specific, non-native resources (often not available) to develop / improve the 2 phases:

Phases 1 + 2. Corpora with non-native speech : JASMIN [from Stevin]; CITO, Triest, Dutch-CAPT

Phase 1. word recognition, contentResources, information to model non-native 'behavior', in order to improve:

Acoustic Models: mainly by training on non-native audio (from speech corpora)Lexicon & Language Model: data-driven, from non-native audio, or knowledge based, from lit. etc.



More specific, non-native resources (often not available) to develop / improve the 2 phases:

phase 2. error detection (classifiers), strict; A. Decide which errors to address,

criteria + selection => inventorydata-driven and/or knowledge based

B. Develop classifiers, train and test; data-drivenA & B. data-driven => Resources needed: annotations for audioLevels:

o Pronunciation: sounds [& prosody, not in DISCO]o Grammar: syntax & morphology


Resources obtained during developmentBlue-print of the designContent

specifications for exercises and feedback strategies a list of predicted correct and incorrect utterances

Modules for the 2 phases: 1. word recognition, 2. error detectionThe CALL system itself, the whole system,prototype with content


Resources obtained using a CALL systemAudio recordingsLog-files: user + system 'behavior'Videos


ConclusionsLanguage Resources

important role in relation to CALL systemsLanguage Resources

are needed to develop a CALL systemcan be obtained during development of a CALL systemcan be obtained using a CALL system

Language Resources obtained give rise to new opportunities:

researchsystem development


Website DISCOlands.let.ru.nl/~strik/research/DISCO/


Stevin project DISCOTrainen van spreekvaardigheid

uitspraak, morfologie, syntax

CorrectVoorbeeld Ik loop naar huis

FoutenUitspraak Ik lop nar guisMorfologie Ik lopen naar huisSyntax Ik naar huis lopen

Fouten automatisch detecterenm.b.v. spraaktechnologie


ASR

DisplayLogic

PromptGenerator

Words

Segmentation

ErrorDetectionGrading

FeedbackGeneration

Documents

Language Resources and CALL