English language learning activity using spoken language

and intelligent computer-assisted technologies

Maria Fuentes, Meritxell González TALP Research Center, Universitat Politècnica de Catalunya

Team

Marta Guardiola (marta.guardiola@tsc.upc.edu )

Nacida en Besalú (Girona) en 1984. Ingeniera de Telecomunicaciones (2008) y European Master of Research on Information and Communication Technoligies (MERIT) el 2009 por la UPC. Desde 2009 realiza el doctorado en el Departamento de Teoría del Señal y comunicaciones (TSC) de la UPC. Su investigación se focaliza en el ámbito de la formación de imagen con microondas para aplicaciones biomédicas, que comprende desde el desarrollo de sensores hasta el desarrollo de los algoritmos de reconstrucción de imagen.

Lluís Jofre (lluis.jofre@upc.edu)

Professor at the Department of Signal Theory and Communications and Director of the UPC-Telefónica endowed Chair.

Jordi Romeu (jordi.romeu@upc.edu)

Professor at the Department of Signal Theory and Communications and Vice-Director of the UPC-Telefónica Chair.

Francesc Vallverdú (francesc.vallverdu@upc.edu)

Dr. Ingeniero en Telecomunicaciones (1988) por la Universitat Politècnica de Catalunya. Es profesor de la Escuela Técnica Superior de Ingenieros de Telecomunicaciones de Barcelona de la UPC desde 1985, impartiendo cursos en materias relacionadas con el procesado de la señal y señales y sistemas. Ha sido vicerrector de metodología educativa, innovación e investigación de la Universitat Oberta de Catalunya (UOC) en 1998-2006. Es investigador del grupo de tratamiento del habla de la UPC desde la su creación, y actualmente ejerce de responsable. Ha participado en proyectos de investigación nacionales e internacionales.

Climent Nadeu (climent.nadeu@upc.edu)

Professor on signal processing at the Signal processing and Communication Department , co-founder of the Speech Processing Group of the UPC, and Director of the TALP Research Center TALP from 1998 till 2004.

Lluís Padró (lluis.padro@upc.edu)

Associate Professor at the Software Department, member of the Natural Language Processing Group and the TALP Research Center.

Maria Fuentes

Nacida en Girona en 1973. Ingeniera en Informática (1998) y doctora en coordinadora de I+D+i del grupo de Procesado de Lenguaje Natural.

Meritxell González

Post-doc Researcher at the Natural Language Processing Group at UPC and member of the TALP Research Center.

TECHNICAL ADVICE

Design & Developme

nt

STAGING

DIRECTION

References[1] Fuentes, M., Gonzàlez, M., Guardiola, M., Jofre, L., Romeu, J., Vallverdú, F. ``Cuaderno: Aprendizaje y Asistencia Virtual en Red''. Càtedra Telefónica-UPC, 2011.[2] Bolaños, D., Cole, R., Ward, W., Borts, E., Svirsky, E. ``The FLORA, Fluent Oral Reading Assessment for Children's Speech''. Submitted in June 2010 to the ACM Transaction on Speech and Language Processing (Special Issue on Speech and Language Processing of Children's Speech for Child-Machine Interaction Applications). [3] Meurers, D., Ziai, R., Amaral, L., Boyd, A. Dimitrov, A., Metcalf, V., Ott, N. ``Enhancing authentic web pages for language learners''. In Proceedings of the NAACL HLT 2010 Fifth Workshop on Innovative Use of NLP for Building Educational Applications (IUNLPBEA '10). Association for Computational Linguistics, Stroudsburg, PA, USA, 10-18, 2010.

Moodle 2.0Integrated user authentification

Independent tasksEach task is controlled from the moodle system

Results are used to control tghe interaction

Planting a Garden Activity

Grammar

Writing

Record and listen yourself

Compare with the synthetiser

Java Applet Javasonics www.javasonics.com

Chat with a Dialog SystemNatural Languge Interaction

Plan-based DS Semantic-Syntactic parser Pattern-based NL generator

English grammar practice with any text

Werti system [3] Firefox extension

Center for Language and Speech Technologies and

Applications

Motivation

Virtual Environment Architecture

Use of ITC in educational environments [1]

Promote the use of speech and natural language processing technologieswithin the educative process

Focus on language learning through reading

Fluency assessment (WCPM)

FLORA system [2] Log of voices Detection of word insertion, deletion, substitution

Reading

Text-to-Speech Synthetiser

Ericsson Labshttps://labs.ericsson.com/apis/text-to-speech/

Listening

Conclusions and Future work

Speaking

This first experiment allowed us to empirically verify the performance of the proposed technologies in a real environment

In general, the pilot carried out at secondary school was really positive

Future directions:To integrate more cognitive and comprehensive abilities in the dialogue system in order to evaluate the meaning of the student’s answers.To include multimodal interactionTo include additional feedback to each task