1

Instituto de Engenharia de Sistemas e Computadores Investigação e Desenvolvimento em Lisboa

technologyfrom seed

Spoken Language System Lab

technologyfrom seed

2

• History– Work on speech processing for Portuguese since the 90s– Creation: 2001

• Goal– Bring together several groups in the area of spoken language

processing for European Portuguese, united by the problem we want to solve, not by the technology we share.

• Mission– Creating technology to bridge the gap between natural spoken language

and the underlying semantic information.

• Interdisciplinary background : Signal processing, natural language processing, linguistics, etc.– Researchers: 10– Post-Grad. Students: 15 (PhD)– Invited researchers: CLUL, UAlg, UBI

About L2F

technologyfrom seed

3

L2F (A. Abad, L. Coheur, D. Matos, N. Mamede, H. Meinedo, J. Neto, L. Oliveira, T.Pellegrini, A. Serralheiro, I. Trancoso)

CLUL UALG UBI

(M.C. Viana) (J. Baptista) (G. Dias)

Researchers

technologyfrom seed

4

Team

technologyfrom seed

5

Speech processing – Text-to-speech synthesis

• Automatic process for building new voices• Limited domain synthesis• Expressive speech synthesis• Audio-visual synthesis

– Automatic speech recognition• Robust speech recognition• Speaker adaptation• Large vocabulary continuous recognition

– Speech coding– Speech enhancement– Speaker and language identification

Text processing– Morphological analysis– Syntactic analysis– Semantic analysis– Discourse analysis– NL Generation– Named entity extraction– Information retrieval– Summarization– Question answering– Machine translation

Spoken language processing – Speech understanding– Speech synthesis from concepts– Spoken/multimodal dialog systems – Classification of multimedia documents– Summarization of spoken documents– Rich Transcription of multimedia documents – Speech-to-speech machine translation– Question answering on multimedia documents– Language tutoring– etc.

Core technologies

technologyfrom seed

Rich transcription of multimedia documents

• chove forte desde sábado no rio grande do sul e em santa catarina pelo menos cinqüenta cidades foram atingidas pelo temporal desde mil pessoas estão desabrigadas ou rio grande do sul um homem morreu e duas pessoas estão desaparecidas metade de são sebastião do caí a setenta quilómetro de porto alegre está debaixo de água

• [spk1001] Chove forte desde sábado no Rio Grande do Sul e em Santa Catarina pelo menos, cinqüenta cidades foram atingidas pelo temporal, desde mil pessoas estão desabrigadas ou Rio Grande do Sul um homem morreu e duas pessoas estão desaparecidas. [spk2002] Metade de São Sebastião do caí a setenta quilómetro de Porto Alegre está debaixo de água.Tema: segurança, nacional

technologyfrom seed

Off-line BN processing

technologyfrom seed

Automatic captioning

•On line at RTP since March 2008•Extended to all varieties of Portuguese•Computer enhanced human-to-human communication

–Meeting browser–Lecture browser–Courtroom transcriptions

•Other languages: English, Spanish

technologyfrom seed

9

• Components:– speech recognition, dialogue management, speech

synthesis, talking faces, question answering• Applications:

– IVR systems (via telephone network)– Domotic: "intelligent" demo room controllable by voice– Museum guides

Spoken/Multi-modal Dialogue Systems

technologyfrom seed

10

• Project PT-STAR– Cooperation with CMU– Cooperation with Univ. Macau– Targeting Broadcast News, Classroom

lectures and TED Talks

Speech to Speech Machine Translation

technologyfrom seed

Computer-Assisted Language Learning

• Project REAP.PT– Cooperation with CMU– Learning vocabulary from current texts, on areas of the student’s interest

technologyfrom seed

12

Extension to all varieties of Portuguese

• Brazilian and African Portuguese– National Project POSTPORT (ongoing)– Cooperation with Brazil

technologyfrom seed

13

• "Eugénio“: word prediction tool for people with motor impairments

• ARIA – Ambient-Assisted Reading Interfaces for the Ageing Society (on-going)

• VITHEA – Virtual Therapist for Aphasia Treatment (on-going)

E-inclusion

technologyfrom seed

14

European Projects (on going)

• LIREC– LIving with Robots and intEractive Companions

• VIDIVideo– Interactive semantic video search with a large thesaurus

of machine learned audio-visual concepts

• I-DASH– The Investigator’s Dashboard (Safer Internet)

• COST 2102– Cross-Modal Analysis of Verbal and Non-verbal

Communication• COST 2103

– Advanced Voice Function Assessment • ECESS

– European Center of Excellence on Speech Synthesis

technologyfrom seed

• Goal: to establish a multi-faceted theory of artificial long-term companions (including memory, emotions, cognition, communication, learning, etc.), embody this theory in robust and innovative technology and experimentally verify both the theory and technology in real social environments.

Whether as robots, social toys or graphical and mobile synthetic characters, interactive and sociable technology is advancing rapidly. However, the social, psychological and cognitive foundations and consequences of such technological artifacts entering our daily lives - at work, or in the home - are less well understood.

LIREC

technologyfrom seed

16

VIDIVIDEO

technologyfrom seed

• Portugal Telecom• Porto Editora• Vodafone• Tecnovoz

Tecnhology transfer

technologyfrom seed

18

• Wide range of products centered on the core technologies:• Audimus –Automatic speech recogntion• DIXI – Text-to-Speech synthesis• DIGA – Dialog platform• FACE – Animated faces

Spin-off

technologyfrom seed

19

Participation in evaluation campaigns– Synthesis, Named Entity Recognition, Translation, Question

Answering, Language recognition

Editorial Activities – Editor in Chief of the IEEE Transactions on Audio and Speech

Processing (2003-2005)

– Editorial Board of Signal Processing Magazine (2006-2008)

– Guest Editor of Special Issue on Iberian Languages (Speech Comm., 2008)

Representation in international organizations– IEEE STC - Speech Technical Committee (1999-)

– IEEE - Signal Proc. Society Board of Governors (2006-2008); Nominations & Appointments Committee (2010-2011); Meritorious Service Award (2010)

– ISCA – Int. Speech Communication Association Board (1993-2001; Vice-President: 2005-2007, President: 2007-2011

Organization of scientific events– INTERSPEECH 2005, Lisbon

International visibility

technologyfrom seed

20

technologyfrom seed

L2 F - Spoken Language Systems Laboratory

www.l2f.inesc-id.pt