TheALSTprojectTechnologiesforaudiovisualtransla8onAnnaMatamala(UniversitatAutònomadeBarcelona)

anna.matamala@uab.catTransla8ngandtheComputer37(TC37-AsLing2015)London,26-27November2015Funding:FFI-2012-31024,2014SGR0027

1

AnnaMatamala(2015).“Technologiesforaudiovisualtransla>on:theALSTproject”Aspresentedatthe37thConferenceTransla1ngandtheComputer,London,UK,November26-27,2015.©AsLing,TheInterna>onalAssocia>onforAdvancementinLanguageTechnology,2015.Distribu>onwithouttheauthorisa>onfromASLINGisnotallowed.AsLingaskingtobeinformedofsuchpos>ngs,includingURLsorURIswhereavailable,totheemailaddress:<presenta>ons@asling.org>."

2

The ALST project: aims

•  Speechrecogni>on• Machinetransla>on•  Speechsynthesis

inaudiodescrip>on(AD) invoice-over(VO)

3

Project ra5onale

•  Implementa>onofexis>ngtechnologies•  Twomodeschosenasinstancesoflinguis>c/sensorialaccessibility• Oralmodes•  Focusontheprofessionalsandontheaudiences

•  Limita>ons:3years,limitedfunding• Projectteamand2PhDstudents

4

Speech Technologies

5

Speech recogni5on in voice-over (I)

• Par>cipants:10professionaltranscribers•  3condi>ons:•  Manualtranscrip>on•  Respeaking•  Revisionofanautoma>cally-generatedscript(ASR)

• Quan>ta>vedata:>mera>o,outputquality(NER)• Qualita>vedata:opinionsonusefulness,speed,accuracy,overallquality,effort,boredom.

Source:Matamala,Romero-Fresco&Daniluk(forthcoming)6

Speech recogni5on in voice-over (II)

•  Speed:manual,respeaking,ASRrevision.•  Accuracy:manual,ASRrevision,respeaking.•  Respeakingallowedthehighestnumberofpar>cipantstofinish.

•  Respeaking,begerscoresinself-reportedeffortandboredom.• Manualtranscript,begerscoresinaccuracyandoverallquality.•  Respeaking:impressedwithmanypossibili>esandmorejobsa>sfac>onbutneedforspecifictrainingandfurtherresearch.

7

Speech recogni5on in audio descrip5on

•  Soundtrackextrac>on,speechac>vitydetec>on,speakerdiariza>on,andspeech-to-texttranscrip>on

•  Film“Closer”(English,Catalan)

• DERandWERmeasures:lowperformancebecauseoftrainingcondi>onsofthesystemandemployedADmaterials.

Source:Delgado,Matamala&Serrano(forthcoming)

8

Speech synthesis in voice-over

• On-going

9

Speech synthesis in audio descrip5on (I)

•  Pre-testtoselectthe“best”human/naturalmale/femalevoices

• Mainexperiment:67blindandvisually-impairedvolunteers•  Ques>onnaireinspiredbyITU(1994)

Source:Fernández-Torné&Matamala(2015)

10

Speech synthesis in audio descrip5on (II)

•  Naturalvoiceshavehighervaluesthanar>ficialvoices

•  Nosta>s>caldifferencesbetweenmale/femalenaturalvoices

•  Overallimpression/acceptance:meanhigherthan3.2(ona5-pointscale)•  94%par>cipants:TTSisanalterna>veacceptablesolu>onalthoughnotthepreferredone

Source:Fernández-Torné&Matamala(2015)

11

Machine Transla5on

12

Machine transla5on in VO: effort

•  Par>cipants:12AVTMAstudents•  Condi>ons:

•  humantransla>on•  post-edi>ng

• Materials:twoshortwildlifedocumentaryexcerpts(En>Es)•  Temporal,technical,cogni>veeffortusingInputlog(www.inputlog.net/)

Source:Or>z-Boix&Matamala(forthcoming)

13

Machine transla5on in VO: results

• Resultsobtainedfor:bothexcerpts,excerpt1,excerpt2

•  Temporaleffort:post-edi>ngfasteralthoughresultsareonlysta>s>callysignificantinexcerpt1

•  Technicalandcogni>veeffort:post-edi>ngrequireslesseffort,althoughdifferencesareonlysta>s>callysignificantgloballyandinexcerpt1

Source:Or>z-Boix&Matamala(forthcoming)

14

Machine transla5on in VO: QA • QAbyexperts:• Pre-correc>ongrading/Correc>onbasedonMQM/Ques>onnaire/Finalmark/Isitatransla>onortheresultofapost-edi>ng?

• QAinDubbingstudio:observa>onalnotes,correc>onsmadebydubbingdirector

• QAbyend-users:pre-taskques>onnaires,viewing,post-taskques>onnaires(comprehension,self-reportedenjoyment,preferencesandinterest)

Source:Or>z-Boix&Matamala(2015)

15

Machine transla5on in VO: results (I)

• QAbyexperts:passesround1(45/72transla>on,37/72post-edi>ng),round2(41and38/72,respec>vely)

• QAbyexperts:correc>ons(12.862transla>onvs.17.957PE)

• QAbyexperts:iden>fytransla>onsassuchmoreeasily

•  Transla>onsareslightlybeger Source:Or>z-Boix&Matamala(2015)

16

Machine transla5on in VO: results (II)

• QAbydubbingstudio:

•  excerpt1(5changesintransla>onvs6inPE),•  excerpt2(4intransla>on,noneinPEduetobadquality)

•  Transla>onisbegerin2ndexcerpt,minimaldifferencesinthe1st

Source:Or>z-Boix&Matamala(2015)

17

Machine transla5on in VO: results (III)

•  QAbyusers:•  Translatedversionsslightlybegerunderstoodthanpost-editedversionsinglobalanalysisandexcerpt1(post-editedisbegerinexcerpt2).

•  Begerresultsfortransla>on(enjoyment,preferencesandinterest)

•  Transla>onqualityisslightlyhigherinallcondi>onsbutresultsarenotsta>s>callysignificantandtherearedifferencesaccordingtoexcerpts/groups.

Source:Or>z-Boix&Matamala(2015)

18

Machine transla5on in audio descrip5on (I)

• Pre-testtoselecttheMTengine:GoogleTranslate.

• Par>cipants:5volunteers•  5MTengines• Materials:clipfrom“Closer”(240wordsin14ADunits,3’09’’)•  Tool:PET

Source:Fernández-Torné&Matamala(2014)

19

Machine transla5on in audio descrip5on (II)

• Mainexperiment:•  Par>cipants:12AVTMAstudents•  Condi>ons:humancrea>on/transla>on/post-edi>ng•  Languagepair:EnglishintoCatalan•  Materials:comparableexcerptsfrom“Closer”•  Tool:Sub>tleWorkshop

•  Temporal,technicalandcogni>veeffort(Inputlog,www.inputlog.net)•  Subjec>veopinions(pre-taskandpost-taskques>onnaires)

Source:Fernández-Torné&Matamala(inprep.)

20

MT in AD: results

•  Temporaleffort:nosta>s>caldifferences.•  Technicaleffort:ADcrea>onandADtransla>on,morekeyboardac>onthanpost-edi>ng.ADtransla>onandpost-edi>ng,highernumberofmousescrollsthanADcrea>on.• Cogni8veeffort:sta>s>callyhigherinADcrea>on.

Source:Fernández-Torné&Matamala(inprep.)

21

MT in AD: results (II)

•  Subjec8vedata(ona10-pointLikertscale):

Source:Fernández-Torné&Matamala(inprep.)

22

ADcrea8on ADtransla8on ADpost-edi8ng

Pre Post Pre Post Pre Post

Effort 8.25 7.17 6.17 5.58 6.50 7.50

Crea>vityimpairment

3.09 3.82 7.45 7.27 8.45 9.36

Boredom 2.09 1.82 4.18 4.18 6.73 7.27

Calques 1.25 2.00 5.25 5.42 6.93 8.33

Conclusions

•  Small-scaleexploratoryproject,exis>ngtechnology

• Moreresearchneededwithwidersamples,moreexcerpts,otherlanguagepairs

• Notonly>me(produc>vity)butalsoopinionsofprofessionalsandendusers

23

TheALSTprojectTechnologiesforaudiovisualtransla8onAnnaMatamala(UniversitatAutònomadeBarcelona)

anna.matamala@uab.catTransla8ngandtheComputer37(TC37-AsLing2015)London,26-27November2015Funding:FFI-2012-31024,2014SGR0027

24