View
1
Download
0
Category
Preview:
Citation preview
iSTEP 2015: Cross-Cultural Technology Development Toward Language Access for the Deaf and
Hard of Hearing
Maya Lassiter, Amal Nanavati, Erik Pintar, Minnar Xie, Ermine Teves, M. Bernardine Dias
CMU-RI-TR-16-32
The Robotics Institute Carnegie Mellon University
Pittsburgh, Pennsylvania 15213
June 2016
Copyright © 2016 TechBridgeWorld at Carnegie Mellon University
1
iSTEP2015:Cross-CulturalTechnologyDevelopmentTowardLanguageAccessfor
theDeafandHardofHearingMayaLassiter,AmalNanavati,ErikPintar,MinnarXie,ErmineTeves,M.BernardineDias
ACKNOWLEDGEMENTSPartnerinvolvementhasbeenessentialforthiswork,andwewerefortunatetocollaboratewiththeMathruEducationalTrustfortheBlindandtheirCenterfortheDeafandDifferently-Abledinneedsassessmentandusertesting.TheauthorsespeciallythankMs.GubbiR.MukthaandMathruCenterstaffmembersformakingourprojectspossible.Furthermore,manyindividualsparticipatedinourneedsassessment,generouslyvolunteeringtheirtime.Becauseofresearchcomplianceprocedures,wecannotnametheseparticipantsindividually,butwearegratefulfortheirinvolvementandenthusiasmfortheproject.WealsothankBernardineDiasandErmineTevesfromtheTechBridgeWorldresearchgroupfortheirguidancethroughouttheinternship.WewouldalsoliketothankoursponsorswhomadetheiSTEP2015internshippossible,includingindividualdonorsand100@100donors.
ABSTRACTThispapersummarizestheworkdoneoverthecourseofnineweeksattheMathruCenterfortheDeafandDifferently-Abled(MathruCenter)inBangalore,Indiaandaimstoproviderecommendationsregardingtechnologysolutionsfordeafandhardofhearingstudentsandtheirteachersinunderservedcommunities.ThisworkwascarriedoutbyateamoffourundergraduateresearchstudentsparticipatinginaninternshipprogramcallediSTEP–shortforinnovativeStudentTechnologyExPerience(iSTEP)–organizedbytheTechBridgeWorldresearchgroupatCarnegieMellonUniversity.Fromtheteam’sworkindesigninganddevelopingeducationaltoolswithstudentandteacherparticipants,thispaperexploresthepracticeofcreatingsharedownershipovertechnologicalideasthroughdirectrelationshipwiththetargetpopulationandviewingthemasequalsintheprocess.Theteam’spartnershipandworkwiththeMathruCenterresultedintwonewlycreatededucationaltoolsaimedatprovidinglanguageaccessforDeafprimaryschoolstudentsandtheirteachers.Thefirst,SignBook,providesgreatersignlanguageaccessthroughasignlanguagedictionarycreationtoolwithabilitiesforcustomvideoandpicturecapturingandcategorizationofentriesandwordsintotopics.Thesecond,SpeakUp!,motivatesstudentstostriveforgreaterverballanguageaccessthroughasuiteofvoice-poweredgamesaimedtofamiliarizepre-andpartially-verbaluserswiththepoweroftheirvoice.
2
TABLEOFCONTENTSAcknowledgements..............................................................................................................................1
Abstract....................................................................................................................................................1
TableofFigures.....................................................................................................................................3
ListofTables..........................................................................................................................................3
Introduction...........................................................................................................................................4TechBridgeWorld............................................................................................................................................4iSTEP....................................................................................................................................................................4TheMathruEducationalTrustfortheBlind..........................................................................................5InformationCommunicationTechnologyforDevelopmentinIndia............................................5CultureandCommunity................................................................................................................................6iSTEP2013Findings.......................................................................................................................................8Motivations........................................................................................................................................................9
LanguageAccessTools.....................................................................................................................10StudySetting...................................................................................................................................................10StudyParticipants.........................................................................................................................................11FindingsandDevelopment........................................................................................................................11SignBook:ASignLanguageDictionary................................................................................................................12SpeakUp!VoicePoweredGameSuite.................................................................................................................20
Discussion............................................................................................................................................26
AdditionalConsiderationsforFutureWork............................................................................27
References...........................................................................................................................................30
Appendix..............................................................................................................................................33OurFindingsattheMathruCenterclassroomforStudentswithMulti-SensoryImpairments(MSI).......................................................................................................................................33
3
TABLEOFFIGURESFIGURE1.THEMATHRUCENTERFORTHEDEAFANDDIFFERENTLYABLED...........................................................................................5FIGURE2.FLASHCARDSUSEDATTHESCHOOL................................................................................................................................................8FIGURE3.FIRSTGRADESPEECHCLASS.........................................................................................................................................................11FIGURE4.ASAMPLINGOFAMERICANSIGNLANGUAGEWORDSWRITTENWITHSIGNWRITING........................................................14FIGURE5.“MYNATIVETONGUE–1”THEONLYOFFICIALSIGNLANGUAGEREFERENCEBOOKATTHEMATHRUCENTER...........16FIGURE6.ANINTRICATEDRAWINGBYATEACHER,USEDTOEXPLAINACLASSROOMLESSONINTHEFIRSTGRADECLASSROOM.17FIGURE7.THESIGNBOOKINTERFACE...........................................................................................................................................................18FIGURE8.THESPEAKUP!MAINMENU.........................................................................................................................................................24FIGURE9.AMATHRUCENTERSTUDENTUSINGTHESPEAKUP!VOICEPOWEREDGAMESUITE..........................................................24FIGURE10.ATEACHERUSINGSPEAKUP!COMMUNALLYWITHHERENTIRECLASS..............................................................................25FIGURE11.MATHRUCENTERTEACHERSCREATINGASIGNINSIGNBOOK.............................................................................................26FIGURE12.TECHNOLOGYISOFTENTREATEDASANOVELTY,SINCESTUDENTSDONOTENCOUNTERTECHNOLOGYINEVERYDAY
SETTINGS..................................................................................................................................................................................................27
LISTOFTABLESTABLE1.SIGNBOOKTABLEOFFEATURES...................................................................................................................................................19
4
INTRODUCTION
TECHBRIDGEWORLDTechBridgeWorld(TBW)isaresearchgroupbasedintheRoboticsInstituteatCarnegie
MellonUniversity(CMU).Thegroupformsandbuildsuponpartnershipsinunderservedcommunitiesinternationallytocreateculturallyappropriatetechnologythatadherestoeachcommunity’svisionofprogress.Foundedin2004,TBWleadstheinnovationandimplementationofsolutionsthathelpaddresschallengesincommunitiesnotordinarilyservedbytechnologydevelopmentefforts[[1]].Thisworkofteninvolvestheinventionofnewtoolsandcustomizationofexistingtechnologyaswellaseffortstoinspireandtrainfutureresearchersandtechnologistsinthiskindofwork.TechBridgeWorldemploystheknowledgeandimaginationoffaculty,staff,andstudentsinallofitsprojects.TechBridgeWorldaimstocreatetechnologysolutionsaccessibleandrelevanttoalltoimprovepeople’slives,enhanceandshareadvancesintechnologyacrosscultures,andpromotesustainabledevelopmentaroundtheglobe.
ISTEPTBWlaunchedtheInnovativeStudentTechnologyExPerience(iSTEP)internship
programinthesummerof2009toprovideCMUstudentswithreal-worldexperienceinapplyingtheirknowledgeandskillstowardservingmarginalizedorunderservedcommunities.Throughaunique10-weeksummerresearchinternship,studentsconducttechnologyresearchprojectsalongsidepartnercommunitiesacrosstheworld.
iSTEPoffersselectedstudentstheopportunitytoengineerwithcompassionthroughfirst-handexperience.OvertheyearsiSTEPhasbroughttogetherinternteamsofundergraduateandgraduatestudentsfromCMU’sPittsburghandDohacampuseswithdiverseexperiencesanddisciplinarybackgrounds.TogetherwithTBW,eachiSTEPteamcollaborateswithlocalpartnerstobetterunderstandtheneedsofthecommunityanddesigntechnologicalsolutionsthatcouldhelpaddresssomeofthechallengesthatthecommunityfaces.AspartofTBW’sphilosophy,allsolutionsarebuiltwiththecommunityanddevelopedthroughtrust.
PastiSTEPlocationsincludedTanzaniain2009,Bangladeshin2010,Uruguayin2011,Ghanain2012,andIndiain2013withprojectsinassistivetechnology,literacytools,informationexchange,andenvironmentalsustainability[[2],[3],[4],[5],[6]].
ThemultidisciplinaryiSTEP2015teamwascomprisedoffourundergraduatestudentsfromvariousschoolsatCMUincludingtheCarnegieInstituteofTechnology,CollegeofFineArts,DietrichCollegeofHumanitiesandSocialSciences,SchoolofComputerScience,andtheBXAIntercollegeProgram.iSTEP2015internshavebuiltupontheiSTEP2013team’sworkbydevelopingnewassistivetechnologyprojectsincollaborationwithTBW’spartner,theMathruEducationalTrustfortheBlind.
5
THEMATHRUEDUCATIONALTRUSTFORTHEBLINDWhilenowworkingwith
numerousglobalpartners,TBW’slongeststandingpartnershiphasbeenwiththeMathruEducationalTrustfortheBlind(MathruTrust).TheMathruTrusthousesboththeMathruSchoolfortheBlind(MathruSchool)andTheMathruCenterfortheDifferentlyAbled(MathruCenter).Ms.GubbiR.Mukthafoundedbothschoolsafterherowndebilitatinginjury.TBWhasbeenpartneringwiththeMathruTrustsince2006tobuildassistivetechnologybasedonobservedandcommunicatedneedswithcontinualcommunityinputandfeedback.Additionally,TBWhasbeenpartneringwiththenewMathruCentersinceitsinceptionin2012.In2013,aniSTEPteamcompletedaNeedsAssessmentreportandsincethen,anongoingpartnershipwiththeMathruCenterhascontinued.
TheMathruCenterwasinspiredafterillnesscausedablindstudentattheMathruSchooltosuffersignificanthearingloss,thusbeingunabletocontinueonattheschoolwithoutmorespecializedaccommodationsandsupport.Today,theMathruCenterisaresidentialschoolofferingfreeeducationforstudentswithhearingimpairmentsormulti-sensorydisabilities.ThegoaloftheMathruCenteristorecognizethecapabilityandpowerofallindividuals,regardlessofthelimitationsimposedbysocietyandenvironmentregardingphysicalability.TheMathruCentercurrentlyhas47studentsofwhich31aredeafandhardofhearingstudentsand16havemulti-sensoryimpairments.WhilethisyeartheCenterservesstudentsfromnurseryto4thstandard,additionalstandardswillbeaddedannuallyasstudentsgrowintothenextgrade.ThecurriculumfortheDeafstudentsattheMathruCenterfollowsthestatesyllabusaswellasteachingtheadditionalcurricularneedsofspeechandsignlanguage.
INFORMATIONCOMMUNICATIONTECHNOLOGYFORDEVELOPMENTININDIAThoughInformationandCommunicationTechnologiesforDevelopment(ICTD)isa
youngdiscipline,Indiahasbeenanepicenterofresearchandfieldstudiesinthisarea[[7]].ThesurgeofInformationandCommunicationTechnologies(ICTs)inIndiahasbeengrowingforseveraldecades,startingfromIndiabeingthefirstcountrywithacomputerintheglobalSouth
Figure1.TheMathruCenterfortheDeafandDifferentlyAbled
6
in1956[[8]].Thishascreatedaspaceforincredibleempowermentaswellasimposedsolutions.
TechnologicaldevelopmentthatcaterstousersintheGlobalNorthoftenmakesassumptionsthatarenottrueofdevelopingcountries.Tonameafew:reliablepower,Internetaccess,previousexperiencewithtechnology,andliteracy.InIndia,acountrywithonly12Internetusersper100[[9]]personsandunequalliteracyratesacrossstatesanddemographics,thesecommonassumptionsbydevelopersdenylargeIndianpopulationsfromtechnologicalsupport[[10],[11],[12]].
Forfurthermarginalizedgroups,suchasthedisabled,accesstoandexperiencewithtechnologyislikelyevenmorelimited[[13]].Thus,additionalprovisionsweretakentoensureasolidunderstandingoftheuniqueneedsofourpartner,theMathruCenter.Likewise,itwascriticaltoemployparticipatoryandempoweringstancesinboththecreationandevaluationofourwork.
CULTUREANDCOMMUNITYCurrently,360millionpeopleworldwidehavedisablinghearingloss,whichtheWorld
HealthOrganizationdefinesas“hearinglossgreaterthan40dBinthebetterhearingearinadults(15yearsorolder)andgreaterthan30dBinthebetterhearingearinchildren”[[14]].Individualswithdisablinghearinglosshavetroublehearingeverydayconversationorperceivingsoundinevenquietenvironments.Asitstands,only10%oftheglobalneedforhearingaidsismet,90%oftheworld’sdeafpopulationhasneverbeentoschool,andonly5%canreadorwrite.Theprevalenceofhearinglossisunequallydistributedacrosstheworldwith80percentoftheworld’sdeafpopulationlivingindevelopingcountries[[15]].Additionally,SouthAsiahasthehighestratesofdeafandhardofhearingindividualswhilethecurrentproductionofhearingaidsmeetslessthan3%ofdevelopingcountries’need[[1]].Inturn,SouthAsianDeafcommunitiesfaceincreasedchallengesandhardships.
UnderIndia’sPersonswithDisabilitiesActof1995,theintegrationofdisabledcommunitiesintomainstreamsocietywasestablishedasagovernmentpriority[[16]].However,officialdocumentationofsignlanguagehasbeendisregardedandunsupported,thuskeepingIndia’sDeafcommunityfrombeingwidelyunderstoodandrecognized.
OurworkwiththedeafandhardofhearingcommunityatMathruCenteraddressedtwocomplexities.First,whiletheMathruCenterhopestoincreaseaccesstosignlanguage,Indiansocietyasawholeneglectssignlanguageandthusprioritizesverballanguageasanecessarypartofdeafpeoples’societalinclusion.Therefore,workinthisareamustacknowledgethepresentexpectationofverbalcommunicationasapathtoself-advocacyfortheDeaf.AlthoughIndia’sDeafcultureisgrowing,theyarestilldecadesawayfromhavingthesamesupportandrecognitionseenbyAmericanDeafculture-somuchsothatitwouldbe
7
disadvantagingMathruCenterstudentstonotenableverballanguageaccessalongsidesignlanguageeducation[[17]].
AmajordifficultythatissomewhatuniquetoIndia’sgrowingDeafcommunityisthevastmultitudeofexistingverballanguages.ThissituationcanleadtheDeaftofeelsocietalexclusionacrossallofIndia’sdiverselanguage-scapeinboththehearingandDeafcommunities.Thus,deafindividualsinIndiaexperiencetheuniquechallengeofhavingtobridgeamultitudeofsignedlanguagesontopofamultitudeofverballanguages.InIndia,deafnessoftenassumesdumbness–thecognitiveinabilitytocommunicate.This,alongwithIndia’sculturaldenunciationofsignlanguages,barsIndia’sDeafcommunitiesfromsupportinbothsignedandspokenlanguages.Despiteanincreasinglyglobalworld,Deafcommunitiesremaindisconnectedduetoalackofunderstanding,lackofinterpreters,andlackofsocietalrespectforsignlanguages[[18]].
Themain“justification”forthelinkagebetweendeafanddumbisthedelayedspeechandcommunicativedevelopmentofdeafchildren.Howeverthisdelayisinspokenlanguage,anunnecessarymetricifprovidedaccesstosignlanguage[[19]].Speechtherapyremainsprohibitivelyexpensiveandtimeintensive.BecausetheMathruCenteralsolackstheresourcestofullyinvestinindividualizeddailyspeechtherapy,progressisslowandunfruitfulintheshortspanoftheschoolyear.AccesstospokenlanguageremainsdifficulttoincorporateandutilizeineducationfordeafIndians,yettheywouldbefullyabletoparticipateinsocietyifgivensupportandencouragement[[16]].
EffortsliketheAzimPremjiFoundation(APF)inBangalore,IndiaandtheDigitalInformationResearchFoundation(DIRF)inChennai,IndiahavesetupthousandsofcentersacrossIndiatogiveaccesstosharedcomputerswithpriorityforthedeafanddisabled.However,APFcourseworkuseslargelyaudio-basedlocallanguageinstructionandDIRFaimstogiveopportunityfortechnicalliteracyforthedeaf.
VeryfewofourrelatedsourcesfocusontheconsiderationsofearlyDeafeducationoriftheydo,assumeahigherthanpresentfamilialsupportnetwork[[20]].Weaimedtocreateassistivetoolsthatwouldstandaloneinfunctionbutnotreplacethepoweroftheteacher.Existingprojectsconsideredaccessibilityinonesense,butnoneaddressedthekeyconstraintsattheMathruCenter:lowtechnicalliteracyandstudentswithlittlemeansforcommunication.
OurworkidentifiedandaddressedtwomainhurdlesthatexistformarginalizeddeafIndiansandtheiraccesstolanguageandcommunication:firstly,formallanguageexpressionandformationthroughsignlanguage,andsecondly,awarenessofvoiceandtheeffectitmayhave.Theformerrequiresmoretrainingandsupportbyteachers,whoneedtoteachandencouragesignlanguageacquisitiontostudents[[21]].Thelatterisanecessaryprerequisiteforverballanguageformationbutistoooftencoupledexclusivelywithspeechtherapy.Ourworkfollowstheideathatawarenessofvoiceandthepowerithascanbeadditionallysupportiveinformallanguageformationandlearnedwithouttheneedofatrainedspeechtherapist.
8
ThegoaloftheTechBridgeWorld-MathrupartnershipandtheworkofthisteamhasbeentocreateassistivetechnologiesthatserveandempowertheMathruCentercommunity.BasedonapriorneedsassessmentattheMathruCenter(furtherdetailedlaterinthisreport)thechosenfocusofourworkistoexplorelanguageaccessattheMathruCenterviabothsignedandverbalmeans.Whilecreatingagencythroughourcustomsignlanguagerepository,wealsoacknowledgetheneedoftheMathrucommunitytousespeechtherapyforinstructioninimmediateskillstoassiststudents’accesstothebroaderIndiansociety.TheseprojectsbegintotacklethegroundworkoftechnologytoassistthecommunicationofthedeafandhardofhearinginIndia:bothinanincreasedrecognitionandaccesstolocalsignlanguagesandinthetrainingofverballanguagethroughspeechandone’svoice.
ISTEP2013FINDINGSThe2013iSTEPteamperformedaneedsassessmentattheMathruCentertoidentify
keyareasofdifficultyorcomplexityinteachingtothedifferentlyabledstudentsattheCenter.Thisincludedatwo-foldconsiderationofteachingmethodologyandstudentneeds.Throughobservations,shadowing,affinitymatching,andinterviews,the2013teamidentifiedkeyproblemareasinhibitingteacherefficacy[[17]].
Thefindingsthroughthisneedsassessmentrecognizedthatthemostprevalentteachingtool,flashcards,hadbothtemporalandphysicalrestraints.Temporally,flashcardstookalongtimetosortbysubjectareaanddifficulty,leavingteacherswithlesstimetoteach.Physically,flashcardsarealimitedmediumrequiringteacherstopresentcardstoanentiregroupwithvaryingwaysoflearning,makingitchallengingformaterialtobecustomizedtoeachstudent’sneeds.Teachersprovidealotofindividualattentionbecausestudentsareatvaryingskilllevels,butthatprovedtobechallengingwithexistingtoolsandresources.Interestingly,thereportnotedthatteachersfeltthatstudentslearnedfasterfromeachother.
Anothersignificantlimitationidentifiedwasteachertraining.BecausetheMathruCenterisinitsinfancyandIndiaasawholelacksteacherstrainedineducationfordeafnessandmultiplesensoryimpairments,theMathruCenteratthetimehadonlyonetrainedteacherduringthesummerof2013.Thus,theyrelyinpartonteacherswithoutexperienceortraininginspecialeducationtoleadclassrooms,whoselackofbackgroundexperienceandhighturnoverleadstomanychallengesrelatingtotrainingandclassroomefficacy.
Figure2.Flashcardsusedattheschool
9
The2013iSTEPteamsuggestedafewproblemspacesinwhichanewtechnologysolutioncouldbeuseful.ThemainsuggestionsforteachersincludedateachertrainingreferenceforteacherstodeveloptheirskillsandteachingstylefrompublishedonlineresourcesandcompilednotesfromtrainedMathruCenterstaff.Thisincludedasignlanguagetrainingtoolforuntrainedteachers,butthiswasnotedasbeingdifficultasthesignlanguageusedattheMathruCenterisadialectofIndianSignLanguagethathasverylittleexistingdocumentation.
Anotherrecommendationforfutureworkwasaflashcardgeneratorandlessonplanningtechnology.Thesedirectlyrespondtoteachers’observeddifficultiesinthetimespentpreparinglessonplans.Suchatool’sdatabaseofwordswouldneedtoconsiderawiderangeofwordsandhighqualityimagesthatarealsoculturallyrelevantandavailable.ItwouldalsoneedtorecognizethattheMathruCentercurrentlydoesnothaveInternetaccess,andthuswouldneedtoworklocallyonthecomputersattheMathruCenter.
Thereportalsonotedthatteachingspeechandwordpronunciationwastime-intensiveanddifficultforteachers,thusleadingtoasuggestionofspeechpracticesoftware.Thissoftwarecouldhelpassessandgivefeedbacktostudentsaboutthecorrectnessofsoundstheyaremaking.
OthernotedtechnologysolutionsuggestionsincludedComputerTraining,Sign/AlphabetTranslating,andaMulti-SensoryClassroomAssistanttosupportthewiderangeofstudentsattendingtheMathruCenter.
MOTIVATIONSTheiSTEP2015researchteamaimedtoaddresstheforemostchallengesaffecting
teachersandstudentsattheMathruCenterwithnewlydevelopedandcustomizedtechnologysolutions.AccordingtotheiSTEP2013team’sneedsassessmentattheMathruCenter,oneofthebiggestchallengesattheMathruCenterwasteacherpreparationandtraining.Inadditiontovalidatingthisneedthroughconversationswithheadstaffandteachers,the2015teamalsoassessedadditionalneedsoftheMathruCenterwheretechnologymayplayarole.Asaresult,wehaveiterateddesignsanddeployedearlyversionsofeducationaltechnologiesthataddresssignlanguageaccessandspeechtherapyattheMathruCenter.
ThisreportinformstheworkoffutureteamsandtheTBWresearchgrouponassistiveeducationaltechnologiesinitiatedduringthe9-weekfieldresearchinternshipattheMathruCenter.Secondarily,thisreportcanservetoeducateonlookersoftheongoingpartnershipbetweenTBWandtheMathruTrust,asthisreportdemonstratestheparticularanduniqueconsiderationstakenwhendevelopingforthedeafandhardofhearingcommunity.Lastly,thisreporthopestoexpresstheimportanceofownershipandempowermentwhenworkingwithdevelopingcommunities.Inparticular,thisprojectcontributestothegrowingbodyofwork
10
aimedtoholisticallydeveloptechnologythatempowersandfreesratherthanimposesandassumes.
Theremainderofthisreportwillprovideanin-depthlookattheiSTEP2015projects.Wewilldiscussmethodology,studystructure,datacollectiontactics,unforeseenconsiderations,andrelatedworksindevelopingoursolutions.Wewilldescribethesolutionsdevelopedandthenconcludeourreportwithadiscussionaboutourreflectionsandourconsiderationsforfuturework.
LANGUAGEACCESSTOOLSThe2015teamarrivedattheMathruCenterforthestartofthe2015-16schoolyear.It
wasimportantforustovalidatetheneedsuncoveredbythe2013team,particularlygiventhattwoyearshadpassedandtheMathruCenterisayounginstitution.Inourfirstthreeweeksintothesummer,twooutofthethreeexistingteachersforthedeafandhardofhearingquitandallthenewhireswerenottrainedineducatingdeafandhardofhearingstudents.OnereasonforthehighturnoverofteachersattheMathruCentercouldbetiedtoIndianculture:manyteachersareyoung,unmarriedwomen,whooftenarepressuredtoleaveteachingwhenexpectationsofmarriagearise.Ontopofthesechallenges,werealizedthatthefindingsofiSTEP2013assumedahigher-than-presentleveloftechnologicalfeasibilityintheclassroomsoftheMathruCenter.Hence,ourprojectsneededtoconsideranothergroundupneedsassessmenttoidentifyfeasibletechnologysolutions,ifrelevant.
WediscusseddifficultiesexperiencedattheMathruCenterwithbothexitingandnewteachers,aswellasadministrators.Teacherswereconcernedwiththeshynessofexpressionandcommunicationoftheirstudents.Studentswouldnotbeabletoexpress,throughsign,verbally,orotherwise,whentheywereupsetorconfused.Thisisprobablylinkedtothevaryingdynamicoftheteachingstaffandinconsistencieswithin-classbehavioralrules.Further,aspointedoutinthe2013report,studentshavedifficultiescommunicatingwitheachotherduetodifferencesinthesignlanguageusedattheschool.Administrators,ontheotherhand,wereconcernedwiththelackofteachertrainingtoolsandstudentdifficultywithverbalcommunication.Thelackofteachertrainingtoolsresultedinalargelearningcurvefornewteachers,andalesseffectiveuseofclasstime.Students’difficultiesinspeechwereaconcernfortheadministratorsbecause,outsideoftheMathruCenter,studentswouldneedtorelyonverbalmeansasthemainmethodofcommunication.
Weusedtheseexperiencestodirectourprojectscreatedinthefield.OurprocessconsistedofspendingtwodaysaweekattheMathruCenterforobservations,interviewsandusertesting.Ourfieldnotescompriseofhandwrittennotes,photos,andvideos.
STUDYSETTING
11
TheMathruCenterisaresidentialschoolcomplexthathasdevelopedthefollowingservices:
• FreeHostelforDifferentlyAbledChildren• CenterforDeafblindandMulti-SensoryImpaired(MSI)Children• VocationalTrainingCenterforDisabledandWomen• FreeSchoolforMSIChildren• FreeSchoolforDeafandHardofHearingChildren(4classrooms)
Ourobservationsandinterviewswereheldafterdailyclassesended.OurregularpointofcontactwasthemostseniorteacherattheMathruCenter,whogavedirectfeedbackonourideasandshapedourtoolsintovaluesensitivetechnologyapplicabletotheclassroom.Weconsultedstudentsandnewteachersmultipletimesduringourdevelopmentphase,keepingtheMathruCentercommunityincludedinallpartsofcreatingourlanguageaccesstools. Afterdevelopingourlanguageaccesstools,usertestingin-andoutsideoftheclassroomwasconductedusinglaptops
donatedbyTBWtotheMathruCenter.Laptopswerepurchasedbecauseteacherspreferredusinglaptopsoverusingtheexistingdesktopcomputersattheschoolandasetoftwotabletsdonatedtotheschool.
STUDYPARTICIPANTS WeobtainedfullparticipationfromallteachersintheCenteraccordingtotheCarnegieMellonUniversityInternalReviewBoard(IRB)protocolfornon-invasiveworkwithhumansubjects–includingtheteacherswhoresignedduringourtimeattheCenter.Weobtainedconsentfromallstudentsinthedeafandhardofhearingclasses,firstthroughfourthstandard,forparticipationinthestudy.Atleasttwoteammemberswereinvolvedineveryinformedconsentprocess.OurinterviewswithteachersandstudentswereconductedinEnglish,sometimeswitheitherastaffmemberorotherteacherpresenttotranslateintoeitherKannadaorHindi.
FINDINGSANDDEVELOPMENT
Figure3.Firstgradespeechclass
12
Whenconsideringsolutionstotheneedsfoundin2013,weforemostfoundalackofsignlanguagedocumentationusedbytheMathruCentercommunity.Lackofsigndocumentationiswhatpredominatelykeepsnewteachersfromgaininganunderstandingofstudentaptitude-asignificantneedconsideringthewiderangeofabilityattheMathruCenter.Becauseteachersneededtolearnsignsfromstudentsorotherteachers,commonclassroompowerbalanceswerechallenged.ThisprovidesaninterestingexistingsocialstructureforapotentialsignlanguagereferencemadebyMathruteachersandstudentsforotherMathruteachersandstudents.
Wealsosawthatteachersforthedeafandhardofhearingcontinuedtoincludeaspeechclassduringthedaydespitenotbeingabletoaffordprofessionalspeechtherapy.Basicsoundsandeverydayvocabularyarepracticedviaaspeak-and-repeatmethod,pairedwithbringingstudents’handsuptoateacher’sthroattofeelthevibrationsandemphasizinglip-reading.Themainlimitationsofspeechclasswerethatstudentswereshytospeak,wouldnotknowthedifferencebetweenmouthingwordsandspeaking,andtheteachercouldnotprovideextensiveindividualattention.TheseobservationsarenotapplicabletotheMSIclassroom,inwhichthecurriculumincludesindividualizedplanscreatedwithquarterlybenchmarkstowardsanannualgoal.SomeMSIstudentsthenhavemorespeechtrainingifrelevanttotheirpersonalgoal,thoughthe2015teamdidnotobservethosesessions.
AlthoughwehadinitiallyhopedtoutilizetheexistingcomputerlabattheCenter,thecomputerlabwasnotareliableresource.Thecomputerswereunderutilizedandnotmaintained,manyofwhichwerevirus-riddenorlackingfunctionalharddrives.Duetothisandaninterestinbringingtechnologytotheclassroom,theMathruTrustandTBWinvestedintwoLenovolaptopstohouseouriSTEP2015projects.Teacherschosetheselaptopsovertabletsbecausetheywill“givestudentsabetterideaofacomputer.”Wefoundthatstudentsandteacherswerehappytoadaptnewwaysoflearningandteachingbutdidnothavetheconfidencetodemandsuchresourcesthemselves.TobeginworkingwiththecomplexandnewenvironmentoftheMathruCenter,ourdevelopmentwascatalyzedbytheinformationprovidedbytheiSTEP2013teamandpriorknowledgeheldbyiSTEP2015teammembers.
SIGNBOOK:ASIGNLANGUAGEDICTIONARY
Deafcommunitiesaroundtheworldutilizeadiversityofsignlanguages,butonlyuntiltherecentpasthavesignlanguagesbeguntoreceiverecognitionasfullycomplexandstructuredlanguagescapableofrichmeaningandabstractexpression[[15]].Thisisinpartduetomisconceptionsofthedeafasthe“deafanddumb”or“deafmute”andofsignlanguageasalanguageofsimplisticgestures[[22]].Stilltoday,accordingtotheWorldFederationfortheDeaf,only44ofabout120countriesrecognizesignlanguageasvalidlanguagesforuseinschoolsandotherofficialmatters(suchasingovernment)[[23],[15]].
13
TheEthnologue,whichisthemostcomprehensiverecordoftheworldlanguagesthatexist,compiledbyhundredsoflinguistsandresearcherssince1951withover7,000livinglanguageslistedandcategorized,notesamere137documented‘Deafsignlanguages’intheworldwithapproximately10.5millionspeakers[[24]].Comparedtotheestimated360millionwithdisablinghearingloss[[1]],evenknowingthatmanyofthosedonotusesignlanguage,thereseemstobeevidenceofagapofdiscoveredandresearchedsignlanguagesandDeafcommunities.
Asonelinguisticfieldresearchernotesaboutsign(visual)languages:
…manualvisuallanguageswerenotrecognizedaslanguagesatall.Rather,as
Bloomfieldput it,“…elaboratesystemsofgesture,deaf-and-dumb language,
…andsoon,turnout,uponinspection,tobemerelyderivativesoflanguage”
(1933, p. 144). Severe under-documentation of an entire class of natural
human languages is the unfortunate legacy of this historical failure to
recognizeandtoattendtosign languagesas languages.Todate,“Therehas
neverbeenathoroughsurveyoftheworld’ssignlanguages”(Meier,2000,p.
1965), and most of the world’s manual visual languages have yet to be
described.[[25]]
Hegoesontowritethattheunder-documentationandrecognitionofsignlanguagesisespeciallytrueof“village”or“indigenous”signlanguages,whichisthetypeusedattheMathruCenter.Thisunfortunatehistoryofsignlanguagesbeingdisregardedleavesaholeintheworld’srecognitionofnotonlythelanguageoftheDeaf,buttheknowledgeofDeafculturesandtheneedsofthislargelyignoredandundervaluedsegmentoftheglobalpopulation. Ofeffortsmadetoassistthedeafindevelopingcommunities,manydonottakeapeople-centeredapproachinrecognizingthevoiceandthoughtsofthedeaftheyaimtohelp.Forexample,muchphilanthropicworkhasbeendonetoassisttheDeafcommunityinKenya,includingthedevelopmentofasignlanguagedictionaryonCDanddistributedwithinKenya[[26]].However,alocalDeafKenyanspeaksofthisandotherprojectsquitenegatively,saying:“theseorganizationshaveneverappreciatedcommunityentryapproaches,viewtheDeafasobjectsofbenevolenceandnotpartnersindevelopmentandare‘copy/pasting’foreignsolutionstocomplexlocalissues”[[27]]. WiththedevelopmentofSignBook,theiSTEPteam’sapproachvaluedthelocalsignlanguageasavalidlanguageandencourageditsuseforeducationandcommunicationattheMathruCenter.Throughouttheproject,wecloselylistenedtotheneedsandperspectivesoftheteachersofthedeafandhardofhearingandthestudentsfromtheverybeginning.Throughthatwecametoasetoffeaturesthataddressedtheneedsofteachertraining,having
14
asignlanguagereference,andprovidingacustomizable,efficient,andreusableteachingtool.Together,wecreatedSignBook,adigitaltoolforcreatingacommunity-constructedsignlanguagedictionary.WehopethisworkcontributesnotonlytoassisttheneedsoftheMathruCenter,butalsotofurtheracommunity-centeredvisionofICTDwork,documentourstepsincross-culturalcreationofassistivetechnologyforthedeaf,andmorebroadly,promoteawarenessofthelegitimacyofsignlanguageandtheneedsanddesiresofthedeafaroundtheworldsothatmoreandmoreofthedeafpopulationcanbeempoweredtowardsbecomingincreasinglyvaluedmembersofsocietythroughadequateaccesstolanguageandeducation.RelatedWorks ThisprojectisthefirstthatempowersDeafcommunitiestobeabletodocumenttheirownsignlanguagethroughadigitalapplication.AdditionallyremarkableisthatnopriorICTDworkaddressesdeafsignlanguagelearning[[28]].ThoughthisisarelativelyoriginalprojectconceptandcustomizedtotheneedsoftheMathruCenter,severalcomponentsoftheprojectdobuildofftheconceptsandideasofexistingworks.
Theideaofavideo-basedsignlanguagedictionaryhasbeenaroundforover15years.OnlineareseveralsignlanguagedictionarieswiththousandsofwordsandvideosofsignsinAmericanSignLanguage[[29],[30],[31],[32]].Themostfeature-richofthese,SigningSavvy,featureswordentrieswithdescriptions,images,memoryaids,Englishexamples,synonyms,abilitytoprintsignsoutonpaper,multiplevideosiftherearemultiplemeanings,andmore.Italsoallowsfortheabilitytocreatewordlistsofyourownthatcanbeusedasflashcardsorinquizzesandhasamobileappforsmartphonesandtablets[[32]].
Theideaofacommunityconstructedandcustomizeddictionaryhasalsobeenimplementedonlineindifferentways.Onemethodhasbeenthroughsignlanguagewikisthathavebeenestablished,usuallywiththehelpofaresearchgroup,thatutilizethecollaborativepowerofwikiswithsignvideospostedthroughYouTubeorsimilarmeansthatallowonetorecordavideofromtheirowncomputerwebcam.ThesesiteshavebeenreplicatedandusedforawidevarietyofsignlanguagesincludingFinnishSignLanguageandGermanSignLanguage[[33],[34]].
Anothercommunityconstructedsignlanguagedictionarysite,SignPuddle,usesawrittenversionofsignlanguagecalled
SignWriting(seefigure4),originatedin1974inDenmark[[35]].SignPuddlehasatoolusedto
Figure4.AsamplingofAmericanSignLanguagewordswrittenwithSignWriting
15
createsignsinSignWritingandthenallowonetoaddittothedatabaseanddictionaryofsigns[[36]].
Lastly,anunreleasedtechnologydevicetitledUNIwhichmainlyservestotranslatesignlanguagetospeechandtextisalsomarketingtoincludewithitsrelease“theworld'sfirsteasytousesignaddingsoftware”forcreatingcustomizablesignlanguagedictionariesaswellascloudmanagementsoftwaretouploadsigns,sharesigns,andfurthercombinesigndictionariestogether[[37]].
Takingalloftheseexistingsolutionsintoaccount,thereismuchtodrawfrombutnoexistingsolutioncouldaddresstheMathruCenter’sneeds.Mostimportantly,theMathruCentercurrentlyhasnoInternetaccess,renderinganyonlinesignlanguagedictionaryoutofreach.Further,theschooldoesnothavethecapabilitytolearnanduseawrittensystemofsignlanguagetodocumenttheirlocalsignlanguage.Manyexistinginterfacesarealsotime-intensivetodocumentsigns,suchastheprocessofcreatinganduploadingaYouTubevideothencreatinganewwikipageforeachsign,oneatatime.ThusanidealsolutionfortheMathruCenterwouldallowforaquickwaytorecordsignvideosandstorethemaswordsinanorganizeddatabaseofsignsthatarealsoquicklyaccessibleasadictionaryreference.
Wealsolookedindepthatthevisualpresentationofmaterialsinaneducationalsetting[[38]]andconsultedotherdigitalflashcardapplicationstoshapethevisualpresentationforSignBookincludingQuizlet,KitzKitz,andCram[[39],[40],[41]].Quizletallowsuserstoplacebothimagesandtextonflashcards,andrecordaudiotogoalongwithit.KitzKitz’ssolepurposewasprintingflashcards,however,wefoundtheinterfacedifficult.Cramoffereddifferentviews(mixturesoffrontandback)oftheflashcards. Methodologies
“Anotherindicationofthestatusofsignlanguageinacountryistheexistence
ofoneormoresignlanguagedictionaries.Adictionaryisafundamentaltoolto
bothpromotethestatusofandenhancetheuseofsignlanguage”[[16]].
DuringthefirstthreetofourweeksattheMathruCentertheteamengagedinmanyconversationswithteachers,spentmuchtimeobservinginclassrooms,andevenhadeverydaycasualinteractionsthathelpedtheteamslowlybutsurelyunderstandmoreabouttheschoolandbuildconnectionswiththeteachersandstudents.
TheteamhadbeeninitiallytoldthattheschoolusedAmericanSignLanguage(ASL).Oneoftheteammembers,havingabackgroundinASL,recognizedafewsignsincommonwithASL,butquicklyrealizedthattheschoolusedanothersignlanguagealtogether.Askingtheprojectmanagermoreaboutthesignlanguagethatwasused,theteamwasgivenanIndian
16
SignLanguage(ISL)dictionarytitled“TalkingHands”[[42]].However,thissetofsignsalsohadsomeoverlapbutwasfoundtodefinitelynotcorrespondtothesignlanguageusedattheMathruCenter.Eventually,onebookwasfoundwithbeginnersignswithKannadacaptions(thelocallanguageofBangalore)witharound480signscontainedinit.Thisbookwastheclosestwefoundtothelocallyusedsignlanguage,buttherewerestillsomemajordifferencesbetweenthebookandobservedsignlanguage,usuallyobservedwhensignsseemedtobeadoptedfromASLandISLsuchasthedaysoftheweekbeingthesameasASL.TheKannadabookofsignsalsocontainedonlynouns,lackingevenbasic
pronounsandverbs,whicharevitallynecessaryforlanguageandexpression.
ThisseriesofobservationsandfindingsledtothemaindiscoverythatinspiredthecreationofSignBook:thereisnoexistingcomprehensivereferenceordictionaryforthesignlanguageusedattheMathruCenter.Thesuggestionofaschooldictionarywasidentifiedinthe2013reportabouttheMathruCenter,butknowledgeofjusthowscarcetheresourceswereforthelocalsignlanguagewassurprisingtotheteamandhighlightedtheneedforsomeformofcommunityreferencetolearnfromandtobeusedinteachertraining.
Throughclassroomobservationswefurthernoticedthatteachersoftenusedfragmentedsignlanguagesentenceswhilethechildrenusedmorecompletesentences,astheyreliedonsignlanguageastheirmainformofcommunication.Manyteachers,especiallynewteachers,woulduseoneortwosignsinasentenceandtalkloudlythefullEnglishorKannadasentencetheyweresaying.However,thechildrenwouldoftenlookconfusedifthesignandfacialexpressionoftheteacherdidnotfullycapturethesentencemeaning.Thisindicatedthatthestudentswereheavilyreliantuponsignwhiletheteachersweremoreverballyfocusedduetoalackoffluencyinsignlanguage.Theseobservationsfurtherunderscoredtheneedforasignlanguagedictionaryreferenceforteacherstolearn,train,andpracticefrom.
Animportantnoteonourmethodologyintheseobservationsisthatweaimedtobeasunimposingandnonintrusiveaspossible.Inmanyearlyobservations,theteacherwouldasktheASLspeakingteammember,whattheAmericansignswerefordifferentwords,andsubsequentlyintroducethemalongwithherlessonplanasalternativesigns.Realizingthatthiscouldaddcomplicationtoteachingtheprimarilyusedlocalsignlanguage,showingASLsignswasstoppedduringobservations.
Figure5.“MyNativeTongue–1”TheonlyofficialsignlanguagereferencebookattheMathruCenter
17
ThedescribedseriesofeventsfollowsagreaterphenomenoncommonincultureswithvillagesignlanguagesasdescribedinResearchMethodsinSignLanguageStudies:APractical
Guide:
“Oneofthereasonswhysignlanguagesmayremainundocumentedforalong
timeistheircoexistencewithalargerormoreprestigioussignlanguage,which
often leads to a negative perception of the local sign language and to a
subsequenttendencytonotovertly identifywith itsuseand/ortoshifttothe
prestigioussignlanguage.”[[43]]
ThisquoteexplainsreasonswhytheschoolstaffperhapsdidnotidentifyashavingalocalsignlanguageatthebeginningandwereeagertolearnfromamoreprestigiousandwidelyusedsignlanguagelikeASL.
Otherinsightsgleanedfromclassroomobservationswerethatmanylessonsaretopic-driven.Everyclassobservedhadasetofwordsorfactsforthestudentstolearnbytheendoftheperiod.Thistopic-driventeachingstyleledtothetopic-organizedcategorizationsystemofSignBook.
Classeswerealsoveryvisuallybased.Whenateachercouldnotfindaflashcardorposterforthesubjecttodescribe,teacherswouldsometimesspenduptoseveralminutesdrawinganintricatedrawingonthechalkboardshowingthesubjectmatterwhilethestudentswereleftunengagedwithanymaterial.Evenafterschool,asthe2013reportmentions,teachersspenthourspreparingcut-outsofpictures,organizingflashcards,andhand-drawingotherdrawingsforuseinthelessonplanandinclass.ThismotivatedtheinclusionofpicturesinSignBook,andmethodsforuploadingexistingimagesortakingpicturestoallowforanyconceivableandsimplewaytoaddimagestodigitaldictionaryentries.Goingoverthisideawithoneteacher,sheremarkedthatitcantakemuchdescriptionandefforttoconnectawrittenwordandvisualsigntogethertorepresentaunifiedconceptforadeafchild,butseeingtheSignBookdictionaryentry,thestudentwillbeabletoinstantlyunderstandtheconcept.
Figure6.Anintricatedrawingbyateacher,usedtoexplainaclassroomlessoninthefirstgradeclassroom.
18
DevelopmentEarlyinterviewswith
teachersrevealedapreferencetowardlaptopapplicationsovertabletapplications.Basedontheteam’sprogrammingknowledgeandtherequirementsoftheapplication,theprogrammingenvironmentwaschosentobeinweblanguages(HTML,CSS,JavaScript,PHP)butbecompletelylocallyhosted,meaningthewebapplicationwouldnotrelyontheinternetbutuseonlyfilesonthecomputeritself.Thischoiceallowedformorerapidfrontenddevelopment,cross-platformcompatibility,andfuturepotentialoftransferringtheapplicationonlineforbroaderuseandavailability.Thisalsopromotesthelearningofcommoncomputerliteracyskillsdesiredbytheschool,touseaweb-basedapplicationonalaptop.ManyopensourcelibrarieswereusedincludingGoogle’sAngularframework,RecordRTCforvideoandpicturecapture,andPHPDesktopforallowingtheapplicationtoworklikeanativeWindowsapplication.
ThefeaturesoftheapplicationweredevelopedincorrespondencetotheneedsoftheMathruCenteranditerativelydevelopedandmodifiedbasedonusertestswithteachers.Table1(below)featuresdevelopedoverthecourseofthenineweeks,theirdescriptions,andwhatneedstheywerebasedupon.
Feature Description NeedAddressedVideoCapture Recordingtooltoquicklycreatea
.webmvideofilefromthewebcamandconnectitwithadictionaryentry
Simpleandfastwayforteacherstodocumentsigns
PictureCapture Similarcapturingtoolfora.gifpicturefromthewebcamtoconnectwithanentry
Fastwayto“scan”existingflashcardsandimagesortoshowphysicalobjectsandplaces
PictureUpload Uploadanyimagefileasapictureforanentry
Canusepicturesfromothersourcestoaddimagesnoteasilyfoundnearbyortakenviawebcam
Topical/FolderOrganization
Organizewordsintotopicsandsub-topicsinafolderhierarchy
Organizethedictionarysimilarlytothetopicalnatureofclassroomlessons
Figure7.TheSignBookinterface
19
CRUDOperations Create,Read,Update,andDeletedictionaryentriesandtopicfolders
Necessaryactionsformanagementofanydatabasesystem
MultipleVideos Canhavemorethanonesignvideowithaword
Allowsforsignsynonyms,signvariations,orevenmultiplesignlanguagesassociatedwithoneentry(alocalsign,ASLsign,etc.)
MultiplePicturesandMeanings
Canhavemorethanonepictureandmeaninginanentry
AllowsforasignhavingmorethanoneEnglishmeaningandmultiplevisualrepresentationsofaword
ViewModeShow/HideButtons
Show/hideanycombinationofanentry’svideo,picture,andmeaninginViewMode
Flash-cardlikeusageofthesigndictionary(showthesign,hidetheword,quizthestudent)
Cross-PlatformCompatibility
TheapplicationrunsonWindowsand(ifChromeisinstalled)Macstoo
CurrentdevelopmentwasdoneonMacs,withschoolusageonWindows
FastLoadTimes Navigatingbetweenentriesisdoneasynchronously(withoutnewpageloads)
Quickertouseandalsolesserrorproneinreadingandwritingfromthedatabase
KeyboardShortcutsforNavigation
Arrowkeysjumpbetweenwords,otherkeysactasshow/hideornext/previousshortcuts
Mousemovementsbetweenmanyfrequentlyusedbuttonsproveddifficultandforteacherstouseandslowedthemdown
HelpText/HoverText/VideoOverlays
Manyinstructionsandfunctionalityareprovidedviaon-screentext,whenyouhoveroverabutton,oroverthevideo/picturecapturescreen
On-demandhelpandinstructionsassistnewusersinmorequicklylearningandrememberinghowtousetheapplication
DatabaseBackups Backupfilesaresavedoftwokeydatastructuresonceadayandrecordedvideosareneverdeletedfromthecomputer
Datarecoveryiscriticaltobeabletodowhensomethinggoeswrong,anddatalosscoulddiscourageuseoftheapplication
Table1.SignBookTableofFeatures
StudyResultsandReflectionsTowardstheendofourfieldwork,wetrainedthemostseniorteacherforthedeafand
hardofhearingstudentsandthemainMathruCenteradministratoronallaspectsofSignBookwiththreeextendedtrainingsessions,aswellastwootherteachersonthebasicsofthetoolwithonetrainingsessioneach.Throughthistraining,teacherswereabletocreateaninitialvocabularyof30wordsacross7topics,almostallentriesincludingthethreeentrycomponentsofaword,video,andpicture.
20
TeacherssharedtheirresponsestoSignBookduringtrainingwithpositiveandexcitedcomments.Oneteacherremarked:
“This isverygoodbecauseIcanlearnonmyownandbefineteachingtothe
children.IfIask[theotherteacher]shewillteachmeafter4:00PM,butnowI
canaskandshewillenterinthis.Iwillseeagain,repeated,howtodothesign
–itistheeasyway!Icaneasilyseethisandthenteachtothechildren.”
OneofthemainfundersandsupportersoftheMathruTrust,uponseeingademonstrationofSignBookcommentedthat“Nowanyonecandocumenttheirsignlanguage,andevengiveacopytootherstolearn!Thishassomuchpotential.Potentialtoprovideaccesstoteachers,families,andcommunitiestocommunicatewiththedeaf.”
Therewasnotsufficienttimetoobserveclassroomuseofthetool,butaninformalmomentaroseattheendofateachertrainingwhen15childrengatheredaroundtolookatwhattheteacherwasdoing.AteammemberledthechildrenthroughViewModeforalloftheexistingwordssofar,showingandhidingthewordsandvideostoquizthestudents.Theyexcitedlyfinger-spelledorsignedtheanswersinresponseandexchangedathumbs-upandhandshakewhengettingtheanswercorrect.
ThefutureofSignBookisyettobeknown,butinitialreactions,users’quicklearninganduseofthetool,andexcitementfromteachers,students,andadministratorsshowpromisingpotential.MuchworkremainstobedoneinprovidingsignlanguageaccesstoDeafcommunitiesaroundtheworld,especiallythekindsoflocalvillagesignlanguagesasdiscoveredattheMathruCenter.SignBookisauniqueandexcitingstartinusingmoderntechnologytowardstheefforttoprovidethiskindofaccess.WehopetheworkanddocumentationofthisprojectwillcontinueontoinspireeffortinICTDworkandassistivetechnologyforthedeafandhardofhearingtogivethosewhomaynothaveearsthathearagreaterabilitytolearn,communicate,anddiscovermoreoftheknowledgeandexperiencestobehadinthegreatwideworldallaroundthem,richinvocabulary,visuals,andmeaning.
AdemoofSignBookisavailableonlineathttp://erikpintar.com/signBookDemo.
SPEAKUP!VOICEPOWEREDGAMESUITE
Themajorityoftheworldtakestheabilitytocommunicateverballyforgrantedeveryday.Throughouthistory,voicehasbeenthesocietalnormforcommunication;yetsuchcommunicationremainsinaccessibletodeafindividuals[[44]].InIndiathisisoftencoupledwiththeassumptionofdumbness–theinabilitytocommunicate.Whilethisassumptionisunderstandableindevelopingcommunitieswheredeafnessisoftentheresultofconditionsalsoaffectingmentalorotherphysicalability,itisinnowayanaccurategeneralizationofadeafperson’scapabilities.
21
TheSpeakUp!VoicePoweredGameSuite(SpeakUp!)wasinspiredbyaconversationwiththefounderoftheMathruTrust,Ms.GubbiMuktha.Shenotedboththeimportanceofspeechininteractingwithbroadersociety,aswellasexplainedthefinancialinfeasibilityofhiringprofessionalspeechtherapists.Shehadpreviouslyhiredaspeechtherapistforspeechclassesonceortwiceaweek,howevertherewas“notenoughprogress”tojustifythecost.Whengivensuchsmallamountsofdesignatedtimeforspeechtherapyandotherwiseremainingsilent,studentscannotmeaningfullyimprovetheirspeech.Whiletheexpertiseoftrainedspeechtherapistsisnecessaryforformallanguageacquisition,manyofthestudentsatMathruarepre-verbalandstruggletounderstandthepoweroftheirvoice.WewereinspiredtodevelopSpeakUp!asameansofencouragingstudentstoenjoyusingtheirvoiceandtodevelopmorecontrolovertheirvocalizations.
RelatedWorksBecauseIndiansocietypresentlydoesnotsupportanotionofDeafcommunity,thereis
anexpectationofthedeaftoassimilateintobroaderculture.“Peoplewithcommunicationdisabilitiesareoftenassumedtobeconsiderablylessablethantheyreallyare”[[45]].Thegoalofourvoice-poweredgamesistoempowerandencouragedeafandhardofhearingstudentstospeakforthemselves.Oursuiteofgamestargetspre-verballanguageskillsinafunandengagingwaybyhavingstudentsmodulatepitch,volume,andvocalization.
Speechtherapyisoneofthemostimportantroutesfordeafindividualstobeunderstoodbeyondacommunitythatusessignlanguage.Theimportanceofearlychildhoodspeechtherapyisalsocrucial,giventhatsoundformationandproductionbecomesfarmoredifficultwithage[[21]].Yet,prohibitivelyexpensivespeechtherapistfeeskeeptheMathruCenterfromincorporatingregularprofessionalspeechtherapyintotheirclassrooms.TheMathruCenteridentifiedmoresources/toolsforspeechtherapyasoneoftheirlargestneedsattheschool.
Whileopen-sourcespeechtechnologiesarestillunabletoproviderobustandindividualizedspeechtherapy,wefoundafewpackagesthatdosupportspeechtherapyforusewithatrainedtherapist.Onesuchpackage,VideoVoice3.0[[46]],requiresaspeechtherapisttoconfigurethegamesandpracticemodes,andallowsthespeechtherapisttotracktheirstudents’progress.ConsideringtheMathruCenterservesstudentswithwidelyvaryingcommunicationdisabilities,wehopethatthroughvoice-poweredgames,theMathruCenterstudentswillgainmorecomfortincommunicatingandexpressingthemselvesverballyoutsideoftheclassroom.Ourgamesusearangeofbasicaspectsofsound—volume,pitch,continuousvocalization,andtimedvocalization—tocontrolgameplay.Ourcodeisdocumentedandorganizedinhopesofencouragingfurtherdevelopment.Wedevelopedthegames’inputmetricsanduserinterfacethroughconversations,interviews,andtests,withtheconstanthopeofaidinginthemostpressingdifficultiesofspeech.
22
Wetransitionedfromspeechtherapytoolstogamesduetoourlackoftraininginprimaryeducationaswellastheinabilityforouropensourcesoftwaretorecognizesingularphonemes.Unlikehumans,speechrecognitionsoftwareoftenneedscontextualdatatoproperlyidentifywords,datathatwouldbeabsentwhenidentifyingisolatedphonemes[[47]].Phonemerecognitiontechniqueshaveachievedassmallasa25%errorrate[[48]],buteventhatistoolargefortheMathruCenter,giventhatmanystudentsarepre-verbal.Further,manymembersoftheMathruCentercommunitylacktechnologicalconfidence,sowewerewaryaboutusingerror-pronetechnologyasthebaseofoursuite.Hence,wedecidedtofocusonotheraspectsofsoundthataresimilarlyinstrumentalinachievingeffectiveverbalcommunication.
Methodologies WedesignedSpeakUp!toworkbestwithateacherorinstructorpresent.ThiskeepstheuseofSpeakUp!incontroloftheteacheranddoesnotimposetechniqueorstylelimitations.Ourgoalwastoallowstudentstovisualizetheirvoiceinasmanywaysaspossible.Providingdifferentinterpretationsofvoicetranslatesintoanunderstandingofthedifferenteffectsthatvolume,tone,andvocalizationhave.Wewantedtoofferavarietyofmodesandgameplay:modesthatsolelyprovidevisualizationsofvoice,modeswithnoendthatcanbeusedbyateacherinaclassroomsetting,andmodeswithacleargoalthatcanmotivatestudentstousedifferentaspectsofvoice.
TheSpeakUp!processwasinformedbyobservingteachers’speechclasses.The1stStandardteacherfocusedmostlyonindividualsounds.Shewentone-by-one,showedthestudentthelipandtonguemovements,heldstudentshandonherthroatorinfrontofhermouthsotheycouldfeelthevibrationsorair,andhadthemrepeatthesoundsmultipletimes.Thiswasverytaxingandtime-consumingfortheteacher,andbytheendhermouthwasdryandshewasexhausted.The4thStandardteacher,ontheotherhand,focusedonwordsandtaughtcommunally.Shewouldshowthestudentsobjectsandflashcards,andsaythecorrespondingwordloudly,accentuatingthelipandtonguemovements.Shewouldthenlaytheobjectstogetheronthetable,sayaword,andhavethestudentsidentifytheobjectshewasreferringto.Thisteacherlaterelaboratedonhertechnique.“Inthisschoolweusesigns,butoutsideoftheschoolpeopledon’t.Thatiswhywemustteachstudentstocatchlipmovements.”
Fromourclassroomobservations,wegainedanunderstandingofthebreadthofstudents’currentvocalcapacities.ItwasimportantforustodesignthevariousmodesofSpeakUp!toberelevantandusefultostudentsacrossthepre-verbalandpartially-verbalspectrum.ItwasalsocrucialtomakeSpeakUp!usablebymanydifferentsystemsandcomputers,notonlythenewestandmostexpensivemachines.
23
Development TheSpeakUp!projectbeganwithabroadsamplingofrelatedworksandunderstandinglocallimitationsoftechnologicalsolutions:lackofInternetaccess,unfamiliaritywithtechnology,andalreadyfullclassschedules.Ouroriginalidearevolvedaroundmoreconventionalspeechtherapytechniques.Wewantedtomakeanapplicationthattellsstudentstosayaspecificsound,andprovidesfeedbackonthesoundtheysaid(eitherbytellingthemwhattheysaidorscoringhowclosetheyweretothedesiredsound).However,afterasignificantamountofresearchintoopen-sourcespeechrecognitionAPIs,werealizedthattechnologytounderstandindividualphonemesdoesnotyetexist.Alargelimitationofspeechrecognitionsoftwareisthecommondependencyoncontextneededfordecipheringwordsfromsentences.Theproblemforusisthatthesesophisticatedstatisticalmethodsneedmorethanisolatedsounds–preciselywhatwewanttodetect.Forthesereasonsitbecameclearthatoursoundgamesweregoingtoneedateacher’sguidanceratherthanhavingitsownsoundrecognitionscheme.Further,withmoreresearchandobservationsattheMathruCenter,werealizedthattherearemanyaspectsofvoiceotherthanspeechthatourtechnologycanhelpstudentsattheCentergrasp. Westartedoffwithtwosimplegames:abattlegame,whichemphasizestimedvocalization,andarickshawgame,whichemphasizescontinuousvocalization.Wetestedthesetwogameswithateacherandthreestudentsafterschool,andthereactionswereverypositive.Theteacherthoughtitwasusefulinmakingspeech“fun”forstudents,andthestudentsseemedtoenjoyworkingtogetheronthesegames.Theteacherthengaveusfeedbackforfuturegames:shethoughtstudentswouldenjoy“funny”games,suchasagamerelatedtopickingfruitfromatree. WhileworkingontheFruitTreegame,wealsobegandevelopinggamesthatrespondtodifferentvolumeandpitchlevels.Wefeltitwasimportanttoshowteachersallthecapabilitiesofourgamesuitebeforefocusingononeortwoaspectsforgames.Asweshowedthesegamestoteachersandstudentsthroughoutprojectdevelopment,werealizedthattheconceptofvolumewasrelativelyeasytoconveytothem.Teachersfeltthatvolumegameswouldbeusefulinstrengtheningstudent’svoices,aswellasexplainingtothemwhichvolumestouseinwhichsituations.However,theconceptofpitchwasverydifficulttoexplain,andoftengotmiscommunicatedormisunderstoodasvolume.Basedonthisreaction,weshiftedfocusawayfrompitchgames,andmoretowardsvolumeandvocalizationgames.
Basedonourobservationsofthe1stStandardclass,wedevelopedmultipleFreePlaygames,whichrespondtoanystudentsoundandgoonforever,toallowstudentstopracticethesoundstheyaretaughtinawaythatisnottaxingtotheteacher.Basedonourobservationsof4thStandard,wemadeapicturegame;inwhichasthestudentsspeaksapicturegraduallyappears.Thisgameallowsteacherstoaddandcategorizetheirownwords
24
andpictures,andwillallowstudentstopracticesayingwordsandassociatingthemwithobjectsinamoreengagingway. Intheend,wedeveloped10modesacrossthreecategories.The“BeginnerSkills”modes:VolumeMeterandPitchMeter,focusedongivingsimplevisualizationsofaspectsofvoice.The“FreePlay”modes:FruitTree,Rickshaw,CollectTheFruit,andPictureThat!areendless,andareintendedtobeusedintheclassroomtohelpstudentspracticewords,sounds,andotheraspectsofspeech.The“Games”modes:Spaceships,DriveToMathru,FlappyBird,andFishGameallhavecleargoalsanddistinctends.Thesegameswereintendedtoencouragestudentsusedifferentaspectofvoice,andallowthemtohealthilycompetewitheachother.StudyResultsandReflections
InthelastthreeweeksinthefieldwefocusedontrainingtheteachersonhowtouseSpeakUp!andwereabletoobserveteachersusetheprograminclass.Oneteacherhadallthestudentssittingtogetheraroundthecomputerusingitcommunally.Oncertaingames,shewouldhavethemwatchherandmimicherlipmovementsforcertainwords.ThiswasespeciallytruewiththeFruitTreeandPictureThat!game,in
whichspecificobjectsappear.Withothergames,shegavestudentsinstructionsonhowtousethegames,andletthemmakeanysoundswiththeirvoicetooperatethem.
Teachersbegandesignatingthelibraryasthespeechclassspace.Thelibraryprovidesamoreopenspacewhereteachershavestudentsgatheraroundthelaptopwithoutobstacleslikedesksandchairsintheclassrooms.Oneteacheralsobroughtteachingaidestothelibrary(animalmodelsandflashcards),spreadthemout,andreferredtothemasstudentsusedthe
Figure8.TheSpeakUp!mainmenu
Figure9.AMathruCenterstudentusingtheSpeakUp!voicepoweredgamesuite
25
games.Thelibraryisalsoaquieterroom,isolatedfromotherclassrooms,whichprovideslessnoiseinterferenceinthegames.
WhileitistooearlytoassesstheimpactofSpeakUp!,theprogramseemspromisingbasedonformativeassessments.Therehavebeenclearbenefitsofthesoundgamestoindividualstudentsthatwehaveobserved.Onestudentalwaysmadethelipmovementforawordbutnevermadesound;shedidnotunderstandhowshecouldmakesound.However,afterjustafewminuteswithSpeakUp!shebeganmakingloudsounds,andcontinuedtomakethemthroughoutthegames.TeachersweresurprisedathowexcitedandengagedstudentsbecamewhenSpeakUp!gameswereused.Thoughteachersarestilltryingoutdifferentwaystousethegamesintheirclass,sofarPictureThat!isthemostcommonlyused,followedbygameslikeRickshawandVolumeMeter,whichallowstudentstopracticeanysoundtheteacherwantsthemtomake.
Therearesomeareasofconcernwehaveseensofar.Manyofthegameswereintendedforindividualstudentuse,butsometeacherspreferusingtheprogramcommunally.
Thedownsideofcommunaluseisthatstudentstendtobemakingsoundsinthebackground,soevenwhentheteachertriestofocusononestudent,otherstudents’voicesaretheonescausingthechangeonthescreen.Thiscanbeconfusingandcounter-instructiveforstudents,astheymaynotbespeakingandyetthegameisrecognizingvoice.Onepotentialsolution
wouldbetousethegameswithanexternalmicrophone,sinceseeingonestudent
holdingamicrophonewouldencourageothersnottotalk.Unfortunately,wewerenotabletotraintheteachersonexternalmicrophoneuse,butitisapotentialdesigndecisionworthexploringinthefuture.
WhenweobservedoneteacherusingSpeakUp!inclass,shewassoexcitedsheuseditfortwiceaslongasatypicalclassperiod.Shelargelyletstudentsuseitastheypleased,ratherthanincorporatingitintoherlesson.ThisraisestheconcernofwhetherteacherswillcontinuetouseSpeakUp!evenafterwearegone,becauseitmaynotdirectlyrelatetoschoolcurriculum.Unfortunately,wewerenotinthefieldlongenoughtoobservethenoveltyofSpeakUp!fade.Thus,allteachertraining,usertesting,andobservationswedidwereconfoundedbynotbeingabletoobservelong-termuse.However,duringourlastweekinthefield,wedidwitnessthenewestMathruCenterteacherusingthePictureThat!gamewithherthirdstandardclass.Whilewehadnottrainedherspecifically,shehadlearnedhowtousethesoftwarefromthefourthstandardteacherwehadtrainedtheweekprior.Sheenjoyedusing
Figure10.AteacherusingSpeakUp!communallywithherentireclass
26
thesoftwareandknewhowtoreopen,close,andshutdownthelaptops;allaspectsoftrainingwecoveredwiththeotherteachers.ThisdemonstratesthepotentialfortheMathruCentercommunitytoself-trainandspreadknowledgetofuturestaffandteachers.
DISCUSSIONWhentechnologyiscreatedwithcompassionandinrespectfulpartnershipwithusers,it
canhavethetransformativepowerofaffectingpositivechangeinthelivesofthedisempowered.TogetherwiththeMathruCenterfortheDeafandDifferently-abled(MathruCenter)andTechBridgeWorld,ourteamteamexploredhowtechnologycouldempowertheteachersandstudentsoftheCenter’sSchoolfortheDeaf.
BecauseofTechBridgeWorld’songoingrelationshipwiththeMathruEducationalTrustfortheBlind(MathruTrust)since2006,ourteamexperiencedthebenefitsthatcomewithastrongpartnership,suchastrustandrespect.WestillhadtobuildtrustanddidsobygettingtoknowMathruCenterteachers,students,andstaff.WealsoconductedaneedsassessmenttolearnabouttheMathruCenter.WhileapreviousiSTEPteamconductedasimilarneedsassessmentin2013,itwasusefulforourteamtounderstandthecurrentsituationattheMathruCenter.Asaresultofthe2013and2015needsassessmentfindings,wedevelopedtwoneweducationaltools.Thefirst,SignBook,providedgreatersignlanguageaccessthroughasignlanguagedictionarycreationtoolwithabilitiesforcustomvideoandpicturecapturingandcategorizationofentriesandwordsintotopics.Thesecond,SpeakUp!,motivatedstudentstostriveforgreaterverballanguageaccessthroughasuiteofvoice-poweredgamesaimedtofamiliarizepre-andpartially-verbaluserswiththepoweroftheirvoice.
ThetoolsweredevelopedinclosepartnershipwiththeMathruCenter.WetestedprototypesoftheSignBookandSpeakUp!toolswithMathruCenterstudentsandteachersandwiththeirfeedback,wefurtherimprovedthetools.Whenthetoolswereinastablestate,inthefinalweeksoftheinternship,weinvestedinthesustainabilityofthesolutionsbyconductingextensivetrainingwithteachersanddevelopingin-depthuserguides.Furthermore,TechBridgeWorldreleasedthetoolsopensourcesothatothersmaybenefitfromthesesolutionsaswellascontinuetoenhancethemindifferentcommunitysettings.
Figure11.MathruCenterteacherscreatingasigninSignBook
27
Withthesetools,whichwerecreatedwithcompassion,weaimedtoprovidelanguageaccessforDeafprimaryschoolstudentsandtheirteachers.OurworkwiththeMathruCentersupportedtheprocessofenablingIndia’sDeafcommunityintheircampaignforself-advocacythroughaccesstolanguage.Todisseminatethiscauseandraiseawarenessofdevelopingtechnologywithcompassion,theiSTEP2015teamgaveatalktoover100engineeringstudentsatPESUniversityinBangalore.Asaresultofthispresentation,aPESengineeringprofessorhasreachedouttoTechBridgeWorldaboutcollaboratingontechnologyprojectswiththeMathruEducationalTrustfortheBlind.
Thefutureofthisworkisbright.WhetheritisfurtherdevelopmentofSignBookorSpeakUp!oranadaptationofdevelopingtechnologywithcompassion,whattheiSTEP2015internshipwasabletodeliverafternineweeksinthefieldisanimportantrecordofcommunity-specificICTDworkandtwofunctioningprojectsbywhichtoshowcaseit.
ADDITIONALCONSIDERATIONSFORFUTUREWORK BytheendofnineshortweeksattheMathruCenter,theiSTEPteamwasabletocreateaSignBook,alocaldatabaseofanimationsdepictingasignandisusedtosupportnewteachers
andstaffinlearningthelocalsignlanguage,andSpeakUp!,asuiteofvoice-poweredgamesthatencouragesstudentstovocalizeandtoexercisethevolumeandpitchoftheirvoice.Muchmorecanbecontinuedinthisworktocreateassistivetechnologyforthedeafandhardofhearing. WithSignBook,theteamhadjustenoughtimetoincorporatethekeyfeaturesasderivedfromourobservationsandinterviewswithteachersandstudents.Thelargestadditionalrequestthatwedidnothavetimetoimplementwasforreviewandstudygamestogoalongwiththedictionarydatabase.Anexampleofareviewgamecouldallowateachertochooseanyofthethreedisplaytypes(video,picture,word)foraquestionandthenanotherdisplaytypeformultiplechoiceansweroptions.Thiscouldbeplayedwithallwordsorjustplaythroughonetopic’s
words.FunctionalityisalsolackingwithSignBooktomergewordsandtopicsfromtwodictionariestogether,whichlimitstheuseofSignBookacrosscomputersandwouldbean
Figure12.Technologyisoftentreatedasanovelty,sincestudentsdonotencountertechnologyineverydaysettings
28
excellentenhancement.FurtherideasforfutureworkaresummarizedinSignBook’sdeveloperdocumentationinitspubliconlinerepository1.
AsweworkedontheSpeakUp!gamesandwentthroughusertesting,werealizedthediverseusecasesitneededtoencompass.Thestudentshadwidelydifferentvocalcapabilitiesinareassuchasvolume,pitch,andphonemegeneration–theabilitytomakespecificsounds.Additionally,becauseourgameswereusedinextremelyvariantenvironmentsfromthenoisyclassroomtothesilentlibrary,itwasimpossibletomakegamesthataccountedforallthesedeviations.WebelievetechniquesfrommachinelearningcouldgreatlyenhancethefutureofSpeakUp!.Machinelearningwouldallowthecalibrationmenutoaccountfornotonlybackgroundnoiselevelsbutalsobackgroundnoisepatterns,suchasachirpingbird,alawnmower,orrelativesilence.MachinelearningingamesthatscalevolumeandpitchwouldallowSpeakUp!toshiftawayfromtheabsolutescaleitcurrentlyuses,toonethatcustomizesforeachstudent.Perhapsextremelysophisticatedmachinelearningtechniqueswouldallowgamestorecognizewhenstudentsaregettingclosertosayingaword,afeatureateacherrequestedinPictureThat!game.Unfortunately,wedidnothavethetimeorexpertisetopursuetheseoptionsinthefield.Therefore,wetriedtomakethegamesasdifferentlyusefulaspossible.Thereisstillalotofroomforimprovement.ThereisnodoubtthatmachinelearningtechniqueswillbeextremelyusefulinhelpingSpeakUp!reachthefurtherlevelsofusability,usefulness,andadaptability. OneofthemostimportantchallengesweencounteredwhenworkingwiththeMathruCenterwastechnologicalliteracy.Futureworkneedstobecarefulabouttheuseoftechnologythatiscurrent,butalsolocallyavailable.Accesstotechnologyislimited–whethertooexpensive,hardtofind,orotherwiseunavailable.TheMathruCenterdidnothaveInternetandlackedareliablecomputerlab.AmajordrawbacktotheCenter’scomputerlabisthatitsdonatedcomputersareoldorbroken.AfterreinstallingWindows7andprovidingabasiccomputertrainingsessionforteachers,theincorporationoftechnologyisnowmorefeasible.Furthermore,webroughttheMathruSchool’scomputerlabmanagertotheMathruCentertoworkonandassesstheconditionofthecomputerlabwithus.WeareconfidentthathecanfurthersupportthetechnologyneedsoftheMathruCenteronanas-neededbasis,thoughadditionalworkneedstoconsiderthefactthatnostaffmemberisfullydedicatedtotechnicalsupportattheMathruCenter. Whilewehopethatthetechnologyweintroducedthissummercontinuestodevelopcomputerliteracy,theMathruCenteralsostruggledwithhavingarobustorganizationalstructure.Wehavedoneourbesttoplantrootsforsustainabilitywithourtechnologyprojects,
1http://www.techbridgeworld.org/istep/downloads.html
29
werealizethattruesustainabilitydependsheavilyontheorganizationandteacherscontinuingtotrainothersandreceivetrainingonthesetools. Wetrulyencouragefuturedevelopmentandhopethatfutureteamsimproveuponwhatwewereabletodoinsuchabriefamountoftime.WewantfuturedeveloperstorememberthatICTDworkisnotaboutproductbutalsoaboutprocess.ThereisnodoubtabouttheaptitudeandgreatskillofdeveloperswhochosetouptakeICTs.Withouttechnicalsolutionstrulyunderstandingandbeinginformeddirectlybythecommunitytheytarget,theywilloftentimesmissthemarkofefficacyandrelevance.Butbyworkinghardtofirstdevelopandcultivatedeeperlevelsofcompassion,communication,andrelationship,thenICTDworkbecomesmoremeaningfulandmoreeffective.ThisisthehopeofICTD,thehopeofourwork,andthecallwegivetofuturedevelopers,designers,andresearchersinthismostrewarding,meaningful,andimpactfulwork.
30
REFERENCES[1] M.B.Dias,G.A.Mills-Tettey,andJ.Mertz,“TheTechBridgeWorldInitiative:Broadening
PerspectivesinComputingTechnologyEducationandResearch,”CarnegieMellonUniversity,Baltimore,MD,1-59593-301-8/06/01,Jun.2005.
[2] “iSTEP2009Tanzania.”CarnegieMellonUniversity,Summer-2009.[3] “iSTEP2010Bangladesh.”CarnegieMellonUniversity,Summer-2010.[4] “iSTEP2011Uraguay.”CarnegieMellonUniversity,Summer-2011.[5] “iSTEP2012Ghana.”CarnegieMellonUniversity,Summer-2012.[6] “iSTEP2013India.”CarnegieMellonUniversity,Summer-2013.[7] L.Dodson,S.R.Sterling,andJ.K.Bennett,“ConsideringFailure:EightYearsofITID
Research,”ProceedingsoftheFifthInternationalConferenceonInformationand
CommunicationTechnologiesandDevelopment,vol.9,no.2,pp.19–34,Mar.2012.[8] Rajaraman,“HistoryofComputinginIndia,”UniversityofMinnesota,Minneapolis,MN,
2010.[9] “TheStateoftheWorld’sChildren,”UnitedNationsChildren’sFundUNICEF,NewYork,NY,
Nov.2014.[10] M.R.Ho,T.N.Smyth,M.Kam,andA.Dearden,“Human-ComputerInteractionfor
Development:ThePast,Present,andFuture,”USCAnnenbergSchoolforCommunication&Journalism,LosAngeles,InformationTechnologiesandInternationalDevelopmentVolume5,Number4,Winter2009.
[11] N.Kumar,“Mobiles,Media,andtheAgencyofIndianYouth,”InformationManegementandSystems,UniversityofCalifornia,Berkeley,2013.
[12] M.Kam,D.Ramachandran,A.Raghavan,J.Chiu,U.Sahni,andJ.Canny,“PracticalConsideationsforParticipatoryDesignwtihRuralSchoolChildreninUnderdevelopedRegions:EarlyReflectionsfromtheField,”ACM,Tampere,Finland,1-59593-316-6/06/07,2006.
[13] C.Allen,“GlobalSurveyReportWFDRegionalSecratariatforAsiaandthePacific,”WorldFederationoftheDeafandSwedishNationalAssociationoftheDeaf,Helsinki,Finland,May2008.
[14] WorldHealthOrganization,“WHOglobalestimatesonprevalenceofhearingloss,”WorldHealthOrganization,2012.
[15] G.MathurandD.Napoli,Deafaroundtheworld.Oxford:OxfordUniversityPress,2011.[16] CenterforInternationalRehabilitationandKrishnanNehra,“InternationalDisability
RightsCopendium2003,”CenterforInternationalRehabilitation,Chicago,USA,2003.[17] TechBridgeWorld,“iSTEP2013:ExploringtheFeasibilityandSustainabiliyofAsstive
TechnologyattheMathruCenterfortheDifferently-Abled.”Aug-2013.[18] MediaCentre,“DeafnessandHearingLoss.”WorldHealthOrganization,Mar-2015.[19] NAD,“AuslanPolicy.”NationalAssociationoftheDeaf,25-Apr-1997.
31
[20] “StatesThatRecognizeAmericanSignLanguageasaForeignLanguage,”LaurentClercNationalDeafEducationCenter,GallaudetUniversity,Washington,DC,2004.
[21] T.Skutnabb-Kangas,“TSKLinguisticHumanRights.”NationalAssociationoftheDeaf.[22] S.Wilcox,“ASLasaForeignLanguageFactSheet.”UniversityofNewMexico,1991.[23] “WorldFederationoftheDeaf,2015.[Online].Available:https://wfdeaf.org/.
[Accessed:05-Aug-2015].[24] Ethnologue,'Summarybylanguagefamily',2015.[Online].Available:
https://www.ethnologue.com/statistics/family.[Accessed:04-Aug-2015].[25] Sciencedirect.com,'Estimatingsize,scope,andmembershipofthespeech/sign
communitiesofundocumentedindigenous/villagesignlanguages:TheBanKhorcasestudy',2015.[Online].Available:http://www.sciencedirect.com/science/article/pii/S0271530909000202.[Accessed:04-Aug-2015].
[26] Peacecorps.gov,'KenyanSignLanguage',2015.[Online].Available:http://www.peacecorps.gov/ksl/.[Accessed:04-Aug-2015].
[27] M.Bubu,'MzeeBubu-DeafPrideofKenya',Deafpride.blogspot.com,2010.[Online].Available:http://deafpride.blogspot.com.[Accessed:05-Aug-2015].
[28] AzimPremjiFoundation,“PerspectiveonContentDevelopment.”AzimPremjiFoundation,2011.
[29] E.Pintar,SignLanguageThroughPopMusic.Pittsburgh,PA,2014.[30] Aslpro.com,2015.[Online].Available:http://www.aslpro.com.[Accessed:04-Aug-
2015].[31] Handspeak.com,'AboutHandSpeak',2015.[Online].Available:
http://www.handspeak.com/info/index.php?info=about.[Accessed:04-Aug-2015].[32] SigningSavvy,'AboutSigningSavvy|ASLSignLanguageVideoDictionary',2015.
[Online].Available:https://www.signingsavvy.com/about.[Accessed:04-Aug-2015].
[33] SignWiki,'Um',2015.[Online].Available:http://is.signwiki.org/index.php/SignWiki:Um.[Accessed:04-Aug-2015].
[34] Wikisign.org,'WikiSign',2015.[Online].Available:http://www.wikisign.org/.[Accessed:04-Aug-2015].
[35] Signwriting.org,'SignWritingHistory',2015.[Online].Available:http://www.signwriting.org/library/history/.[Accessed:05-Aug-2015].
[36] Signbank.org,'SignPuddleOnline',2015.[Online].Available:http://www.signbank.org/signpuddle/.[Accessed:04-Aug-2015].
[37] Motionsavvy.com,'MotionSavvy',2015.[Online].Available:http://www.motionsavvy.com.[Accessed:04-Aug-2015].
[38] E.Hirshorn,“VisualAttentionandDeafness(ResearchBriefNo.3),”VisualLanguageandVisualLearningScienceofLearningCenter,Washington,DC.
[39] Quizlet,“OurMission.”QuizletInc.,2015.[40] Kitzel,“FreePrintableFlashCardMaker.”05-Nov-2007.
32
[41] “AboutCram.com.”StudyMode,2015.[42] B.Vacha,R.Ghate,M.Banerji,B.NanavatiandM.Gokhale,TalkingHands:Sign
LanguageBook.NASEOHRDCPublications,1980.[43] E.Orfanidou,B.WollandG.Morgan,Researchmethodsinsignlanguagestudies.John
Wiley&Sons,2015.[44] O.Sacks,SeeingVoices.Vintage,1989.[45] C.RowlandandOregonHealth&ScienceUniversity,“AssessingCommunicationand
LearninginYoungChildrenWhoareDeafblindofWhohaveMultipleDisabilities,”U.S.DeparptmentofSpecialEducation,Washington,DC,Health&ScienceH324D030001,2009.
[46] M.V.Corporation,“EntertainingGamesandDisplaysforSpeechTherapy.”08-Jun-2015.[47] “CMUSphinx:PhonemeRecognition.”CarnegieMellonUniversity,06-Aug-2015.[48] S.Fernández,A.Graves,andJ.Schmidhuber,“PhonemeRecognitioninTIMITwith
BLSTM-CTC.”TUMunich,22-Apr-2008.
33
APPENDIX
OURFINDINGSATTHEMATHRUCENTERCLASSROOMFORSTUDENTSWITHMULTI-SENSORYIMPAIRMENTS(MSI) ThebrevityofourfieldexperiencekepttheMSIclassroomoutsidethescopeofourprojects.WhilewewerenotabletodevelopnewtechnologiesspecificallyfortheMSIclassroom,wedidobservetheclassmultipletimesandhaddiscussionswiththeMSIteachersaboutthemostinhibitiveproblemsintheirwork.ThemostdifficultcircumstanceintheMSIcurriculumisthatitisuniqueforeachstudent;wecouldnotrelyonthesameacademicstructurethataffordedtheadoptionofourtoolsinagivenfixedcontext.SowiththecurriculaofMSIstudentsbeingbuiltindividually,theMSIteachersuggestedusinga“creativepractice” throughwhichatoolcouldworkformultiplestudents.ThesuggestionsforfutureworkastoldbytheMSIteacherincludehighcontrastimaging,tactilestorybooks,andsensorytools.Theseprojectideasare“thingsthatwouldhelpmanystudents” becausetheyexploredifferentsensoryexercisesaswellasprovidecreativityandenjoyment.
AdifficultpartoftheMathruCentersystemisthevaryinglevelsoffamilyandparentengagement.MoststudentsattheMathruCenterareresidential,howeverMSIinparticularalsohashome-basedanddaystudents.WhiletechnologyisnotviableinthehomesofmostMathruCenterdaystudents,toolsthatencourageengagementwithlocalsurroundingsandcommunitiescouldcreatepotentialwaysforparentsandfamiliestoengagewiththeMathruCentereducation.
Recommended