iSTEP 2015: Cross-Cultural Technology Development Toward Language Access for the Deaf and

Hard of Hearing

Maya Lassiter, Amal Nanavati, Erik Pintar, Minnar Xie, Ermine Teves, M. Bernardine Dias

CMU-RI-TR-16-32

The Robotics Institute Carnegie Mellon University

Pittsburgh, Pennsylvania 15213

June 2016

Copyright © 2016 TechBridgeWorld at Carnegie Mellon University

1

iSTEP2015:Cross-CulturalTechnologyDevelopmentTowardLanguageAccessfor

theDeafandHardofHearingMayaLassiter,AmalNanavati,ErikPintar,MinnarXie,ErmineTeves,M.BernardineDias

ACKNOWLEDGEMENTSPartnerinvolvementhasbeenessentialforthiswork,andwewerefortunatetocollaboratewiththeMathruEducationalTrustfortheBlindandtheirCenterfortheDeafandDifferently-Abledinneedsassessmentandusertesting.TheauthorsespeciallythankMs.GubbiR.MukthaandMathruCenterstaffmembersformakingourprojectspossible.Furthermore,manyindividualsparticipatedinourneedsassessment,generouslyvolunteeringtheirtime.Becauseofresearchcomplianceprocedures,wecannotnametheseparticipantsindividually,butwearegratefulfortheirinvolvementandenthusiasmfortheproject.WealsothankBernardineDiasandErmineTevesfromtheTechBridgeWorldresearchgroupfortheirguidancethroughouttheinternship.WewouldalsoliketothankoursponsorswhomadetheiSTEP2015internshippossible,includingindividualdonorsand100@100donors.

ABSTRACTThispapersummarizestheworkdoneoverthecourseofnineweeksattheMathruCenterfortheDeafandDifferently-Abled(MathruCenter)inBangalore,Indiaandaimstoproviderecommendationsregardingtechnologysolutionsfordeafandhardofhearingstudentsandtheirteachersinunderservedcommunities.ThisworkwascarriedoutbyateamoffourundergraduateresearchstudentsparticipatinginaninternshipprogramcallediSTEP–shortforinnovativeStudentTechnologyExPerience(iSTEP)–organizedbytheTechBridgeWorldresearchgroupatCarnegieMellonUniversity.Fromtheteam’sworkindesigninganddevelopingeducationaltoolswithstudentandteacherparticipants,thispaperexploresthepracticeofcreatingsharedownershipovertechnologicalideasthroughdirectrelationshipwiththetargetpopulationandviewingthemasequalsintheprocess.Theteam’spartnershipandworkwiththeMathruCenterresultedintwonewlycreatededucationaltoolsaimedatprovidinglanguageaccessforDeafprimaryschoolstudentsandtheirteachers.Thefirst,SignBook,providesgreatersignlanguageaccessthroughasignlanguagedictionarycreationtoolwithabilitiesforcustomvideoandpicturecapturingandcategorizationofentriesandwordsintotopics.Thesecond,SpeakUp!,motivatesstudentstostriveforgreaterverballanguageaccessthroughasuiteofvoice-poweredgamesaimedtofamiliarizepre-andpartially-verbaluserswiththepoweroftheirvoice.

2

TABLEOFCONTENTSAcknowledgements..............................................................................................................................1

Abstract....................................................................................................................................................1

TableofFigures.....................................................................................................................................3

ListofTables..........................................................................................................................................3

Introduction...........................................................................................................................................4TechBridgeWorld............................................................................................................................................4iSTEP....................................................................................................................................................................4TheMathruEducationalTrustfortheBlind..........................................................................................5InformationCommunicationTechnologyforDevelopmentinIndia............................................5CultureandCommunity................................................................................................................................6iSTEP2013Findings.......................................................................................................................................8Motivations........................................................................................................................................................9

LanguageAccessTools.....................................................................................................................10StudySetting...................................................................................................................................................10StudyParticipants.........................................................................................................................................11FindingsandDevelopment........................................................................................................................11SignBook:ASignLanguageDictionary................................................................................................................12SpeakUp!VoicePoweredGameSuite.................................................................................................................20

Discussion............................................................................................................................................26

AdditionalConsiderationsforFutureWork............................................................................27

References...........................................................................................................................................30

Appendix..............................................................................................................................................33OurFindingsattheMathruCenterclassroomforStudentswithMulti-SensoryImpairments(MSI).......................................................................................................................................33

3

TABLEOFFIGURESFIGURE1.THEMATHRUCENTERFORTHEDEAFANDDIFFERENTLYABLED...........................................................................................5FIGURE2.FLASHCARDSUSEDATTHESCHOOL................................................................................................................................................8FIGURE3.FIRSTGRADESPEECHCLASS.........................................................................................................................................................11FIGURE4.ASAMPLINGOFAMERICANSIGNLANGUAGEWORDSWRITTENWITHSIGNWRITING........................................................14FIGURE5.“MYNATIVETONGUE–1”THEONLYOFFICIALSIGNLANGUAGEREFERENCEBOOKATTHEMATHRUCENTER...........16FIGURE6.ANINTRICATEDRAWINGBYATEACHER,USEDTOEXPLAINACLASSROOMLESSONINTHEFIRSTGRADECLASSROOM.17FIGURE7.THESIGNBOOKINTERFACE...........................................................................................................................................................18FIGURE8.THESPEAKUP!MAINMENU.........................................................................................................................................................24FIGURE9.AMATHRUCENTERSTUDENTUSINGTHESPEAKUP!VOICEPOWEREDGAMESUITE..........................................................24FIGURE10.ATEACHERUSINGSPEAKUP!COMMUNALLYWITHHERENTIRECLASS..............................................................................25FIGURE11.MATHRUCENTERTEACHERSCREATINGASIGNINSIGNBOOK.............................................................................................26FIGURE12.TECHNOLOGYISOFTENTREATEDASANOVELTY,SINCESTUDENTSDONOTENCOUNTERTECHNOLOGYINEVERYDAY

SETTINGS..................................................................................................................................................................................................27

LISTOFTABLESTABLE1.SIGNBOOKTABLEOFFEATURES...................................................................................................................................................19

4

INTRODUCTION

TECHBRIDGEWORLDTechBridgeWorld(TBW)isaresearchgroupbasedintheRoboticsInstituteatCarnegie

MellonUniversity(CMU).Thegroupformsandbuildsuponpartnershipsinunderservedcommunitiesinternationallytocreateculturallyappropriatetechnologythatadherestoeachcommunity’svisionofprogress.Foundedin2004,TBWleadstheinnovationandimplementationofsolutionsthathelpaddresschallengesincommunitiesnotordinarilyservedbytechnologydevelopmentefforts[[1]].Thisworkofteninvolvestheinventionofnewtoolsandcustomizationofexistingtechnologyaswellaseffortstoinspireandtrainfutureresearchersandtechnologistsinthiskindofwork.TechBridgeWorldemploystheknowledgeandimaginationoffaculty,staff,andstudentsinallofitsprojects.TechBridgeWorldaimstocreatetechnologysolutionsaccessibleandrelevanttoalltoimprovepeople’slives,enhanceandshareadvancesintechnologyacrosscultures,andpromotesustainabledevelopmentaroundtheglobe.

ISTEPTBWlaunchedtheInnovativeStudentTechnologyExPerience(iSTEP)internship

programinthesummerof2009toprovideCMUstudentswithreal-worldexperienceinapplyingtheirknowledgeandskillstowardservingmarginalizedorunderservedcommunities.Throughaunique10-weeksummerresearchinternship,studentsconducttechnologyresearchprojectsalongsidepartnercommunitiesacrosstheworld.

iSTEPoffersselectedstudentstheopportunitytoengineerwithcompassionthroughfirst-handexperience.OvertheyearsiSTEPhasbroughttogetherinternteamsofundergraduateandgraduatestudentsfromCMU’sPittsburghandDohacampuseswithdiverseexperiencesanddisciplinarybackgrounds.TogetherwithTBW,eachiSTEPteamcollaborateswithlocalpartnerstobetterunderstandtheneedsofthecommunityanddesigntechnologicalsolutionsthatcouldhelpaddresssomeofthechallengesthatthecommunityfaces.AspartofTBW’sphilosophy,allsolutionsarebuiltwiththecommunityanddevelopedthroughtrust.

PastiSTEPlocationsincludedTanzaniain2009,Bangladeshin2010,Uruguayin2011,Ghanain2012,andIndiain2013withprojectsinassistivetechnology,literacytools,informationexchange,andenvironmentalsustainability[[2],[3],[4],[5],[6]].

ThemultidisciplinaryiSTEP2015teamwascomprisedoffourundergraduatestudentsfromvariousschoolsatCMUincludingtheCarnegieInstituteofTechnology,CollegeofFineArts,DietrichCollegeofHumanitiesandSocialSciences,SchoolofComputerScience,andtheBXAIntercollegeProgram.iSTEP2015internshavebuiltupontheiSTEP2013team’sworkbydevelopingnewassistivetechnologyprojectsincollaborationwithTBW’spartner,theMathruEducationalTrustfortheBlind.

5

THEMATHRUEDUCATIONALTRUSTFORTHEBLINDWhilenowworkingwith

numerousglobalpartners,TBW’slongeststandingpartnershiphasbeenwiththeMathruEducationalTrustfortheBlind(MathruTrust).TheMathruTrusthousesboththeMathruSchoolfortheBlind(MathruSchool)andTheMathruCenterfortheDifferentlyAbled(MathruCenter).Ms.GubbiR.Mukthafoundedbothschoolsafterherowndebilitatinginjury.TBWhasbeenpartneringwiththeMathruTrustsince2006tobuildassistivetechnologybasedonobservedandcommunicatedneedswithcontinualcommunityinputandfeedback.Additionally,TBWhasbeenpartneringwiththenewMathruCentersinceitsinceptionin2012.In2013,aniSTEPteamcompletedaNeedsAssessmentreportandsincethen,anongoingpartnershipwiththeMathruCenterhascontinued.

TheMathruCenterwasinspiredafterillnesscausedablindstudentattheMathruSchooltosuffersignificanthearingloss,thusbeingunabletocontinueonattheschoolwithoutmorespecializedaccommodationsandsupport.Today,theMathruCenterisaresidentialschoolofferingfreeeducationforstudentswithhearingimpairmentsormulti-sensorydisabilities.ThegoaloftheMathruCenteristorecognizethecapabilityandpowerofallindividuals,regardlessofthelimitationsimposedbysocietyandenvironmentregardingphysicalability.TheMathruCentercurrentlyhas47studentsofwhich31aredeafandhardofhearingstudentsand16havemulti-sensoryimpairments.WhilethisyeartheCenterservesstudentsfromnurseryto4thstandard,additionalstandardswillbeaddedannuallyasstudentsgrowintothenextgrade.ThecurriculumfortheDeafstudentsattheMathruCenterfollowsthestatesyllabusaswellasteachingtheadditionalcurricularneedsofspeechandsignlanguage.

INFORMATIONCOMMUNICATIONTECHNOLOGYFORDEVELOPMENTININDIAThoughInformationandCommunicationTechnologiesforDevelopment(ICTD)isa

youngdiscipline,Indiahasbeenanepicenterofresearchandfieldstudiesinthisarea[[7]].ThesurgeofInformationandCommunicationTechnologies(ICTs)inIndiahasbeengrowingforseveraldecades,startingfromIndiabeingthefirstcountrywithacomputerintheglobalSouth

Figure1.TheMathruCenterfortheDeafandDifferentlyAbled

6

in1956[[8]].Thishascreatedaspaceforincredibleempowermentaswellasimposedsolutions.

TechnologicaldevelopmentthatcaterstousersintheGlobalNorthoftenmakesassumptionsthatarenottrueofdevelopingcountries.Tonameafew:reliablepower,Internetaccess,previousexperiencewithtechnology,andliteracy.InIndia,acountrywithonly12Internetusersper100[[9]]personsandunequalliteracyratesacrossstatesanddemographics,thesecommonassumptionsbydevelopersdenylargeIndianpopulationsfromtechnologicalsupport[[10],[11],[12]].

Forfurthermarginalizedgroups,suchasthedisabled,accesstoandexperiencewithtechnologyislikelyevenmorelimited[[13]].Thus,additionalprovisionsweretakentoensureasolidunderstandingoftheuniqueneedsofourpartner,theMathruCenter.Likewise,itwascriticaltoemployparticipatoryandempoweringstancesinboththecreationandevaluationofourwork.

CULTUREANDCOMMUNITYCurrently,360millionpeopleworldwidehavedisablinghearingloss,whichtheWorld

HealthOrganizationdefinesas“hearinglossgreaterthan40dBinthebetterhearingearinadults(15yearsorolder)andgreaterthan30dBinthebetterhearingearinchildren”[[14]].Individualswithdisablinghearinglosshavetroublehearingeverydayconversationorperceivingsoundinevenquietenvironments.Asitstands,only10%oftheglobalneedforhearingaidsismet,90%oftheworld’sdeafpopulationhasneverbeentoschool,andonly5%canreadorwrite.Theprevalenceofhearinglossisunequallydistributedacrosstheworldwith80percentoftheworld’sdeafpopulationlivingindevelopingcountries[[15]].Additionally,SouthAsiahasthehighestratesofdeafandhardofhearingindividualswhilethecurrentproductionofhearingaidsmeetslessthan3%ofdevelopingcountries’need[[1]].Inturn,SouthAsianDeafcommunitiesfaceincreasedchallengesandhardships.

UnderIndia’sPersonswithDisabilitiesActof1995,theintegrationofdisabledcommunitiesintomainstreamsocietywasestablishedasagovernmentpriority[[16]].However,officialdocumentationofsignlanguagehasbeendisregardedandunsupported,thuskeepingIndia’sDeafcommunityfrombeingwidelyunderstoodandrecognized.

OurworkwiththedeafandhardofhearingcommunityatMathruCenteraddressedtwocomplexities.First,whiletheMathruCenterhopestoincreaseaccesstosignlanguage,Indiansocietyasawholeneglectssignlanguageandthusprioritizesverballanguageasanecessarypartofdeafpeoples’societalinclusion.Therefore,workinthisareamustacknowledgethepresentexpectationofverbalcommunicationasapathtoself-advocacyfortheDeaf.AlthoughIndia’sDeafcultureisgrowing,theyarestilldecadesawayfromhavingthesamesupportandrecognitionseenbyAmericanDeafculture-somuchsothatitwouldbe

7

disadvantagingMathruCenterstudentstonotenableverballanguageaccessalongsidesignlanguageeducation[[17]].

AmajordifficultythatissomewhatuniquetoIndia’sgrowingDeafcommunityisthevastmultitudeofexistingverballanguages.ThissituationcanleadtheDeaftofeelsocietalexclusionacrossallofIndia’sdiverselanguage-scapeinboththehearingandDeafcommunities.Thus,deafindividualsinIndiaexperiencetheuniquechallengeofhavingtobridgeamultitudeofsignedlanguagesontopofamultitudeofverballanguages.InIndia,deafnessoftenassumesdumbness–thecognitiveinabilitytocommunicate.This,alongwithIndia’sculturaldenunciationofsignlanguages,barsIndia’sDeafcommunitiesfromsupportinbothsignedandspokenlanguages.Despiteanincreasinglyglobalworld,Deafcommunitiesremaindisconnectedduetoalackofunderstanding,lackofinterpreters,andlackofsocietalrespectforsignlanguages[[18]].

Themain“justification”forthelinkagebetweendeafanddumbisthedelayedspeechandcommunicativedevelopmentofdeafchildren.Howeverthisdelayisinspokenlanguage,anunnecessarymetricifprovidedaccesstosignlanguage[[19]].Speechtherapyremainsprohibitivelyexpensiveandtimeintensive.BecausetheMathruCenteralsolackstheresourcestofullyinvestinindividualizeddailyspeechtherapy,progressisslowandunfruitfulintheshortspanoftheschoolyear.AccesstospokenlanguageremainsdifficulttoincorporateandutilizeineducationfordeafIndians,yettheywouldbefullyabletoparticipateinsocietyifgivensupportandencouragement[[16]].

EffortsliketheAzimPremjiFoundation(APF)inBangalore,IndiaandtheDigitalInformationResearchFoundation(DIRF)inChennai,IndiahavesetupthousandsofcentersacrossIndiatogiveaccesstosharedcomputerswithpriorityforthedeafanddisabled.However,APFcourseworkuseslargelyaudio-basedlocallanguageinstructionandDIRFaimstogiveopportunityfortechnicalliteracyforthedeaf.

VeryfewofourrelatedsourcesfocusontheconsiderationsofearlyDeafeducationoriftheydo,assumeahigherthanpresentfamilialsupportnetwork[[20]].Weaimedtocreateassistivetoolsthatwouldstandaloneinfunctionbutnotreplacethepoweroftheteacher.Existingprojectsconsideredaccessibilityinonesense,butnoneaddressedthekeyconstraintsattheMathruCenter:lowtechnicalliteracyandstudentswithlittlemeansforcommunication.

OurworkidentifiedandaddressedtwomainhurdlesthatexistformarginalizeddeafIndiansandtheiraccesstolanguageandcommunication:firstly,formallanguageexpressionandformationthroughsignlanguage,andsecondly,awarenessofvoiceandtheeffectitmayhave.Theformerrequiresmoretrainingandsupportbyteachers,whoneedtoteachandencouragesignlanguageacquisitiontostudents[[21]].Thelatterisanecessaryprerequisiteforverballanguageformationbutistoooftencoupledexclusivelywithspeechtherapy.Ourworkfollowstheideathatawarenessofvoiceandthepowerithascanbeadditionallysupportiveinformallanguageformationandlearnedwithouttheneedofatrainedspeechtherapist.

8

ThegoaloftheTechBridgeWorld-MathrupartnershipandtheworkofthisteamhasbeentocreateassistivetechnologiesthatserveandempowertheMathruCentercommunity.BasedonapriorneedsassessmentattheMathruCenter(furtherdetailedlaterinthisreport)thechosenfocusofourworkistoexplorelanguageaccessattheMathruCenterviabothsignedandverbalmeans.Whilecreatingagencythroughourcustomsignlanguagerepository,wealsoacknowledgetheneedoftheMathrucommunitytousespeechtherapyforinstructioninimmediateskillstoassiststudents’accesstothebroaderIndiansociety.TheseprojectsbegintotacklethegroundworkoftechnologytoassistthecommunicationofthedeafandhardofhearinginIndia:bothinanincreasedrecognitionandaccesstolocalsignlanguagesandinthetrainingofverballanguagethroughspeechandone’svoice.

ISTEP2013FINDINGSThe2013iSTEPteamperformedaneedsassessmentattheMathruCentertoidentify

keyareasofdifficultyorcomplexityinteachingtothedifferentlyabledstudentsattheCenter.Thisincludedatwo-foldconsiderationofteachingmethodologyandstudentneeds.Throughobservations,shadowing,affinitymatching,andinterviews,the2013teamidentifiedkeyproblemareasinhibitingteacherefficacy[[17]].

Thefindingsthroughthisneedsassessmentrecognizedthatthemostprevalentteachingtool,flashcards,hadbothtemporalandphysicalrestraints.Temporally,flashcardstookalongtimetosortbysubjectareaanddifficulty,leavingteacherswithlesstimetoteach.Physically,flashcardsarealimitedmediumrequiringteacherstopresentcardstoanentiregroupwithvaryingwaysoflearning,makingitchallengingformaterialtobecustomizedtoeachstudent’sneeds.Teachersprovidealotofindividualattentionbecausestudentsareatvaryingskilllevels,butthatprovedtobechallengingwithexistingtoolsandresources.Interestingly,thereportnotedthatteachersfeltthatstudentslearnedfasterfromeachother.

Anothersignificantlimitationidentifiedwasteachertraining.BecausetheMathruCenterisinitsinfancyandIndiaasawholelacksteacherstrainedineducationfordeafnessandmultiplesensoryimpairments,theMathruCenteratthetimehadonlyonetrainedteacherduringthesummerof2013.Thus,theyrelyinpartonteacherswithoutexperienceortraininginspecialeducationtoleadclassrooms,whoselackofbackgroundexperienceandhighturnoverleadstomanychallengesrelatingtotrainingandclassroomefficacy.

Figure2.Flashcardsusedattheschool

9

The2013iSTEPteamsuggestedafewproblemspacesinwhichanewtechnologysolutioncouldbeuseful.ThemainsuggestionsforteachersincludedateachertrainingreferenceforteacherstodeveloptheirskillsandteachingstylefrompublishedonlineresourcesandcompilednotesfromtrainedMathruCenterstaff.Thisincludedasignlanguagetrainingtoolforuntrainedteachers,butthiswasnotedasbeingdifficultasthesignlanguageusedattheMathruCenterisadialectofIndianSignLanguagethathasverylittleexistingdocumentation.

Anotherrecommendationforfutureworkwasaflashcardgeneratorandlessonplanningtechnology.Thesedirectlyrespondtoteachers’observeddifficultiesinthetimespentpreparinglessonplans.Suchatool’sdatabaseofwordswouldneedtoconsiderawiderangeofwordsandhighqualityimagesthatarealsoculturallyrelevantandavailable.ItwouldalsoneedtorecognizethattheMathruCentercurrentlydoesnothaveInternetaccess,andthuswouldneedtoworklocallyonthecomputersattheMathruCenter.

Thereportalsonotedthatteachingspeechandwordpronunciationwastime-intensiveanddifficultforteachers,thusleadingtoasuggestionofspeechpracticesoftware.Thissoftwarecouldhelpassessandgivefeedbacktostudentsaboutthecorrectnessofsoundstheyaremaking.

OthernotedtechnologysolutionsuggestionsincludedComputerTraining,Sign/AlphabetTranslating,andaMulti-SensoryClassroomAssistanttosupportthewiderangeofstudentsattendingtheMathruCenter.

MOTIVATIONSTheiSTEP2015researchteamaimedtoaddresstheforemostchallengesaffecting

teachersandstudentsattheMathruCenterwithnewlydevelopedandcustomizedtechnologysolutions.AccordingtotheiSTEP2013team’sneedsassessmentattheMathruCenter,oneofthebiggestchallengesattheMathruCenterwasteacherpreparationandtraining.Inadditiontovalidatingthisneedthroughconversationswithheadstaffandteachers,the2015teamalsoassessedadditionalneedsoftheMathruCenterwheretechnologymayplayarole.Asaresult,wehaveiterateddesignsanddeployedearlyversionsofeducationaltechnologiesthataddresssignlanguageaccessandspeechtherapyattheMathruCenter.

ThisreportinformstheworkoffutureteamsandtheTBWresearchgrouponassistiveeducationaltechnologiesinitiatedduringthe9-weekfieldresearchinternshipattheMathruCenter.Secondarily,thisreportcanservetoeducateonlookersoftheongoingpartnershipbetweenTBWandtheMathruTrust,asthisreportdemonstratestheparticularanduniqueconsiderationstakenwhendevelopingforthedeafandhardofhearingcommunity.Lastly,thisreporthopestoexpresstheimportanceofownershipandempowermentwhenworkingwithdevelopingcommunities.Inparticular,thisprojectcontributestothegrowingbodyofwork

10

aimedtoholisticallydeveloptechnologythatempowersandfreesratherthanimposesandassumes.

Theremainderofthisreportwillprovideanin-depthlookattheiSTEP2015projects.Wewilldiscussmethodology,studystructure,datacollectiontactics,unforeseenconsiderations,andrelatedworksindevelopingoursolutions.Wewilldescribethesolutionsdevelopedandthenconcludeourreportwithadiscussionaboutourreflectionsandourconsiderationsforfuturework.

LANGUAGEACCESSTOOLSThe2015teamarrivedattheMathruCenterforthestartofthe2015-16schoolyear.It

wasimportantforustovalidatetheneedsuncoveredbythe2013team,particularlygiventhattwoyearshadpassedandtheMathruCenterisayounginstitution.Inourfirstthreeweeksintothesummer,twooutofthethreeexistingteachersforthedeafandhardofhearingquitandallthenewhireswerenottrainedineducatingdeafandhardofhearingstudents.OnereasonforthehighturnoverofteachersattheMathruCentercouldbetiedtoIndianculture:manyteachersareyoung,unmarriedwomen,whooftenarepressuredtoleaveteachingwhenexpectationsofmarriagearise.Ontopofthesechallenges,werealizedthatthefindingsofiSTEP2013assumedahigher-than-presentleveloftechnologicalfeasibilityintheclassroomsoftheMathruCenter.Hence,ourprojectsneededtoconsideranothergroundupneedsassessmenttoidentifyfeasibletechnologysolutions,ifrelevant.

WediscusseddifficultiesexperiencedattheMathruCenterwithbothexitingandnewteachers,aswellasadministrators.Teacherswereconcernedwiththeshynessofexpressionandcommunicationoftheirstudents.Studentswouldnotbeabletoexpress,throughsign,verbally,orotherwise,whentheywereupsetorconfused.Thisisprobablylinkedtothevaryingdynamicoftheteachingstaffandinconsistencieswithin-classbehavioralrules.Further,aspointedoutinthe2013report,studentshavedifficultiescommunicatingwitheachotherduetodifferencesinthesignlanguageusedattheschool.Administrators,ontheotherhand,wereconcernedwiththelackofteachertrainingtoolsandstudentdifficultywithverbalcommunication.Thelackofteachertrainingtoolsresultedinalargelearningcurvefornewteachers,andalesseffectiveuseofclasstime.Students’difficultiesinspeechwereaconcernfortheadministratorsbecause,outsideoftheMathruCenter,studentswouldneedtorelyonverbalmeansasthemainmethodofcommunication.

Weusedtheseexperiencestodirectourprojectscreatedinthefield.OurprocessconsistedofspendingtwodaysaweekattheMathruCenterforobservations,interviewsandusertesting.Ourfieldnotescompriseofhandwrittennotes,photos,andvideos.

STUDYSETTING

11

TheMathruCenterisaresidentialschoolcomplexthathasdevelopedthefollowingservices:

• FreeHostelforDifferentlyAbledChildren• CenterforDeafblindandMulti-SensoryImpaired(MSI)Children• VocationalTrainingCenterforDisabledandWomen• FreeSchoolforMSIChildren• FreeSchoolforDeafandHardofHearingChildren(4classrooms)

Ourobservationsandinterviewswereheldafterdailyclassesended.OurregularpointofcontactwasthemostseniorteacherattheMathruCenter,whogavedirectfeedbackonourideasandshapedourtoolsintovaluesensitivetechnologyapplicabletotheclassroom.Weconsultedstudentsandnewteachersmultipletimesduringourdevelopmentphase,keepingtheMathruCentercommunityincludedinallpartsofcreatingourlanguageaccesstools. Afterdevelopingourlanguageaccesstools,usertestingin-andoutsideoftheclassroomwasconductedusinglaptops

donatedbyTBWtotheMathruCenter.Laptopswerepurchasedbecauseteacherspreferredusinglaptopsoverusingtheexistingdesktopcomputersattheschoolandasetoftwotabletsdonatedtotheschool.

STUDYPARTICIPANTS WeobtainedfullparticipationfromallteachersintheCenteraccordingtotheCarnegieMellonUniversityInternalReviewBoard(IRB)protocolfornon-invasiveworkwithhumansubjects–includingtheteacherswhoresignedduringourtimeattheCenter.Weobtainedconsentfromallstudentsinthedeafandhardofhearingclasses,firstthroughfourthstandard,forparticipationinthestudy.Atleasttwoteammemberswereinvolvedineveryinformedconsentprocess.OurinterviewswithteachersandstudentswereconductedinEnglish,sometimeswitheitherastaffmemberorotherteacherpresenttotranslateintoeitherKannadaorHindi.

FINDINGSANDDEVELOPMENT

Figure3.Firstgradespeechclass

12

Whenconsideringsolutionstotheneedsfoundin2013,weforemostfoundalackofsignlanguagedocumentationusedbytheMathruCentercommunity.Lackofsigndocumentationiswhatpredominatelykeepsnewteachersfromgaininganunderstandingofstudentaptitude-asignificantneedconsideringthewiderangeofabilityattheMathruCenter.Becauseteachersneededtolearnsignsfromstudentsorotherteachers,commonclassroompowerbalanceswerechallenged.ThisprovidesaninterestingexistingsocialstructureforapotentialsignlanguagereferencemadebyMathruteachersandstudentsforotherMathruteachersandstudents.

Wealsosawthatteachersforthedeafandhardofhearingcontinuedtoincludeaspeechclassduringthedaydespitenotbeingabletoaffordprofessionalspeechtherapy.Basicsoundsandeverydayvocabularyarepracticedviaaspeak-and-repeatmethod,pairedwithbringingstudents’handsuptoateacher’sthroattofeelthevibrationsandemphasizinglip-reading.Themainlimitationsofspeechclasswerethatstudentswereshytospeak,wouldnotknowthedifferencebetweenmouthingwordsandspeaking,andtheteachercouldnotprovideextensiveindividualattention.TheseobservationsarenotapplicabletotheMSIclassroom,inwhichthecurriculumincludesindividualizedplanscreatedwithquarterlybenchmarkstowardsanannualgoal.SomeMSIstudentsthenhavemorespeechtrainingifrelevanttotheirpersonalgoal,thoughthe2015teamdidnotobservethosesessions.

AlthoughwehadinitiallyhopedtoutilizetheexistingcomputerlabattheCenter,thecomputerlabwasnotareliableresource.Thecomputerswereunderutilizedandnotmaintained,manyofwhichwerevirus-riddenorlackingfunctionalharddrives.Duetothisandaninterestinbringingtechnologytotheclassroom,theMathruTrustandTBWinvestedintwoLenovolaptopstohouseouriSTEP2015projects.Teacherschosetheselaptopsovertabletsbecausetheywill“givestudentsabetterideaofacomputer.”Wefoundthatstudentsandteacherswerehappytoadaptnewwaysoflearningandteachingbutdidnothavetheconfidencetodemandsuchresourcesthemselves.TobeginworkingwiththecomplexandnewenvironmentoftheMathruCenter,ourdevelopmentwascatalyzedbytheinformationprovidedbytheiSTEP2013teamandpriorknowledgeheldbyiSTEP2015teammembers.

SIGNBOOK:ASIGNLANGUAGEDICTIONARY

Deafcommunitiesaroundtheworldutilizeadiversityofsignlanguages,butonlyuntiltherecentpasthavesignlanguagesbeguntoreceiverecognitionasfullycomplexandstructuredlanguagescapableofrichmeaningandabstractexpression[[15]].Thisisinpartduetomisconceptionsofthedeafasthe“deafanddumb”or“deafmute”andofsignlanguageasalanguageofsimplisticgestures[[22]].Stilltoday,accordingtotheWorldFederationfortheDeaf,only44ofabout120countriesrecognizesignlanguageasvalidlanguagesforuseinschoolsandotherofficialmatters(suchasingovernment)[[23],[15]].

13

TheEthnologue,whichisthemostcomprehensiverecordoftheworldlanguagesthatexist,compiledbyhundredsoflinguistsandresearcherssince1951withover7,000livinglanguageslistedandcategorized,notesamere137documented‘Deafsignlanguages’intheworldwithapproximately10.5millionspeakers[[24]].Comparedtotheestimated360millionwithdisablinghearingloss[[1]],evenknowingthatmanyofthosedonotusesignlanguage,thereseemstobeevidenceofagapofdiscoveredandresearchedsignlanguagesandDeafcommunities.

Asonelinguisticfieldresearchernotesaboutsign(visual)languages:

…manualvisuallanguageswerenotrecognizedaslanguagesatall.Rather,as

Bloomfieldput it,“…elaboratesystemsofgesture,deaf-and-dumb language,

…andsoon,turnout,uponinspection,tobemerelyderivativesoflanguage”

(1933, p. 144). Severe under-documentation of an entire class of natural

human languages is the unfortunate legacy of this historical failure to

recognizeandtoattendtosign languagesas languages.Todate,“Therehas

neverbeenathoroughsurveyoftheworld’ssignlanguages”(Meier,2000,p.

1965), and most of the world’s manual visual languages have yet to be

described.[[25]]

Hegoesontowritethattheunder-documentationandrecognitionofsignlanguagesisespeciallytrueof“village”or“indigenous”signlanguages,whichisthetypeusedattheMathruCenter.Thisunfortunatehistoryofsignlanguagesbeingdisregardedleavesaholeintheworld’srecognitionofnotonlythelanguageoftheDeaf,buttheknowledgeofDeafculturesandtheneedsofthislargelyignoredandundervaluedsegmentoftheglobalpopulation. Ofeffortsmadetoassistthedeafindevelopingcommunities,manydonottakeapeople-centeredapproachinrecognizingthevoiceandthoughtsofthedeaftheyaimtohelp.Forexample,muchphilanthropicworkhasbeendonetoassisttheDeafcommunityinKenya,includingthedevelopmentofasignlanguagedictionaryonCDanddistributedwithinKenya[[26]].However,alocalDeafKenyanspeaksofthisandotherprojectsquitenegatively,saying:“theseorganizationshaveneverappreciatedcommunityentryapproaches,viewtheDeafasobjectsofbenevolenceandnotpartnersindevelopmentandare‘copy/pasting’foreignsolutionstocomplexlocalissues”[[27]]. WiththedevelopmentofSignBook,theiSTEPteam’sapproachvaluedthelocalsignlanguageasavalidlanguageandencourageditsuseforeducationandcommunicationattheMathruCenter.Throughouttheproject,wecloselylistenedtotheneedsandperspectivesoftheteachersofthedeafandhardofhearingandthestudentsfromtheverybeginning.Throughthatwecametoasetoffeaturesthataddressedtheneedsofteachertraining,having

14

asignlanguagereference,andprovidingacustomizable,efficient,andreusableteachingtool.Together,wecreatedSignBook,adigitaltoolforcreatingacommunity-constructedsignlanguagedictionary.WehopethisworkcontributesnotonlytoassisttheneedsoftheMathruCenter,butalsotofurtheracommunity-centeredvisionofICTDwork,documentourstepsincross-culturalcreationofassistivetechnologyforthedeaf,andmorebroadly,promoteawarenessofthelegitimacyofsignlanguageandtheneedsanddesiresofthedeafaroundtheworldsothatmoreandmoreofthedeafpopulationcanbeempoweredtowardsbecomingincreasinglyvaluedmembersofsocietythroughadequateaccesstolanguageandeducation.RelatedWorks ThisprojectisthefirstthatempowersDeafcommunitiestobeabletodocumenttheirownsignlanguagethroughadigitalapplication.AdditionallyremarkableisthatnopriorICTDworkaddressesdeafsignlanguagelearning[[28]].ThoughthisisarelativelyoriginalprojectconceptandcustomizedtotheneedsoftheMathruCenter,severalcomponentsoftheprojectdobuildofftheconceptsandideasofexistingworks.

Theideaofavideo-basedsignlanguagedictionaryhasbeenaroundforover15years.OnlineareseveralsignlanguagedictionarieswiththousandsofwordsandvideosofsignsinAmericanSignLanguage[[29],[30],[31],[32]].Themostfeature-richofthese,SigningSavvy,featureswordentrieswithdescriptions,images,memoryaids,Englishexamples,synonyms,abilitytoprintsignsoutonpaper,multiplevideosiftherearemultiplemeanings,andmore.Italsoallowsfortheabilitytocreatewordlistsofyourownthatcanbeusedasflashcardsorinquizzesandhasamobileappforsmartphonesandtablets[[32]].

Theideaofacommunityconstructedandcustomizeddictionaryhasalsobeenimplementedonlineindifferentways.Onemethodhasbeenthroughsignlanguagewikisthathavebeenestablished,usuallywiththehelpofaresearchgroup,thatutilizethecollaborativepowerofwikiswithsignvideospostedthroughYouTubeorsimilarmeansthatallowonetorecordavideofromtheirowncomputerwebcam.ThesesiteshavebeenreplicatedandusedforawidevarietyofsignlanguagesincludingFinnishSignLanguageandGermanSignLanguage[[33],[34]].

Anothercommunityconstructedsignlanguagedictionarysite,SignPuddle,usesawrittenversionofsignlanguagecalled

SignWriting(seefigure4),originatedin1974inDenmark[[35]].SignPuddlehasatoolusedto

Figure4.AsamplingofAmericanSignLanguagewordswrittenwithSignWriting

15

createsignsinSignWritingandthenallowonetoaddittothedatabaseanddictionaryofsigns[[36]].

Lastly,anunreleasedtechnologydevicetitledUNIwhichmainlyservestotranslatesignlanguagetospeechandtextisalsomarketingtoincludewithitsrelease“theworld'sfirsteasytousesignaddingsoftware”forcreatingcustomizablesignlanguagedictionariesaswellascloudmanagementsoftwaretouploadsigns,sharesigns,andfurthercombinesigndictionariestogether[[37]].

Takingalloftheseexistingsolutionsintoaccount,thereismuchtodrawfrombutnoexistingsolutioncouldaddresstheMathruCenter’sneeds.Mostimportantly,theMathruCentercurrentlyhasnoInternetaccess,renderinganyonlinesignlanguagedictionaryoutofreach.Further,theschooldoesnothavethecapabilitytolearnanduseawrittensystemofsignlanguagetodocumenttheirlocalsignlanguage.Manyexistinginterfacesarealsotime-intensivetodocumentsigns,suchastheprocessofcreatinganduploadingaYouTubevideothencreatinganewwikipageforeachsign,oneatatime.ThusanidealsolutionfortheMathruCenterwouldallowforaquickwaytorecordsignvideosandstorethemaswordsinanorganizeddatabaseofsignsthatarealsoquicklyaccessibleasadictionaryreference.

Wealsolookedindepthatthevisualpresentationofmaterialsinaneducationalsetting[[38]]andconsultedotherdigitalflashcardapplicationstoshapethevisualpresentationforSignBookincludingQuizlet,KitzKitz,andCram[[39],[40],[41]].Quizletallowsuserstoplacebothimagesandtextonflashcards,andrecordaudiotogoalongwithit.KitzKitz’ssolepurposewasprintingflashcards,however,wefoundtheinterfacedifficult.Cramoffereddifferentviews(mixturesoffrontandback)oftheflashcards. Methodologies

“Anotherindicationofthestatusofsignlanguageinacountryistheexistence

ofoneormoresignlanguagedictionaries.Adictionaryisafundamentaltoolto

bothpromotethestatusofandenhancetheuseofsignlanguage”[[16]].

DuringthefirstthreetofourweeksattheMathruCentertheteamengagedinmanyconversationswithteachers,spentmuchtimeobservinginclassrooms,andevenhadeverydaycasualinteractionsthathelpedtheteamslowlybutsurelyunderstandmoreabouttheschoolandbuildconnectionswiththeteachersandstudents.

TheteamhadbeeninitiallytoldthattheschoolusedAmericanSignLanguage(ASL).Oneoftheteammembers,havingabackgroundinASL,recognizedafewsignsincommonwithASL,butquicklyrealizedthattheschoolusedanothersignlanguagealtogether.Askingtheprojectmanagermoreaboutthesignlanguagethatwasused,theteamwasgivenanIndian

16

SignLanguage(ISL)dictionarytitled“TalkingHands”[[42]].However,thissetofsignsalsohadsomeoverlapbutwasfoundtodefinitelynotcorrespondtothesignlanguageusedattheMathruCenter.Eventually,onebookwasfoundwithbeginnersignswithKannadacaptions(thelocallanguageofBangalore)witharound480signscontainedinit.Thisbookwastheclosestwefoundtothelocallyusedsignlanguage,buttherewerestillsomemajordifferencesbetweenthebookandobservedsignlanguage,usuallyobservedwhensignsseemedtobeadoptedfromASLandISLsuchasthedaysoftheweekbeingthesameasASL.TheKannadabookofsignsalsocontainedonlynouns,lackingevenbasic

pronounsandverbs,whicharevitallynecessaryforlanguageandexpression.

ThisseriesofobservationsandfindingsledtothemaindiscoverythatinspiredthecreationofSignBook:thereisnoexistingcomprehensivereferenceordictionaryforthesignlanguageusedattheMathruCenter.Thesuggestionofaschooldictionarywasidentifiedinthe2013reportabouttheMathruCenter,butknowledgeofjusthowscarcetheresourceswereforthelocalsignlanguagewassurprisingtotheteamandhighlightedtheneedforsomeformofcommunityreferencetolearnfromandtobeusedinteachertraining.

Throughclassroomobservationswefurthernoticedthatteachersoftenusedfragmentedsignlanguagesentenceswhilethechildrenusedmorecompletesentences,astheyreliedonsignlanguageastheirmainformofcommunication.Manyteachers,especiallynewteachers,woulduseoneortwosignsinasentenceandtalkloudlythefullEnglishorKannadasentencetheyweresaying.However,thechildrenwouldoftenlookconfusedifthesignandfacialexpressionoftheteacherdidnotfullycapturethesentencemeaning.Thisindicatedthatthestudentswereheavilyreliantuponsignwhiletheteachersweremoreverballyfocusedduetoalackoffluencyinsignlanguage.Theseobservationsfurtherunderscoredtheneedforasignlanguagedictionaryreferenceforteacherstolearn,train,andpracticefrom.

Animportantnoteonourmethodologyintheseobservationsisthatweaimedtobeasunimposingandnonintrusiveaspossible.Inmanyearlyobservations,theteacherwouldasktheASLspeakingteammember,whattheAmericansignswerefordifferentwords,andsubsequentlyintroducethemalongwithherlessonplanasalternativesigns.Realizingthatthiscouldaddcomplicationtoteachingtheprimarilyusedlocalsignlanguage,showingASLsignswasstoppedduringobservations.

Figure5.“MyNativeTongue–1”TheonlyofficialsignlanguagereferencebookattheMathruCenter

17

ThedescribedseriesofeventsfollowsagreaterphenomenoncommonincultureswithvillagesignlanguagesasdescribedinResearchMethodsinSignLanguageStudies:APractical

Guide:

“Oneofthereasonswhysignlanguagesmayremainundocumentedforalong

timeistheircoexistencewithalargerormoreprestigioussignlanguage,which

often leads to a negative perception of the local sign language and to a

subsequenttendencytonotovertly identifywith itsuseand/ortoshifttothe

prestigioussignlanguage.”[[43]]

ThisquoteexplainsreasonswhytheschoolstaffperhapsdidnotidentifyashavingalocalsignlanguageatthebeginningandwereeagertolearnfromamoreprestigiousandwidelyusedsignlanguagelikeASL.

Otherinsightsgleanedfromclassroomobservationswerethatmanylessonsaretopic-driven.Everyclassobservedhadasetofwordsorfactsforthestudentstolearnbytheendoftheperiod.Thistopic-driventeachingstyleledtothetopic-organizedcategorizationsystemofSignBook.

Classeswerealsoveryvisuallybased.Whenateachercouldnotfindaflashcardorposterforthesubjecttodescribe,teacherswouldsometimesspenduptoseveralminutesdrawinganintricatedrawingonthechalkboardshowingthesubjectmatterwhilethestudentswereleftunengagedwithanymaterial.Evenafterschool,asthe2013reportmentions,teachersspenthourspreparingcut-outsofpictures,organizingflashcards,andhand-drawingotherdrawingsforuseinthelessonplanandinclass.ThismotivatedtheinclusionofpicturesinSignBook,andmethodsforuploadingexistingimagesortakingpicturestoallowforanyconceivableandsimplewaytoaddimagestodigitaldictionaryentries.Goingoverthisideawithoneteacher,sheremarkedthatitcantakemuchdescriptionandefforttoconnectawrittenwordandvisualsigntogethertorepresentaunifiedconceptforadeafchild,butseeingtheSignBookdictionaryentry,thestudentwillbeabletoinstantlyunderstandtheconcept.

Figure6.Anintricatedrawingbyateacher,usedtoexplainaclassroomlessoninthefirstgradeclassroom.

18

DevelopmentEarlyinterviewswith

teachersrevealedapreferencetowardlaptopapplicationsovertabletapplications.Basedontheteam’sprogrammingknowledgeandtherequirementsoftheapplication,theprogrammingenvironmentwaschosentobeinweblanguages(HTML,CSS,JavaScript,PHP)butbecompletelylocallyhosted,meaningthewebapplicationwouldnotrelyontheinternetbutuseonlyfilesonthecomputeritself.Thischoiceallowedformorerapidfrontenddevelopment,cross-platformcompatibility,andfuturepotentialoftransferringtheapplicationonlineforbroaderuseandavailability.Thisalsopromotesthelearningofcommoncomputerliteracyskillsdesiredbytheschool,touseaweb-basedapplicationonalaptop.ManyopensourcelibrarieswereusedincludingGoogle’sAngularframework,RecordRTCforvideoandpicturecapture,andPHPDesktopforallowingtheapplicationtoworklikeanativeWindowsapplication.

ThefeaturesoftheapplicationweredevelopedincorrespondencetotheneedsoftheMathruCenteranditerativelydevelopedandmodifiedbasedonusertestswithteachers.Table1(below)featuresdevelopedoverthecourseofthenineweeks,theirdescriptions,andwhatneedstheywerebasedupon.

Feature Description NeedAddressedVideoCapture Recordingtooltoquicklycreatea

.webmvideofilefromthewebcamandconnectitwithadictionaryentry

Simpleandfastwayforteacherstodocumentsigns

PictureCapture Similarcapturingtoolfora.gifpicturefromthewebcamtoconnectwithanentry

Fastwayto“scan”existingflashcardsandimagesortoshowphysicalobjectsandplaces

PictureUpload Uploadanyimagefileasapictureforanentry

Canusepicturesfromothersourcestoaddimagesnoteasilyfoundnearbyortakenviawebcam

Topical/FolderOrganization

Organizewordsintotopicsandsub-topicsinafolderhierarchy

Organizethedictionarysimilarlytothetopicalnatureofclassroomlessons

Figure7.TheSignBookinterface

19

CRUDOperations Create,Read,Update,andDeletedictionaryentriesandtopicfolders

Necessaryactionsformanagementofanydatabasesystem

MultipleVideos Canhavemorethanonesignvideowithaword

Allowsforsignsynonyms,signvariations,orevenmultiplesignlanguagesassociatedwithoneentry(alocalsign,ASLsign,etc.)

MultiplePicturesandMeanings

Canhavemorethanonepictureandmeaninginanentry

AllowsforasignhavingmorethanoneEnglishmeaningandmultiplevisualrepresentationsofaword

ViewModeShow/HideButtons

Show/hideanycombinationofanentry’svideo,picture,andmeaninginViewMode

Flash-cardlikeusageofthesigndictionary(showthesign,hidetheword,quizthestudent)

Cross-PlatformCompatibility

TheapplicationrunsonWindowsand(ifChromeisinstalled)Macstoo

CurrentdevelopmentwasdoneonMacs,withschoolusageonWindows

FastLoadTimes Navigatingbetweenentriesisdoneasynchronously(withoutnewpageloads)

Quickertouseandalsolesserrorproneinreadingandwritingfromthedatabase

KeyboardShortcutsforNavigation

Arrowkeysjumpbetweenwords,otherkeysactasshow/hideornext/previousshortcuts

Mousemovementsbetweenmanyfrequentlyusedbuttonsproveddifficultandforteacherstouseandslowedthemdown

HelpText/HoverText/VideoOverlays

Manyinstructionsandfunctionalityareprovidedviaon-screentext,whenyouhoveroverabutton,oroverthevideo/picturecapturescreen

On-demandhelpandinstructionsassistnewusersinmorequicklylearningandrememberinghowtousetheapplication

DatabaseBackups Backupfilesaresavedoftwokeydatastructuresonceadayandrecordedvideosareneverdeletedfromthecomputer

Datarecoveryiscriticaltobeabletodowhensomethinggoeswrong,anddatalosscoulddiscourageuseoftheapplication

Table1.SignBookTableofFeatures

StudyResultsandReflectionsTowardstheendofourfieldwork,wetrainedthemostseniorteacherforthedeafand

hardofhearingstudentsandthemainMathruCenteradministratoronallaspectsofSignBookwiththreeextendedtrainingsessions,aswellastwootherteachersonthebasicsofthetoolwithonetrainingsessioneach.Throughthistraining,teacherswereabletocreateaninitialvocabularyof30wordsacross7topics,almostallentriesincludingthethreeentrycomponentsofaword,video,andpicture.

20

TeacherssharedtheirresponsestoSignBookduringtrainingwithpositiveandexcitedcomments.Oneteacherremarked:

“This isverygoodbecauseIcanlearnonmyownandbefineteachingtothe

children.IfIask[theotherteacher]shewillteachmeafter4:00PM,butnowI

canaskandshewillenterinthis.Iwillseeagain,repeated,howtodothesign

–itistheeasyway!Icaneasilyseethisandthenteachtothechildren.”

OneofthemainfundersandsupportersoftheMathruTrust,uponseeingademonstrationofSignBookcommentedthat“Nowanyonecandocumenttheirsignlanguage,andevengiveacopytootherstolearn!Thishassomuchpotential.Potentialtoprovideaccesstoteachers,families,andcommunitiestocommunicatewiththedeaf.”

Therewasnotsufficienttimetoobserveclassroomuseofthetool,butaninformalmomentaroseattheendofateachertrainingwhen15childrengatheredaroundtolookatwhattheteacherwasdoing.AteammemberledthechildrenthroughViewModeforalloftheexistingwordssofar,showingandhidingthewordsandvideostoquizthestudents.Theyexcitedlyfinger-spelledorsignedtheanswersinresponseandexchangedathumbs-upandhandshakewhengettingtheanswercorrect.

ThefutureofSignBookisyettobeknown,butinitialreactions,users’quicklearninganduseofthetool,andexcitementfromteachers,students,andadministratorsshowpromisingpotential.MuchworkremainstobedoneinprovidingsignlanguageaccesstoDeafcommunitiesaroundtheworld,especiallythekindsoflocalvillagesignlanguagesasdiscoveredattheMathruCenter.SignBookisauniqueandexcitingstartinusingmoderntechnologytowardstheefforttoprovidethiskindofaccess.WehopetheworkanddocumentationofthisprojectwillcontinueontoinspireeffortinICTDworkandassistivetechnologyforthedeafandhardofhearingtogivethosewhomaynothaveearsthathearagreaterabilitytolearn,communicate,anddiscovermoreoftheknowledgeandexperiencestobehadinthegreatwideworldallaroundthem,richinvocabulary,visuals,andmeaning.

AdemoofSignBookisavailableonlineathttp://erikpintar.com/signBookDemo.

SPEAKUP!VOICEPOWEREDGAMESUITE

Themajorityoftheworldtakestheabilitytocommunicateverballyforgrantedeveryday.Throughouthistory,voicehasbeenthesocietalnormforcommunication;yetsuchcommunicationremainsinaccessibletodeafindividuals[[44]].InIndiathisisoftencoupledwiththeassumptionofdumbness–theinabilitytocommunicate.Whilethisassumptionisunderstandableindevelopingcommunitieswheredeafnessisoftentheresultofconditionsalsoaffectingmentalorotherphysicalability,itisinnowayanaccurategeneralizationofadeafperson’scapabilities.

21

TheSpeakUp!VoicePoweredGameSuite(SpeakUp!)wasinspiredbyaconversationwiththefounderoftheMathruTrust,Ms.GubbiMuktha.Shenotedboththeimportanceofspeechininteractingwithbroadersociety,aswellasexplainedthefinancialinfeasibilityofhiringprofessionalspeechtherapists.Shehadpreviouslyhiredaspeechtherapistforspeechclassesonceortwiceaweek,howevertherewas“notenoughprogress”tojustifythecost.Whengivensuchsmallamountsofdesignatedtimeforspeechtherapyandotherwiseremainingsilent,studentscannotmeaningfullyimprovetheirspeech.Whiletheexpertiseoftrainedspeechtherapistsisnecessaryforformallanguageacquisition,manyofthestudentsatMathruarepre-verbalandstruggletounderstandthepoweroftheirvoice.WewereinspiredtodevelopSpeakUp!asameansofencouragingstudentstoenjoyusingtheirvoiceandtodevelopmorecontrolovertheirvocalizations.

RelatedWorksBecauseIndiansocietypresentlydoesnotsupportanotionofDeafcommunity,thereis

anexpectationofthedeaftoassimilateintobroaderculture.“Peoplewithcommunicationdisabilitiesareoftenassumedtobeconsiderablylessablethantheyreallyare”[[45]].Thegoalofourvoice-poweredgamesistoempowerandencouragedeafandhardofhearingstudentstospeakforthemselves.Oursuiteofgamestargetspre-verballanguageskillsinafunandengagingwaybyhavingstudentsmodulatepitch,volume,andvocalization.

Speechtherapyisoneofthemostimportantroutesfordeafindividualstobeunderstoodbeyondacommunitythatusessignlanguage.Theimportanceofearlychildhoodspeechtherapyisalsocrucial,giventhatsoundformationandproductionbecomesfarmoredifficultwithage[[21]].Yet,prohibitivelyexpensivespeechtherapistfeeskeeptheMathruCenterfromincorporatingregularprofessionalspeechtherapyintotheirclassrooms.TheMathruCenteridentifiedmoresources/toolsforspeechtherapyasoneoftheirlargestneedsattheschool.

Whileopen-sourcespeechtechnologiesarestillunabletoproviderobustandindividualizedspeechtherapy,wefoundafewpackagesthatdosupportspeechtherapyforusewithatrainedtherapist.Onesuchpackage,VideoVoice3.0[[46]],requiresaspeechtherapisttoconfigurethegamesandpracticemodes,andallowsthespeechtherapisttotracktheirstudents’progress.ConsideringtheMathruCenterservesstudentswithwidelyvaryingcommunicationdisabilities,wehopethatthroughvoice-poweredgames,theMathruCenterstudentswillgainmorecomfortincommunicatingandexpressingthemselvesverballyoutsideoftheclassroom.Ourgamesusearangeofbasicaspectsofsound—volume,pitch,continuousvocalization,andtimedvocalization—tocontrolgameplay.Ourcodeisdocumentedandorganizedinhopesofencouragingfurtherdevelopment.Wedevelopedthegames’inputmetricsanduserinterfacethroughconversations,interviews,andtests,withtheconstanthopeofaidinginthemostpressingdifficultiesofspeech.

22

Wetransitionedfromspeechtherapytoolstogamesduetoourlackoftraininginprimaryeducationaswellastheinabilityforouropensourcesoftwaretorecognizesingularphonemes.Unlikehumans,speechrecognitionsoftwareoftenneedscontextualdatatoproperlyidentifywords,datathatwouldbeabsentwhenidentifyingisolatedphonemes[[47]].Phonemerecognitiontechniqueshaveachievedassmallasa25%errorrate[[48]],buteventhatistoolargefortheMathruCenter,giventhatmanystudentsarepre-verbal.Further,manymembersoftheMathruCentercommunitylacktechnologicalconfidence,sowewerewaryaboutusingerror-pronetechnologyasthebaseofoursuite.Hence,wedecidedtofocusonotheraspectsofsoundthataresimilarlyinstrumentalinachievingeffectiveverbalcommunication.

Methodologies WedesignedSpeakUp!toworkbestwithateacherorinstructorpresent.ThiskeepstheuseofSpeakUp!incontroloftheteacheranddoesnotimposetechniqueorstylelimitations.Ourgoalwastoallowstudentstovisualizetheirvoiceinasmanywaysaspossible.Providingdifferentinterpretationsofvoicetranslatesintoanunderstandingofthedifferenteffectsthatvolume,tone,andvocalizationhave.Wewantedtoofferavarietyofmodesandgameplay:modesthatsolelyprovidevisualizationsofvoice,modeswithnoendthatcanbeusedbyateacherinaclassroomsetting,andmodeswithacleargoalthatcanmotivatestudentstousedifferentaspectsofvoice.

TheSpeakUp!processwasinformedbyobservingteachers’speechclasses.The1stStandardteacherfocusedmostlyonindividualsounds.Shewentone-by-one,showedthestudentthelipandtonguemovements,heldstudentshandonherthroatorinfrontofhermouthsotheycouldfeelthevibrationsorair,andhadthemrepeatthesoundsmultipletimes.Thiswasverytaxingandtime-consumingfortheteacher,andbytheendhermouthwasdryandshewasexhausted.The4thStandardteacher,ontheotherhand,focusedonwordsandtaughtcommunally.Shewouldshowthestudentsobjectsandflashcards,andsaythecorrespondingwordloudly,accentuatingthelipandtonguemovements.Shewouldthenlaytheobjectstogetheronthetable,sayaword,andhavethestudentsidentifytheobjectshewasreferringto.Thisteacherlaterelaboratedonhertechnique.“Inthisschoolweusesigns,butoutsideoftheschoolpeopledon’t.Thatiswhywemustteachstudentstocatchlipmovements.”

Fromourclassroomobservations,wegainedanunderstandingofthebreadthofstudents’currentvocalcapacities.ItwasimportantforustodesignthevariousmodesofSpeakUp!toberelevantandusefultostudentsacrossthepre-verbalandpartially-verbalspectrum.ItwasalsocrucialtomakeSpeakUp!usablebymanydifferentsystemsandcomputers,notonlythenewestandmostexpensivemachines.

23

Development TheSpeakUp!projectbeganwithabroadsamplingofrelatedworksandunderstandinglocallimitationsoftechnologicalsolutions:lackofInternetaccess,unfamiliaritywithtechnology,andalreadyfullclassschedules.Ouroriginalidearevolvedaroundmoreconventionalspeechtherapytechniques.Wewantedtomakeanapplicationthattellsstudentstosayaspecificsound,andprovidesfeedbackonthesoundtheysaid(eitherbytellingthemwhattheysaidorscoringhowclosetheyweretothedesiredsound).However,afterasignificantamountofresearchintoopen-sourcespeechrecognitionAPIs,werealizedthattechnologytounderstandindividualphonemesdoesnotyetexist.Alargelimitationofspeechrecognitionsoftwareisthecommondependencyoncontextneededfordecipheringwordsfromsentences.Theproblemforusisthatthesesophisticatedstatisticalmethodsneedmorethanisolatedsounds–preciselywhatwewanttodetect.Forthesereasonsitbecameclearthatoursoundgamesweregoingtoneedateacher’sguidanceratherthanhavingitsownsoundrecognitionscheme.Further,withmoreresearchandobservationsattheMathruCenter,werealizedthattherearemanyaspectsofvoiceotherthanspeechthatourtechnologycanhelpstudentsattheCentergrasp. Westartedoffwithtwosimplegames:abattlegame,whichemphasizestimedvocalization,andarickshawgame,whichemphasizescontinuousvocalization.Wetestedthesetwogameswithateacherandthreestudentsafterschool,andthereactionswereverypositive.Theteacherthoughtitwasusefulinmakingspeech“fun”forstudents,andthestudentsseemedtoenjoyworkingtogetheronthesegames.Theteacherthengaveusfeedbackforfuturegames:shethoughtstudentswouldenjoy“funny”games,suchasagamerelatedtopickingfruitfromatree. WhileworkingontheFruitTreegame,wealsobegandevelopinggamesthatrespondtodifferentvolumeandpitchlevels.Wefeltitwasimportanttoshowteachersallthecapabilitiesofourgamesuitebeforefocusingononeortwoaspectsforgames.Asweshowedthesegamestoteachersandstudentsthroughoutprojectdevelopment,werealizedthattheconceptofvolumewasrelativelyeasytoconveytothem.Teachersfeltthatvolumegameswouldbeusefulinstrengtheningstudent’svoices,aswellasexplainingtothemwhichvolumestouseinwhichsituations.However,theconceptofpitchwasverydifficulttoexplain,andoftengotmiscommunicatedormisunderstoodasvolume.Basedonthisreaction,weshiftedfocusawayfrompitchgames,andmoretowardsvolumeandvocalizationgames.

Basedonourobservationsofthe1stStandardclass,wedevelopedmultipleFreePlaygames,whichrespondtoanystudentsoundandgoonforever,toallowstudentstopracticethesoundstheyaretaughtinawaythatisnottaxingtotheteacher.Basedonourobservationsof4thStandard,wemadeapicturegame;inwhichasthestudentsspeaksapicturegraduallyappears.Thisgameallowsteacherstoaddandcategorizetheirownwords

24

andpictures,andwillallowstudentstopracticesayingwordsandassociatingthemwithobjectsinamoreengagingway. Intheend,wedeveloped10modesacrossthreecategories.The“BeginnerSkills”modes:VolumeMeterandPitchMeter,focusedongivingsimplevisualizationsofaspectsofvoice.The“FreePlay”modes:FruitTree,Rickshaw,CollectTheFruit,andPictureThat!areendless,andareintendedtobeusedintheclassroomtohelpstudentspracticewords,sounds,andotheraspectsofspeech.The“Games”modes:Spaceships,DriveToMathru,FlappyBird,andFishGameallhavecleargoalsanddistinctends.Thesegameswereintendedtoencouragestudentsusedifferentaspectofvoice,andallowthemtohealthilycompetewitheachother.StudyResultsandReflections

InthelastthreeweeksinthefieldwefocusedontrainingtheteachersonhowtouseSpeakUp!andwereabletoobserveteachersusetheprograminclass.Oneteacherhadallthestudentssittingtogetheraroundthecomputerusingitcommunally.Oncertaingames,shewouldhavethemwatchherandmimicherlipmovementsforcertainwords.ThiswasespeciallytruewiththeFruitTreeandPictureThat!game,in

whichspecificobjectsappear.Withothergames,shegavestudentsinstructionsonhowtousethegames,andletthemmakeanysoundswiththeirvoicetooperatethem.

Teachersbegandesignatingthelibraryasthespeechclassspace.Thelibraryprovidesamoreopenspacewhereteachershavestudentsgatheraroundthelaptopwithoutobstacleslikedesksandchairsintheclassrooms.Oneteacheralsobroughtteachingaidestothelibrary(animalmodelsandflashcards),spreadthemout,andreferredtothemasstudentsusedthe

Figure8.TheSpeakUp!mainmenu

Figure9.AMathruCenterstudentusingtheSpeakUp!voicepoweredgamesuite

25

games.Thelibraryisalsoaquieterroom,isolatedfromotherclassrooms,whichprovideslessnoiseinterferenceinthegames.

WhileitistooearlytoassesstheimpactofSpeakUp!,theprogramseemspromisingbasedonformativeassessments.Therehavebeenclearbenefitsofthesoundgamestoindividualstudentsthatwehaveobserved.Onestudentalwaysmadethelipmovementforawordbutnevermadesound;shedidnotunderstandhowshecouldmakesound.However,afterjustafewminuteswithSpeakUp!shebeganmakingloudsounds,andcontinuedtomakethemthroughoutthegames.TeachersweresurprisedathowexcitedandengagedstudentsbecamewhenSpeakUp!gameswereused.Thoughteachersarestilltryingoutdifferentwaystousethegamesintheirclass,sofarPictureThat!isthemostcommonlyused,followedbygameslikeRickshawandVolumeMeter,whichallowstudentstopracticeanysoundtheteacherwantsthemtomake.

Therearesomeareasofconcernwehaveseensofar.Manyofthegameswereintendedforindividualstudentuse,butsometeacherspreferusingtheprogramcommunally.

Thedownsideofcommunaluseisthatstudentstendtobemakingsoundsinthebackground,soevenwhentheteachertriestofocusononestudent,otherstudents’voicesaretheonescausingthechangeonthescreen.Thiscanbeconfusingandcounter-instructiveforstudents,astheymaynotbespeakingandyetthegameisrecognizingvoice.Onepotentialsolution

wouldbetousethegameswithanexternalmicrophone,sinceseeingonestudent

holdingamicrophonewouldencourageothersnottotalk.Unfortunately,wewerenotabletotraintheteachersonexternalmicrophoneuse,butitisapotentialdesigndecisionworthexploringinthefuture.

WhenweobservedoneteacherusingSpeakUp!inclass,shewassoexcitedsheuseditfortwiceaslongasatypicalclassperiod.Shelargelyletstudentsuseitastheypleased,ratherthanincorporatingitintoherlesson.ThisraisestheconcernofwhetherteacherswillcontinuetouseSpeakUp!evenafterwearegone,becauseitmaynotdirectlyrelatetoschoolcurriculum.Unfortunately,wewerenotinthefieldlongenoughtoobservethenoveltyofSpeakUp!fade.Thus,allteachertraining,usertesting,andobservationswedidwereconfoundedbynotbeingabletoobservelong-termuse.However,duringourlastweekinthefield,wedidwitnessthenewestMathruCenterteacherusingthePictureThat!gamewithherthirdstandardclass.Whilewehadnottrainedherspecifically,shehadlearnedhowtousethesoftwarefromthefourthstandardteacherwehadtrainedtheweekprior.Sheenjoyedusing

Figure10.AteacherusingSpeakUp!communallywithherentireclass

26

thesoftwareandknewhowtoreopen,close,andshutdownthelaptops;allaspectsoftrainingwecoveredwiththeotherteachers.ThisdemonstratesthepotentialfortheMathruCentercommunitytoself-trainandspreadknowledgetofuturestaffandteachers.

DISCUSSIONWhentechnologyiscreatedwithcompassionandinrespectfulpartnershipwithusers,it

canhavethetransformativepowerofaffectingpositivechangeinthelivesofthedisempowered.TogetherwiththeMathruCenterfortheDeafandDifferently-abled(MathruCenter)andTechBridgeWorld,ourteamteamexploredhowtechnologycouldempowertheteachersandstudentsoftheCenter’sSchoolfortheDeaf.

BecauseofTechBridgeWorld’songoingrelationshipwiththeMathruEducationalTrustfortheBlind(MathruTrust)since2006,ourteamexperiencedthebenefitsthatcomewithastrongpartnership,suchastrustandrespect.WestillhadtobuildtrustanddidsobygettingtoknowMathruCenterteachers,students,andstaff.WealsoconductedaneedsassessmenttolearnabouttheMathruCenter.WhileapreviousiSTEPteamconductedasimilarneedsassessmentin2013,itwasusefulforourteamtounderstandthecurrentsituationattheMathruCenter.Asaresultofthe2013and2015needsassessmentfindings,wedevelopedtwoneweducationaltools.Thefirst,SignBook,providedgreatersignlanguageaccessthroughasignlanguagedictionarycreationtoolwithabilitiesforcustomvideoandpicturecapturingandcategorizationofentriesandwordsintotopics.Thesecond,SpeakUp!,motivatedstudentstostriveforgreaterverballanguageaccessthroughasuiteofvoice-poweredgamesaimedtofamiliarizepre-andpartially-verbaluserswiththepoweroftheirvoice.

ThetoolsweredevelopedinclosepartnershipwiththeMathruCenter.WetestedprototypesoftheSignBookandSpeakUp!toolswithMathruCenterstudentsandteachersandwiththeirfeedback,wefurtherimprovedthetools.Whenthetoolswereinastablestate,inthefinalweeksoftheinternship,weinvestedinthesustainabilityofthesolutionsbyconductingextensivetrainingwithteachersanddevelopingin-depthuserguides.Furthermore,TechBridgeWorldreleasedthetoolsopensourcesothatothersmaybenefitfromthesesolutionsaswellascontinuetoenhancethemindifferentcommunitysettings.

Figure11.MathruCenterteacherscreatingasigninSignBook

27

Withthesetools,whichwerecreatedwithcompassion,weaimedtoprovidelanguageaccessforDeafprimaryschoolstudentsandtheirteachers.OurworkwiththeMathruCentersupportedtheprocessofenablingIndia’sDeafcommunityintheircampaignforself-advocacythroughaccesstolanguage.Todisseminatethiscauseandraiseawarenessofdevelopingtechnologywithcompassion,theiSTEP2015teamgaveatalktoover100engineeringstudentsatPESUniversityinBangalore.Asaresultofthispresentation,aPESengineeringprofessorhasreachedouttoTechBridgeWorldaboutcollaboratingontechnologyprojectswiththeMathruEducationalTrustfortheBlind.

Thefutureofthisworkisbright.WhetheritisfurtherdevelopmentofSignBookorSpeakUp!oranadaptationofdevelopingtechnologywithcompassion,whattheiSTEP2015internshipwasabletodeliverafternineweeksinthefieldisanimportantrecordofcommunity-specificICTDworkandtwofunctioningprojectsbywhichtoshowcaseit.

ADDITIONALCONSIDERATIONSFORFUTUREWORK BytheendofnineshortweeksattheMathruCenter,theiSTEPteamwasabletocreateaSignBook,alocaldatabaseofanimationsdepictingasignandisusedtosupportnewteachers

andstaffinlearningthelocalsignlanguage,andSpeakUp!,asuiteofvoice-poweredgamesthatencouragesstudentstovocalizeandtoexercisethevolumeandpitchoftheirvoice.Muchmorecanbecontinuedinthisworktocreateassistivetechnologyforthedeafandhardofhearing. WithSignBook,theteamhadjustenoughtimetoincorporatethekeyfeaturesasderivedfromourobservationsandinterviewswithteachersandstudents.Thelargestadditionalrequestthatwedidnothavetimetoimplementwasforreviewandstudygamestogoalongwiththedictionarydatabase.Anexampleofareviewgamecouldallowateachertochooseanyofthethreedisplaytypes(video,picture,word)foraquestionandthenanotherdisplaytypeformultiplechoiceansweroptions.Thiscouldbeplayedwithallwordsorjustplaythroughonetopic’s

words.FunctionalityisalsolackingwithSignBooktomergewordsandtopicsfromtwodictionariestogether,whichlimitstheuseofSignBookacrosscomputersandwouldbean

Figure12.Technologyisoftentreatedasanovelty,sincestudentsdonotencountertechnologyineverydaysettings

28

excellentenhancement.FurtherideasforfutureworkaresummarizedinSignBook’sdeveloperdocumentationinitspubliconlinerepository1.

AsweworkedontheSpeakUp!gamesandwentthroughusertesting,werealizedthediverseusecasesitneededtoencompass.Thestudentshadwidelydifferentvocalcapabilitiesinareassuchasvolume,pitch,andphonemegeneration–theabilitytomakespecificsounds.Additionally,becauseourgameswereusedinextremelyvariantenvironmentsfromthenoisyclassroomtothesilentlibrary,itwasimpossibletomakegamesthataccountedforallthesedeviations.WebelievetechniquesfrommachinelearningcouldgreatlyenhancethefutureofSpeakUp!.Machinelearningwouldallowthecalibrationmenutoaccountfornotonlybackgroundnoiselevelsbutalsobackgroundnoisepatterns,suchasachirpingbird,alawnmower,orrelativesilence.MachinelearningingamesthatscalevolumeandpitchwouldallowSpeakUp!toshiftawayfromtheabsolutescaleitcurrentlyuses,toonethatcustomizesforeachstudent.Perhapsextremelysophisticatedmachinelearningtechniqueswouldallowgamestorecognizewhenstudentsaregettingclosertosayingaword,afeatureateacherrequestedinPictureThat!game.Unfortunately,wedidnothavethetimeorexpertisetopursuetheseoptionsinthefield.Therefore,wetriedtomakethegamesasdifferentlyusefulaspossible.Thereisstillalotofroomforimprovement.ThereisnodoubtthatmachinelearningtechniqueswillbeextremelyusefulinhelpingSpeakUp!reachthefurtherlevelsofusability,usefulness,andadaptability. OneofthemostimportantchallengesweencounteredwhenworkingwiththeMathruCenterwastechnologicalliteracy.Futureworkneedstobecarefulabouttheuseoftechnologythatiscurrent,butalsolocallyavailable.Accesstotechnologyislimited–whethertooexpensive,hardtofind,orotherwiseunavailable.TheMathruCenterdidnothaveInternetandlackedareliablecomputerlab.AmajordrawbacktotheCenter’scomputerlabisthatitsdonatedcomputersareoldorbroken.AfterreinstallingWindows7andprovidingabasiccomputertrainingsessionforteachers,theincorporationoftechnologyisnowmorefeasible.Furthermore,webroughttheMathruSchool’scomputerlabmanagertotheMathruCentertoworkonandassesstheconditionofthecomputerlabwithus.WeareconfidentthathecanfurthersupportthetechnologyneedsoftheMathruCenteronanas-neededbasis,thoughadditionalworkneedstoconsiderthefactthatnostaffmemberisfullydedicatedtotechnicalsupportattheMathruCenter. Whilewehopethatthetechnologyweintroducedthissummercontinuestodevelopcomputerliteracy,theMathruCenteralsostruggledwithhavingarobustorganizationalstructure.Wehavedoneourbesttoplantrootsforsustainabilitywithourtechnologyprojects,

1http://www.techbridgeworld.org/istep/downloads.html

29

werealizethattruesustainabilitydependsheavilyontheorganizationandteacherscontinuingtotrainothersandreceivetrainingonthesetools. Wetrulyencouragefuturedevelopmentandhopethatfutureteamsimproveuponwhatwewereabletodoinsuchabriefamountoftime.WewantfuturedeveloperstorememberthatICTDworkisnotaboutproductbutalsoaboutprocess.ThereisnodoubtabouttheaptitudeandgreatskillofdeveloperswhochosetouptakeICTs.Withouttechnicalsolutionstrulyunderstandingandbeinginformeddirectlybythecommunitytheytarget,theywilloftentimesmissthemarkofefficacyandrelevance.Butbyworkinghardtofirstdevelopandcultivatedeeperlevelsofcompassion,communication,andrelationship,thenICTDworkbecomesmoremeaningfulandmoreeffective.ThisisthehopeofICTD,thehopeofourwork,andthecallwegivetofuturedevelopers,designers,andresearchersinthismostrewarding,meaningful,andimpactfulwork.

30

REFERENCES[1] M.B.Dias,G.A.Mills-Tettey,andJ.Mertz,“TheTechBridgeWorldInitiative:Broadening

PerspectivesinComputingTechnologyEducationandResearch,”CarnegieMellonUniversity,Baltimore,MD,1-59593-301-8/06/01,Jun.2005.

[2] “iSTEP2009Tanzania.”CarnegieMellonUniversity,Summer-2009.[3] “iSTEP2010Bangladesh.”CarnegieMellonUniversity,Summer-2010.[4] “iSTEP2011Uraguay.”CarnegieMellonUniversity,Summer-2011.[5] “iSTEP2012Ghana.”CarnegieMellonUniversity,Summer-2012.[6] “iSTEP2013India.”CarnegieMellonUniversity,Summer-2013.[7] L.Dodson,S.R.Sterling,andJ.K.Bennett,“ConsideringFailure:EightYearsofITID

Research,”ProceedingsoftheFifthInternationalConferenceonInformationand

CommunicationTechnologiesandDevelopment,vol.9,no.2,pp.19–34,Mar.2012.[8] Rajaraman,“HistoryofComputinginIndia,”UniversityofMinnesota,Minneapolis,MN,

2010.[9] “TheStateoftheWorld’sChildren,”UnitedNationsChildren’sFundUNICEF,NewYork,NY,

Nov.2014.[10] M.R.Ho,T.N.Smyth,M.Kam,andA.Dearden,“Human-ComputerInteractionfor

Development:ThePast,Present,andFuture,”USCAnnenbergSchoolforCommunication&Journalism,LosAngeles,InformationTechnologiesandInternationalDevelopmentVolume5,Number4,Winter2009.

[11] N.Kumar,“Mobiles,Media,andtheAgencyofIndianYouth,”InformationManegementandSystems,UniversityofCalifornia,Berkeley,2013.

[12] M.Kam,D.Ramachandran,A.Raghavan,J.Chiu,U.Sahni,andJ.Canny,“PracticalConsideationsforParticipatoryDesignwtihRuralSchoolChildreninUnderdevelopedRegions:EarlyReflectionsfromtheField,”ACM,Tampere,Finland,1-59593-316-6/06/07,2006.

[13] C.Allen,“GlobalSurveyReportWFDRegionalSecratariatforAsiaandthePacific,”WorldFederationoftheDeafandSwedishNationalAssociationoftheDeaf,Helsinki,Finland,May2008.

[14] WorldHealthOrganization,“WHOglobalestimatesonprevalenceofhearingloss,”WorldHealthOrganization,2012.

[15] G.MathurandD.Napoli,Deafaroundtheworld.Oxford:OxfordUniversityPress,2011.[16] CenterforInternationalRehabilitationandKrishnanNehra,“InternationalDisability

RightsCopendium2003,”CenterforInternationalRehabilitation,Chicago,USA,2003.[17] TechBridgeWorld,“iSTEP2013:ExploringtheFeasibilityandSustainabiliyofAsstive

TechnologyattheMathruCenterfortheDifferently-Abled.”Aug-2013.[18] MediaCentre,“DeafnessandHearingLoss.”WorldHealthOrganization,Mar-2015.[19] NAD,“AuslanPolicy.”NationalAssociationoftheDeaf,25-Apr-1997.

31

[20] “StatesThatRecognizeAmericanSignLanguageasaForeignLanguage,”LaurentClercNationalDeafEducationCenter,GallaudetUniversity,Washington,DC,2004.

[21] T.Skutnabb-Kangas,“TSKLinguisticHumanRights.”NationalAssociationoftheDeaf.[22] S.Wilcox,“ASLasaForeignLanguageFactSheet.”UniversityofNewMexico,1991.[23] “WorldFederationoftheDeaf,2015.[Online].Available:https://wfdeaf.org/.

[Accessed:05-Aug-2015].[24] Ethnologue,'Summarybylanguagefamily',2015.[Online].Available:

https://www.ethnologue.com/statistics/family.[Accessed:04-Aug-2015].[25] Sciencedirect.com,'Estimatingsize,scope,andmembershipofthespeech/sign

communitiesofundocumentedindigenous/villagesignlanguages:TheBanKhorcasestudy',2015.[Online].Available:http://www.sciencedirect.com/science/article/pii/S0271530909000202.[Accessed:04-Aug-2015].

[26] Peacecorps.gov,'KenyanSignLanguage',2015.[Online].Available:http://www.peacecorps.gov/ksl/.[Accessed:04-Aug-2015].

[27] M.Bubu,'MzeeBubu-DeafPrideofKenya',Deafpride.blogspot.com,2010.[Online].Available:http://deafpride.blogspot.com.[Accessed:05-Aug-2015].

[28] AzimPremjiFoundation,“PerspectiveonContentDevelopment.”AzimPremjiFoundation,2011.

[29] E.Pintar,SignLanguageThroughPopMusic.Pittsburgh,PA,2014.[30] Aslpro.com,2015.[Online].Available:http://www.aslpro.com.[Accessed:04-Aug-

2015].[31] Handspeak.com,'AboutHandSpeak',2015.[Online].Available:

http://www.handspeak.com/info/index.php?info=about.[Accessed:04-Aug-2015].[32] SigningSavvy,'AboutSigningSavvy|ASLSignLanguageVideoDictionary',2015.

[Online].Available:https://www.signingsavvy.com/about.[Accessed:04-Aug-2015].

[33] SignWiki,'Um',2015.[Online].Available:http://is.signwiki.org/index.php/SignWiki:Um.[Accessed:04-Aug-2015].

[34] Wikisign.org,'WikiSign',2015.[Online].Available:http://www.wikisign.org/.[Accessed:04-Aug-2015].

[35] Signwriting.org,'SignWritingHistory',2015.[Online].Available:http://www.signwriting.org/library/history/.[Accessed:05-Aug-2015].

[36] Signbank.org,'SignPuddleOnline',2015.[Online].Available:http://www.signbank.org/signpuddle/.[Accessed:04-Aug-2015].

[37] Motionsavvy.com,'MotionSavvy',2015.[Online].Available:http://www.motionsavvy.com.[Accessed:04-Aug-2015].

[38] E.Hirshorn,“VisualAttentionandDeafness(ResearchBriefNo.3),”VisualLanguageandVisualLearningScienceofLearningCenter,Washington,DC.

[39] Quizlet,“OurMission.”QuizletInc.,2015.[40] Kitzel,“FreePrintableFlashCardMaker.”05-Nov-2007.

32

[41] “AboutCram.com.”StudyMode,2015.[42] B.Vacha,R.Ghate,M.Banerji,B.NanavatiandM.Gokhale,TalkingHands:Sign

LanguageBook.NASEOHRDCPublications,1980.[43] E.Orfanidou,B.WollandG.Morgan,Researchmethodsinsignlanguagestudies.John

Wiley&Sons,2015.[44] O.Sacks,SeeingVoices.Vintage,1989.[45] C.RowlandandOregonHealth&ScienceUniversity,“AssessingCommunicationand

LearninginYoungChildrenWhoareDeafblindofWhohaveMultipleDisabilities,”U.S.DeparptmentofSpecialEducation,Washington,DC,Health&ScienceH324D030001,2009.

[46] M.V.Corporation,“EntertainingGamesandDisplaysforSpeechTherapy.”08-Jun-2015.[47] “CMUSphinx:PhonemeRecognition.”CarnegieMellonUniversity,06-Aug-2015.[48] S.Fernández,A.Graves,andJ.Schmidhuber,“PhonemeRecognitioninTIMITwith

BLSTM-CTC.”TUMunich,22-Apr-2008.

33

APPENDIX

OURFINDINGSATTHEMATHRUCENTERCLASSROOMFORSTUDENTSWITHMULTI-SENSORYIMPAIRMENTS(MSI) ThebrevityofourfieldexperiencekepttheMSIclassroomoutsidethescopeofourprojects.WhilewewerenotabletodevelopnewtechnologiesspecificallyfortheMSIclassroom,wedidobservetheclassmultipletimesandhaddiscussionswiththeMSIteachersaboutthemostinhibitiveproblemsintheirwork.ThemostdifficultcircumstanceintheMSIcurriculumisthatitisuniqueforeachstudent;wecouldnotrelyonthesameacademicstructurethataffordedtheadoptionofourtoolsinagivenfixedcontext.SowiththecurriculaofMSIstudentsbeingbuiltindividually,theMSIteachersuggestedusinga“creativepractice” throughwhichatoolcouldworkformultiplestudents.ThesuggestionsforfutureworkastoldbytheMSIteacherincludehighcontrastimaging,tactilestorybooks,andsensorytools.Theseprojectideasare“thingsthatwouldhelpmanystudents” becausetheyexploredifferentsensoryexercisesaswellasprovidecreativityandenjoyment.

AdifficultpartoftheMathruCentersystemisthevaryinglevelsoffamilyandparentengagement.MoststudentsattheMathruCenterareresidential,howeverMSIinparticularalsohashome-basedanddaystudents.WhiletechnologyisnotviableinthehomesofmostMathruCenterdaystudents,toolsthatencourageengagementwithlocalsurroundingsandcommunitiescouldcreatepotentialwaysforparentsandfamiliestoengagewiththeMathruCentereducation.