A FRAMEWORK FOR EVALUATING APPROPRIATENESS OF EDUCATIONAL TECHNOLOGY USE IN GLOBAL DEVELOPMENT PROGRAMS TheMassachusettsInstituteofTechnology,Cambridge,Massachusetts&TheIndianInstituteofManagement,Ahmedabad,India

2

TheComprehensiveInitiativeonTechnologyEvaluation(CITE)atMITisaprogramdedicatedtodevelopingmethodsforproductevaluationinglobaldevelopment.CITEisledbyaninterdisciplinaryteam,anddrawsupondiverseexpertisetoevaluateproductsanddevelopanunderstandingofwhatmakesproductssuccessfulinemergingmarkets.TheIndianInstituteofManagement,Ahmedabadwasanessentialpartnerinresearchandtestingofthiseducationaltechnologyframework.

ThisreportwasmadepossiblethroughsupportoftheUnitedStatesAgencyforInternationalDevelopment.TheopinionsexpressedhereinarethoseoftheauthorsanddonotnecessarilyreflecttheviewsoftheUnitedStatesAgencyforInternationalDevelopmentortheUSGovernment.

Table of Contents

INTRODUCTION 3DEVELOPINGAFRAMEWORK 4GENERALIZABILITYANDUSEINOTHERCONTEXTS 6USINGTHEFRAMEWORK 7COMPREHENSIVEFRAMEWORK 9CONSIDERATIONSFORFUTUREAPPLICATION 18REFERENCES 18AUTHORS&ACKNOWLEDGEMENTS 19

3

Introduction

Asdigitalmediaproliferate,andincreasingamountsofdailyworkareperformedindigitalenvironments,therearegrowingdemandsfromparents,educators,andgovernmentstodeployeducationaltechnologiesinglobaldevelopmentprograms,whetherasameanstoimprovethequalityofeducationingeneral,orastoolstofamiliarizestudentswiththetechnologiesthatwillshapetheirfuturelives.Thisdemandcanbedrivenby:

• Enthusiasmforemergingtechnologies;

• Expectations(realisticorotherwise)forwhattechnologycanachieve;

• Fearofbeingleftoutofemergingsocio-culturaldevelopments;or

• Alloftheabove.

Whileallschoolsmayshareacommongoalofeducatingstudents,thereisabroaddiversityofmeans,dependingonsuchvariablesas:

• Schoolfunding;

• Teacherpreparedness;

• Educationalphilosophy;or

• Technicalinfrastructure.

Unfortunately,manyconsumersoftechnologicalinterventions—policymakers,administrators,teachers,andparents—failtoaccountforthesevariablesinmakingdecisionsabouttheadoptionofanyparticulartechnology(Davies,2011).Andmanydeveloperscreatetechnologicalinterventionswithoutfullyunderstandingtheeducationalsystemsintowhichtheywillbeintroduced.

Therefore,thereisaneedforevaluativetoolsthatwill:

• Aidvariousstakeholdersindeterminingwhicheducationalinterventionsaremostpromisingforanyparticularcontext;and

• Helpstakeholdersevaluateinterventionsastheyareintheprocessofimplementation(i.e.formativeassessment).

Itwouldbedifficulttooverstatethechallengesinherentinanyefforttoobjectivelyevaluatetheeffectivenessofagiveneducationaltechnology.Tobegin,thenotionofwhatconstituteseffectiveeducationishighlycontested,eveninadevelopedcountrysuchastheUnitedStates,withaneducationalsystemthathashad150yearsofrelativepeaceandprosperityinwhichtoprogressandevolve.Oneneedonlylookatthelivelydebatesthatrevolvearoundsuchquestionsastheusesofstandardizedtesting,commoncorecurricula,charterschools,ortheroleofcomputersinchildren’seducationaldevelopmenttoacknowledgethatthereisnoconsensusonwhateducationalsuccesslookslike,andsimilarlynoconsensusonhowtomeasureanyputativesuccess.Evenwhenoneidentifiesthedesiredandmeasureableoutcomesofparticularintervention,thechallengesandcostsofperforming

4

scientificallyvalidassessmentsusingrandomlyassigned,singlevariabletreatmentandcontrolgroupsinsufficientlylargepopulationscanbestaggering.

Ifweshiftourattentiontothedevelopingworldwearelikelytofindevengreaterchallengesresultingfromunder-fundedschools,ashortageofprofessionaleducators,andalimitedtechnologicalinfrastructure.Inaddition,oneencountersthesamedisagreementsaboutthepurposeofeducationandwhatconstitutessoundpedagogyasarefoundinthedevelopedworld.Anymeaningfulevaluationmustaccommodateitselftoallthesefactors.

Webeginbyacknowledgingthesechallengesnotinthespiritofresignation,butrathertoavoidthegrandioseclaimsthatarealltoooftenmadeonbehalfofeducationaltechnology.Theauthorsofthisreporthaveworkedfordecadesbothdevelopingandevaluatingeducationaltechnologies.Weremainoptimisticabouttheroleoftechnologyineducation,butwealsounderstandthatthegreatestrisktotheadoptionofanytechnologymaybeunrealisticexpectations,andsubsequentdiscouragementandprematurecapitulationtodefeatism.Accordingly,thisreportidentifiesmethodsofevaluatingeducationaltechnologiesthatarebothpracticalandadaptabletoawiderangeofeducationalsettings,andthatwillresultintheadoptionanduseoftechnologiesinwaysthatarepotentiallysustainableindevelopingenvironments.

Developing a Framework

Todevelopausefultoolinsuchcomplexcontexts,wechosetodevelopaframeworkthatpullsfromandsynthesizestheexistingliteraturerelatedtothisspace,aswellasdataandfeedbackfromrealworldcontextsthatareseekingtoeffectivelychooselearningtechnologiesandwouldbenefitfromexternalsupportsandtoolsindoingso.

Frameworksofferseveralbenefits.First,theyclarifycomplexorambiguoussituations(Whetten&Cameron,2014).Agoodframeworklaysoutthedimensionsofthecomplexityoftheproblemspace—manyofwhichwouldoftenbeotherwiseoverlooked.Agoodframeworkputsalloftheseelementsanddimensionsontheradarofthepeopleinvolvedsothateachcanbeconfrontedandaddressedappropriatelyinthereal-worldcontextandproblem.Second,agoodframeworkwillalsohelppromptandsupporteffectiveengagementwitheachofthosedimensionsasitrelatestotheircontext.Inotherwords,theframeworknotonlyframestheentireproblemspace,butitframeshowtotakestepsingettingtowardsaneffectivesolution.Eveninacomplexproblemspace,frameworksserveasbothanchorsand/ortouchstonestoreturnto,providingstabilityinthemidstofconstantchange(Whetten&Cameron,2011).

Wechosetostructureourframeworkasaquestionnaire,withthequestions(andbyextension,theframeworkitself)performingtwofunctions:

1. Whenusedbyanoutsideevaluator,thequestionscanstructuretheexplorationofallthesalientelementsofaproposedintervention,oronealreadyinprocess.

2. Whenusedbyastakeholdereithercreating,oradoptinganintervention,thequestionsactaspromptstohelpthestakeholderfullyreflectontherangeofrelevantissues,someofwhichmayhavepreviouslygoneunconsidered.

5

Thoughitwouldbebeyondthescopeofthisstudy,itwasourhopethatiftheframeworkprovedeffective,wewouldseekthemeanstodevelopanon-lineversionthatcouldbeadministeredeitherbyevaluatorsorstakeholders.Thiswouldtakeadvantagesofthebranchingcapabilitiesinadigitalenvironmenttotailornavigationthroughtheframeworkbasedonuserresponses.

Thegoalfortheinitialpilotwastotesttheusabilityoftheframeworkinthefield,andtomodifyitwhereappropriatebasedonuseexperiences.

Informing the Framework Followinganextensiveliteraturereview,theframeworkwasdevelopedbyresearchersattheMassachusettsInstituteofTechnology,andthentestedinIndia,withasmallteamofinvestigatorsfromtheIndianInstituteofManagement,Ahmedabadcarryingoutfieldworkandanalysis.Theaiminthepilotwastwofold:(1)totesttheframeworkoutinvariouscontextswhereeducationaltechnologywasbeingusedandseewhetherthequestionswewereaskingwererelevantinthesecontextsand(2)whethertherewasanythingofimportancetovariousstakeholdersinthefieldthatwewerenotasking,butshouldbe.Wethereforeusedthisasaniterativeprocesstoimprovetheframeworkbasedondatafromthefield.

Tomeetourrequirementofsimultaneouslytestingandaddingtotheframework,wedecidedonholdingsemi-structuredinterviewsand/orgroupdiscussionswithvariousstakeholders—developers,implementingorfacilitatingagencies,schoolleadersoradministrators,teachers,studentsandwhereavailable,parents—insiteswherethetechnologiesstudiedhadbeenpilotedordeployed.Asemi-structuredformatallowedustoguidethediscussionsbasedontheframework,butalsotobeopentodivergencesandopen-endedresponsesthatmightleadustotopicsthatourframeworkshouldfocuson,butdidnot.However,becausethedatacollectionandanalysisfromsuchmethodsareresource-intensive,thenumberofsitesforourstudywerelimited.

Theschoolsweobservedinthepilotstudyrangedfromthemostbasictothemostmodern,including:

• AruralvillageinUttarPradeshwithoutaschoolbuildingordedicatedclassroom,whereNGOEkalVidyalayahadtrainedvolunteersinthedeliveryofrudimentarymathandreadinglessonstochildrenfromages5to10

• AruralvillageinGujarat,whereanotherNGO,PlanetRead,providedvillagerswithsubtitledBollywoodmoviestobeviewedcommunallytoreinforceliteracy

• RiversideSchool,amodernindependentschoolinAhmedabadwithfacilitiesthatwouldmatchthoseofaprogressiveprivateschoolintheU.S.;studentsusedcomputertechnologyregularlyintheirschoolwork

• AseriesofcomputerlabsinbothpublicschoolsandcommunitycentersinMumbaiandPunerunbythePrathamEducationFoundation,oneofthemostinfluentialNGOsworkingtoimproveeducationinIndia

6

Ourschoolvisitsconfirmedoursuppositionthatinanygivensettingmultiplevariableswereinplay,suchas:funding,technologicalinfrastructure,pedagogicalmodels,andteacherengagement.Foraframeworktobeeffectiveitwouldhavetohelpevaluatorsorstakeholdersdeterminethefitofagiventechnologywithallsuchvariablesaccountedfor.

Inaddition,westudiedthreeEnglishlanguagelearningtechnologyinterventionsthathadeitherbeenrecentlydeployedorwereintheprocessofdeployment,takingintoaccount(a)diversityofcontext,natureofuseandstageofimplementationand(b)accesstothevariousstakeholders.Thetechnologiesincluded:

1. EnglishHelper(EH)RightToRead:Aread-alongsoftwareforclassrooms,describedasmultisensorysinceitsimultaneouslyengagesvisionandhearing.TheprimarytargetseemstobestudentsinearlystagesoflearningtoreadEnglishasasecond(orthird)language.

2. Mindspark(MS)English:Acommercialadaptivelearningsoftware,marketedtobothschoolsandindividuals.Mindsparkisintendedforusedirectlybylearnersanditssubscriptionfeemakesitinaccessibletolower-incomegroupswhoenrolltheirchildreningovernmentschools.

3. EkStep:AprojectthataimstoaddresslearninggapsinEnglishandMathinprimaryeducationatanationalscaleusingtechnology.EkStepwasinterestingtoussinceittargetsbothformalandnon-formaleducation,andaimstoquicklyscaletomillionsoflearnersoncedeployed.

Sincethepilotwasintendedtobeaniterativeprocesstoimprovetheframework,thedatacollectedfromeachsitewasdiscussedextensivelybytheteamandusedtomakeadditionsormodificationstotheframework.Thisupdatedframeworkwasusedatthenextvisit,whethertothesamesiteoranewone.Theframeworkthatfollowsisinformedbythefindingsofthispilotstudy,whicharedetailedinthefullreportavailableatcite.mit.edu.

Generalizability and Use in Other Contexts Asinitiallydevelopedandsubsequentlymodified,theframeworkaddressesquestionsof“fit”betweenaparticularinterventionandagiveneducationalcontext.Forthesakeoffocus,wepilotedtheframeworkbyevaluatingtheuseofEnglishlanguageandliteracyproductsbeingdeployedinIndia.Theparticularcircumstancesweencounteredonthegroundinformedourthinkingandcontributedtofurtherrefinementoftheframework.

Whileitfollowsthatfurtherapplicationsoftheframeworkmightleadtofurtherrefinements,weneverthelessbelievethatthisframeworkisrobustenoughtosuccessfullybedeployedinothercontexts—betheyotheracademicdomainsorothercountries’educationalsystems.Webasethisconfidenceonthenatureofthemodificationsthatoccurredduringthepiloting.Specifically,thesemodificationstendedtointroducenewdimensionstoquestionsalreadypresentintheframework,likethoseofimplementation,equilibrium,orengagement.Thereisnothingintheexperienceofthisinitialpilottosuggesttherearesignificantgapsinitscurrentscope.

7

Using the Framework

Inthecourseofthisstudywedevelopedaframeworktobeusedbyvariousstakeholdersinassessingtheappropriatenessofneweducationalproductsorinterventionsincluding:

• Developersofnewtechnologies;

• Adoptersofnewtechnologies,includingsystem-wideadministrators,schoolprincipalsorteachers;or

• Fundersofnewtechnologies.

Theframeworkisintendedtobeusedbeforetheadoptionofanintervention,orasaformativeassessmentofthatinterventionasitisbeingdeployed.Thoughwepilotedthisframeworkinthecontextofthespecificdomainoflanguagelearning,ourlargergoalwastocreateatoolthatwouldbemorebroadlyapplicable.

Theframeworkisdesignedtoassistmultiplestakeholderstomoresuccessfullyplanandimplementaneducationaltechnologyinagivensetting.Thiscouldrangefromasingleteacherwantingtotryanewtechnologyinhisorherclassroom,toanentiredistrictorstatesystemofeducationseekingtoimplementatechnology-basedlearningprogram.

Forthecaseofindividualteachers,thismightinvolvereviewingtheelementsoftheframeworkinregardstoanyparticulareducationaltechnologytheydesiretouse,andreflectingontheanswersinordertoidentifyandaddressandbarriersthatmaycomeup.Forexample,theframeworkincludeselementssuchas"technologyresourcesrequired,"whereateachermightidentifythatthetoolinquestionrequireshigh-speedWi-Fiinorderforthestudentstoeffectivelydothetypesoflearningactivitiesintheclassroomthattheteacherimagines,andthereforetheteachermighteitherchoosetonotusethetechnologyorfindwaystoworkaroundthis.Anotherexamplewouldbe"accesstoresourcesandsupports,"andinreviewingthispartoftheframeworkteachersmightrealizethattheydon'thaveanyoneintheirschoolwhocanhelpthemandthereforeseekoutsomeoneintheirprofessionalnetworkwhoknowsaboutthistoolthattheycanconnectwithshouldtheyrunintoanyproblemswhileusingthetoolintheclassroom.

Foranyoftheelements,theindividualmightrespondtothepromptinoneoffourways:

1.Idon’tknowtheanswersyet;

2.Nothespecificconditionsarenotpresent;

3.Itwillrequireadditionalresourcestowork;or

4.Thereisafit/theconditionsarepresent

Foragivenaspectoftheframework,iftheresponseis#1,thentheuserneedstodigdeeperandbetterunderstandthetooland/orthecontextandresourcesheorsheisworkingwith.Iftheresponseis#2or#3,theuserneedstoconsiderwhatwouldneedtochange,andifthatchangedesirableorevenpossible.Iftheresponseis#4,theuserhasidentifiedanareaofstrengthforthisimplementation,whichmaybeabletohelpsupportotherareasthatarelacking.

8

Framework Overview

TEACHERS COMMUNITY,SOCIAL,POLITICAL

T.1.ComfortT.2.CompetenceT.3.OpennesstoChangeT.4.RoleT.5.ClassroomManagement

CSP.1.ImplementationCSP.2.Support

STUDENTS LEARNING

S.1.ComfortS.2.AccessS.3.OpennesstoChange

L.1.LearningGoals/ImpactonLearningL.2.PedagogyL.3.Curriculum

CULTURE INFRASTRUCTURE

C.1.CulturallyRelevancy I.1.EquipmentI.2.ElectricityI.3.Internet

SUSTAINABILITY SCALABILITY&MARKETIMPACT

SU.1.FundingSU.2.Maintenance&Repairs

SM.1.BroaderCommunityImpactSM.2.Adoption&Scaling

9

Comprehensive Framework

TEACHERS

T.1. Comfort

T.1.1. ComfortwithTechnology

Howcomfortablearetheteacherswithtechnology?Intermsofgeneraluseaswellasinaneducationalsetting.

T.1.2. ComfortwithTeachingStudentsTechnology

Howcomfortableareteachersinteachingstudentshowtousethetechnology?Asis,andthenwithadditionaltraining.

T.2. Competence

T.2.1. ProfessionalDevelopmentRequired

Howmuchlearningofthetechnologywouldteachersneed?Andwhatisthestructure?(onedayvs.multiplesessions?)

T.2.2. ResourcesforProfessionalDevelopment

Whowouldprovidetheinstruction?Outsidevs.in-schoolemployee

T.2.3. ProfessionalDevelopmentScheduling

Whenwouldtheinstructionhappen?Areadditionalworkhoursneeded?

T.2.4. ProfessionalDevelopmentCosts

Whatadditionalcostsareassociatedwiththeinstruction?Dotheteachers,school,ortechnologycompanycoverthesecosts?

T.3. OpennesstoChange

T.3.1. LearningTechnology

Areteacherswillingtolearnhowtousethetechnology?Howmuchtimearetheywillingtoputintolearnhowtousethetechnology?Isthereanassociatedjobtrainingbenefitoflearningthetechnology?

T.3.2. LearningNewPedagogies

Areteacherswillingtochangetheirpedagogytoaccommodatetheuseoftechnology?Hasitbeenmadecleartoteacherswhytheyareusingthetechnology?Isthetechnologyinalignmentwithteachers’currentlearninggoalsforstudents?Isthetechnologyinalignmentwiththeschool-widegoalsforlearning?

10

T.4. Role

T.4.1. RolewithTechnology

Whatistheroleoftheteacherintheimplementationofthetechnology?Isthetechnologyseenasan“addedresponsibility”ora“teacherreplacement”withoutanybenefits?Isthetechnologyperceivedinapositivelight,asatooltoaidinteaching/learning?Howdoestheteacherinteractwithstudentsusingthetechnology?

T.5. ClassroomManagement

T.5.1. MonitoringTechnologyUse

Howwillthetechnologyusebemonitored(sostudentscannotaccessinappropriatecontent)?Doesthetechnologycompanyputrestrictionsinplace?Aretheteacher/schoolresponsibleformonitoringcontent?Dotheyknowhowtoeffectivelysetupmonitoring?

T.5.2. DemandsbytheTechnology

Doesthetechnologycreateaburdenofextramanagementfortheteacher?Doesthetechnologymakelearningmoreefficientandeffectiveintermsoftimefortheteacher?Istheteacherawareofhowthestudentsareusingthetechnologyatanindividuallevel?Doestheteacherreceiveusageandprogressreportsorcantheymonitorusageeasily?Doesmonitoringtheusagetakealotofextraeffortfortheteacher?

STUDENTS

S.1. Comfort

S.1.1. ComfortwithTechnology

Whatdotheyknowhowtodo/whatistheircomfortlevelwithtechnology?Includingthekindsoftechnologytheyhavecomfortwith(phones,tablets,PCs,etc.)andtheactionstheyarecomfortablewithusingthetechnologyfor(i.e.wordprocessing,apps,internet,etc.)

S.1.2. StudentSupport

Howmuchinstructionwouldstudentsneedtousethetechnology?WHENwouldthishappen?Inschoolvs.afterschoolhoursWouldstudentsbewillingtocomeinafterschoolfortheinstruction?WHOwouldprovidetheinstruction?Teachersvs.outsidefacilitatorsHOWMUCHwouldtheinstructioncost?Intermsofteachertimeoroutsidefacilitatorcost/timeWillstudentsmastertechnologywithgreaterfacilitythanteachers?Couldtheyassisttheteacherinmasteringthetechnology?

11

S.2. Access

S.2.1. StudentHomeAccess

Whattechnology,ifany,dostudentshaveaccesstoathome?Inotherpublicspaces(i.e.publiclibrary,afterschoolprograms,etc.)?Dotheyshareaccessordotheyhaveindividualdevices?

S.2.2. Student-TechnologyAccessNeeds

Wouldtheyneedaccesstothetechnologyathomeaswellasinschool?Iftheyneedaccessathome,howfrequentlyandhowmuchtimeperuse?Istechnologyequallyaccessibleinallhomes?

S.2.3. EquitableAccess

Willtherebeequitableaccesstotechnologyforstudentsbetweengendersandage-levels(whereappropriate)?Howcanthisbeensured?

S.3. OpennesstoChange

S.3.1. LearningTechnology

Howwillingarestudentstousethetechnologyinschool/athome?

S.3.2. PerspectiveonNewTechnologies

Dostudentsviewtechnologyasanopportunityoraburden?Arestudentsexcitedaboutthechancetousethistechnology?Aretheynervousaboutusingthistechnology?Doesthisdependontheirage/gender?

12

COMMUNITY/SOCIAL/POLITICAL

CSP.1. Implementation

CSP.1.1. OutsideFacilitatorNeeds

Isthereafacilitatorfromoutsideoftheschoolthatisnecessarytoimplementthetechnology?Howaretheyassociatedwiththeschool?Whatisthedynamicbetweenthefacilitatorandtheteachers/students/administration?(intermsofinteractionandculture)Doestheschoolhavetopaythefacilitator?Howmuch?Whatarethemotivationsoftheoutsidefacilitator?Isthefacilitatingagencystableoverthelonghaul?

CSP.1.2. TeacherRequirements

Whatisthedegreethatthetechisimplementablewithouttheteacher?Doteachers/facilitatorsneedtoalwaysbepresent?Sometimespresent?Neverpresent?

CSP.2. Support

CSP.2.1. PerceptionsofTechnology

Howisthetechnologyperceivedbythecommunity?Opportunityvs.unnecessary

CSP.2.2. SupportoftheTechnology

Istherepoliticalsupportfortheuseofthetechnology?Inwhatform?Localgovernment?Stategovernment?

CSP.2.3. ReportingNeeds

Isitnecessarytoprovidereportstoanycommunity/political/fundingorganizations?Howoftenmustthesereportsbedone?Whatisthenecessarycontentofthereports?Doesthecompletion/accuracy/contentofthereportsdeterminefundingorsupportoftheprogram?

CSM.2.4. Governmental/AdministrativeApproval

Isgovernmentapprovalnecessarytousethetechnology?Ifso,howdoesapprovaloccur?Istherealongtimelagtogainapproval?

13

LEARNING

L.1. LearningGoals/ImpactonLearning

L.1.1. LearningGoals

Whatarethelearninggoals?(teacherandschool-based)Arethesegoalscurrentlybeingmet?Isthetechnologyappropriateforaddressingthesegoals?Isthetechnologynecessarytoachievethesegoals?Istechnologythebestmethodforachievingthesegoals?Whatlearninggoalswillnotbemetbythetechnology?

L.1.2. EvidenceofLearning

Isthereevidencethattheuseofthistechnologyaidslearning?Whatistheevidence?Isitreliable?Istheevidencegeneralizabletothiscontext?

L.1.3. MeasurementofLearning

Howwilllearningasaresultofthetechnologybemeasured?Standardassessments,pre/posttests?Qualitativemeasures?Willassessmentsalignwithexistinglearninggoalsorbetailoredtothetechnology’saffordances?

L.2. Pedagogy

L.2.1. CurrentPedagogicalModel

Whatisthepedagogicalmodelrightnow?DirectInstructionbyteachers?CollaborativeLearning?Inquiry-basedLearning?Project-basedLearningProblem-basedLearningArepedagogicalapproachesuniformacrosstheschoolordoteachershavesomeautonomyintermsofteachingstyles?

L.2.2. CurrentClassroomTools

Whattoolsareusedtoteachintheclassroom?TextbooksWorksheetsHand-outsGamesHands-onmodelsOther?Arecomputersusedintheclassroom?Ifso,how?Aremobiledevicesusedintheclassroom?Ifso,how?

L.2.3. ProposedPedagogicalModel

14

Whatshouldthepedagogicalmodellooklikewiththetechnology?Doesitneedtobeschool-wide(allteachersadopt)?Studentcenteredvs.teachercentered

L.2.4. BlendedLearningCapacity

Towhatextentareteacherspreparedtoimplementablendedlearningenvironment?Areteacherswillingtochangetheirpedagogicalpracticestoutilizethetechnology?(seeteachers>willingnesstochange)

L.2.5. ImpactonCurrentPractices

Whatwouldbetheimpactonteachingpracticeswithtechnologyintheclassroom?Newpedagogicalmodel?Needtoadapttoanewstyleofteaching?Needtoworkcloselywithasecondteacher/facilitatortohelpstudentsusethetechnology?

L.3 Curriculum

L.3.1. Technology-CurriculumFit

Howdoesthetechnologyfitwithinthecurrentcurriculum?Isitanaturaladditiontosupportlearning?Wouldthetechnologysignificantlychangethecurrentcurriculum?Islocaladaptation/modificationofthetechnologypossible,andatwhatcost?

L.3.2. Technology’sRole

Isthetechnologydesignedtobeastand-alonetoolortoprovideextrasupportforthecontentthatisalreadybeingtaught?Istheschoolwillingandabletousethetechnologyasithasbeendesignedtobeused?(i.e.throwouttheoldcurriculumifnecessary)Doesthetechnologyempowerlearnerstocreateknowledge,orrequirerelianceondominant/mainstreamsourcesofknowledge.

CULTURE

C.1. CulturalRelevancy

C.1.1. CulturallyAppropriate

Isthetechnologyculturallyappropriate?Intermsof…Content?Structure?Agelevel?Implementationmodel(i.e.doesitengageallnecessarystake-holdersInteractionbetweenstudents/teachers/genders?

15

INFRASTRUCTURE

I.1. Equipment

I.1.1. EquipmentRequired

Whatequipmentisnecessaryforthetechnology?Besidesthemainequipment(i.e.computers/tablets/other),arethereotheraccessories(keyboards,projectors,etc.)thatwouldbenecessarytousethetechnology?Howmuchimpactwouldtheadditionalequipment(accessories)haveonlearning?(i.e.aretheyessential?)

I.1.2. EquipmentSourcing

Whoisprovidingthisequipment?Aretheydonatingtheequipmentoristhereacost?Whatisthecost?Isitpaidonetimeorasanannualfee?Willthedistributorcoverrepairsandmaintenanceorwillthatbecoveredattheschoollevel?Ifcoveredattheschoollevel,whatisanestimationofthecost?Isitpaidasinsurance(annually)orasproblemsarise(feeeachtimearepairneedstobemade)?Doesthetechnologyhelpreduceotheroperationalorcapitalcosts?

I.1.3. Storage

How,andwhere,willtheequipmentbestored?Isthespacesecure?Howwillitbeaccessed?Whowillhaveaccesstoit,andwhen?

I.1.4. Maintenance

Whatongoingmaintenancecanbeanticipated?Whowillconductandmanagethismaintenance?Arethereareenoughresourcesavailabletosupportthis?

I.2 Electricity

I.2.1. ElectricityRequirements

Willelectricitybenecessary?Ifso,isitjustforcharging?Ordoesthetechnologyalwaysneedtobepluggedin?Istheelectricityreliable?(Howoftendoesitgooutattheschoolandforhowlong?)

I.3. Internet

I.3.1. InternetRequirements

Doesthetechnologyrequireinternetaccess?Ifso,whatkindofspeedisnecessaryperdevice?

I.3.2. SchoolInternetResources

Doestheschoolhaveinternetaccess?

16

Ifso,where?Howreliableisit?(alwaysworksvs.sometimesworks)Istheresufficientbandwidthtosupportthetechnology?Howmanystudentscouldbeonlineatonceusingthedevice/program?

SUSTAINABILITY

SU.1. Funding

SU.1.1. TechnologyCosts

Whatarethecostsofthetechnology?Howmuchdoesthedeveloper/donoragencypay?Howmuchdoestheschoolpay?Arestudents/familiesresponsibleforanycostsassociatedwiththetechnology?Areallfamiliesabletoaffordthesecosts?Willcostbeadeterrenttoparticipation?Aretherewaystosupportstudentswhosefamiliescan’torwon’tpaythecost?

SU.1.2. TechnologyFunding

Whatdoesthebudgetfortechnologyattheschool-levellooklike?Isthereabudgetconstructedforthetechnology(capitalexpensevs.overhead)?Isthereasustainableplantocontinuefundingthetechnologyoveraperiodoftime?Arethecostspaidannuallyoronsomeothertimescheduleorrandomlyastheyoccur?

SU.1.3 TechnologyReturnonInvestment

Whatarethetrade-offs(intermsofresourceallocation),ifany,ofimplementingthetechnology?Iftheschoolpaysasignificantprice,whataretheycuttingtohavethatmoneyavailablefortechnology?

SU.2 Maintenance&Repairs

SU.2.1. TechnologyMaintenance&Support

Willfrequentmaintenanceandrepairsbeneeded?Whatarethelikelymaintenanceandrepairneeds?Canteachers/students/communitylearntomaintainequipment?Ifnot,issupport/repaireasilyaccessible.Aretherebackupsforwhenthetechnologyfails?

SU.2.2. SupportPlan

Arethereplansandfundingforthenecessarymaintenanceandrepairs?Insurancevs.payingcostsastheyarise?Dedicatedbudgetformaintenance?

17

SU.2.3. ImplementationSupport

Dotheteachersknowhowtoreportproblemsandaccessmaintenancefortheequipment?

SCALABILITY/MARKETIMPACT

SM.1. BroaderCommunityImpact

SM.1.1. KeyStakeholders

Arethereotherstakeholdersforthistechnology(outsideoftheteachers,students,school,anddevelopers)?

SM.1.2. CommunicationPlan

Whatisthecommunicationplanforinformingthestakeholdersaboutthetechnologyandsharingitmorebroadly?

SM.1.3 BestPracticeSharing

Howwillbestpracticesbesharedthroughoutthecommunityusingthetechnology?Doesthedeveloperoradonorhaveanappropriatenetworkorchanneltoshareinformation?

SM.2 Adoption&Scaling

SM.2.1. EconomicBenefits

Arethereeconomicbenefitstousingthistechnology?Tangibleskillsforstudents,teachers,orfacilitatorsthatwouldaidearningpotentialnoworlater?Arethesebasedoncontentknowledgeordigitalliteracy?

SM.2.2. IncentivesforAdoption

Doincentivestoencouragetechnologyadoptionexist?Ifso,whataretheyandhowinfluentialarethey?

SU.2.3. AdoptionMechanisms

Doinformationalstructurestolearnaboutbenefitsandscalethetechnologyexist?

Considerations for Future Application

Torepeatapointmadeinourinitialproblemstatement,thereareanumberofvariablesthatarenotalwaysfullyaccountedforbyvariousdecisionmakersintheeducationalsystem.Theframeworkwasintentionallyconstructedasaseriesofquestionssothatatechnologydeveloper,anadministratororateachermightuseittopromptreflectionastheycontemplatethecreationof,oradoptionofanewtechnologicalintervention.

Thisseconduseoftheframeworkasaself-administeredevaluationwouldlenditselfwelltoanonlineimplementation.Usersloggingintotheframeworkwouldbepromptedtoidentifytherolestheyplayinaneducationalsystem,andthatwouldinturninfluencethenatureandsequenceofthepromptquestionstowhichtheywouldbeexposed.Certainkeyquestionswouldbeusedastriggerstoleadtoadditionalresources,betheyresearchaboutaparticulardomain,orevaluationsofexistingproducts,orresourcesforsolvingparticularimplementationproblemsidentifiedthroughtheprocessofcompletingtheframework.

Thebenefitstotheuserfromaself-administeredframeworkwouldbetwo-fold.Thefirstbenefitwouldderivefromthereflectionitfostersinusers,alertingthemtoissuestheymaynothavefullyanticipatedinadoptinganintervention.Thisbenefitwouldnotbedependentonautomatedresponsesfromtheonlineframework.Nevertheless,thebranchingpathsofanautomatedframeworkwouldmakeitmoreefficienttouse.

Thesecondbenefitwouldresultfromthecomputationalenginebehindtheonlineframework.Weanticipatethatbasedonusers’responses,theframeworkcouldcategorizetheirparticularcontextasbelongingtospecific,familiarlevelsoftechnologicalpreparednessandcapability,andcouldthenpointuserstobothresearchandproductsthathadbeenpreviouslyidentifiedasbeingrelevantforthosecircumstances.Theframeworkcouldn’tnecessarilyevaluatetheappropriatenessofaproposedinterventionifthatinterventionhadnotpreviouslybeendocumentedwithintheframework’sknowledgebase.However,ifusersweretoaddinformationastohowtheyevaluatedagiveninterventionbasedontheframeworkandwithregardtotheircontext,thatinterventionmightbeincorporatedintotheknowledgebaseforthebenefitoffutureusers.

Theframeworkwouldbeadaptabletomultiplelanguages,andcouldbemaintainedbyanetworkofNGOsoracademicinstitutionsaroundtheworld,withonlyamodestinvestmentofeffort.Properlystructured,suchanetworkmightalsofunctionasacommunityofpractice,continuallybuildingouttheframeworkandassociatedresources.Intheprocessthisnetworkwouldbecreatingandsustaininganinteractiveknowledgebase,builtuponsharedfindingsasthetoolisusedwithincreasingfrequencythroughoutthedevelopingworld.

References Davies,R.S.(2011).Understandingtechnologyliteracy:Aframeworkforevaluatingeducational

19

technologyintegration.TechTrends,55(5),45-52.

Whetten,D.&Cameron,K.(2011).DevelopingManagementSkills.PrenticeHall.

Authors & Acknowledgements PrincipalInvestigatorsEricKlopfer&AnkurSarin

ReportAuthorsScotOsterweil,PrateekShah,StaceyAllen,JenniferGroff,SaiPriyaKodidala,andIlanaSchoenfeldThisprojectwouldnothavebeenpossiblewithoutthegenerousassistanceoftheindividualsandNGOswhograciouslyofferedtheirtimeandresourcesaswevisitedtheirsitesormetwiththeminconversation.Itisourhopethattheywillfindthisreportandtheresultingframeworkusefulintheirongoingwork,andthatitwilloffersomerecompenseforthetimetheysokindlygavetous.

WearealsogratefulfortheadviceofourcolleaguesontheCITEproject,andatUSAID.Theirinputwasessentialintheinitialformulationandongoingexecutionofthisproject,andwebenefittedgreatlyfromthemethodologiestheyhavedeveloped,andtheirextensiveexperienceperformingevaluationsinotherdomains.