Download pdf - CIRCL PORTFOLIO MONITORING: 2015 FAST FACTS · CIRCL PORTFOLIO MONITORING: 2015 FAST FACTS ... including forms of social media and forms of publication regarding their work that occured

CIRCLPORTFOLIOMONITORING:2015FASTFACTSCURRENTSTATEOFCYBERLEARNING

InvestmentsincyberlearningresearchanddevelopmentareapriorityattheNationalScienceFoundation(NSF).Thecyberlearningthemeaimstointersectadvancesintechnologieswithprogressmadeintheunderstandingofhowpeoplelearn.SeveraldirectorateshavecontributedtotheCyberlearningandFutureLearningTechnologiesProgramandcyberlearning-relatedawardssince2014(SeeFigure1).TheseincludedtheDirectoratesforComputer&InformationScience&Engineering(CSEorCISE),DirectorateforEngineering(ENG),Education&HumanResources(EHR),andSocial,Behavioral&EconomicSciences(SBE).WithintheseNSFdirectorates,multipleprograms,underdifferentdivisions,havefundedcyberlearningandcyberlearning-relatedawards(SeeFigure2).

0

50

100

150

200

250

300

Cohort1 Cohort2 Cohort3 Cohort4 Cohort5

Num

berofAwards

Figure1:AwardsbyDirectorate

CSE ENG EHR SBE

0

20

40

60

80

100

120

140

160

180

Num

berofAwards

Figure2:CyberlearningandRelatedAwardsbyDivision

ACI

CNS

IIS

IIP

DGE

DRL

DUE

HRD

BCS

SMA

SURVEYS

NSF’sCyberlearninginvestmentthemespansavarietyofprograms,includingtheCyberlearningandFutureLearningTechnologyprogram1.Asofcalendaryear2013theCyberlearningprogramhadawardedover$73,000,000infunds,withanadditional$20,000,000fundedthroughcalendaryear2014.ProjectsfundedbyCyberlearn&FutureTechhavealsoreceivedfundsfromotherprogramsacrossNSF,includingDRK-12,REAL,AISLandTUES.CIRCL2hassentoutfivesurveystothePIsofprojectsfundedbytheCyberlearningprogram(NSFProgramElement:8020)orprojectsthathavebeenlabeledascyberlearning-related(NSFProgramReferenceCode:8244).Thesurveycoveredavarietyoftopics,includingprojectfocus/participants,approachestoresearch,productdevelopment/dissemination,andopportunitiestostrengthentheportfolio.Therewerethreesurveyssentout,onein2014andtwoin2015(SeeTable1).ThefirstsurveywassentouttoCohort1,whichwasthefirstgroupofCyberlearningawardrecipients.ThesecondsurveywassentouttoCohorts2and3,whichwerethefirstgroupofcyberlearning-relatedawardrecipientsandthesecondgroupofCyberlearningawardrecipients,respectively.Thethirdandfinalsurvey,sentouttowardstheendof2015,wassenttoCohorts4and5,whichincludedthesecondgroupofcyberlearning-relatedawardrecipientsandthethirdgroupofCyberlearningawardrecipients.

Table1:SummaryofCohortsCohort Program

Element/CodeNumberofAwards

NumberofSurveyRespondents

%ofAwardeeswhoResponded

%AwardsExpiredasofJune30,2016

1 8020 99 73 74% 43%2 8244 242 105 43% 27%3 8020 47 19 40% 21%4 8244 71 42 59% 3%5 8020 37 26 70% 0%

HIGHLIGHTSFROMSURVEYS

PROJECTDISCIPLINES

Surveyrespondentswereaskedtoselectallofthedisciplinesthattheirprojectsaddressedfromalistofsix

disciplines:Science,Mathematics,Reading,Engineering,ComputerandInformationSciences,SocialSciences,

orStatisticalMethodsorResearchDesign.TheCIRCLportfolio’sstrengthisinitssupportofScienceaswell

asComputerandInformationSciences.TherearealmostnoprojectsthatPIsconsidertoaddressReading.

TARGETEDDEMOGRAPHICSOFPROJECTS

1Theprogramwasstartedin2011byNSFSolicitation14-526.

2TheCenterforInnovativeResearchinCyberlearning(CIRCL)isanNSF-fundedsupportprogramforthebroaderCyberlearningtheme,inpartnershipwithSRIInternational,EducationDevelopmentCenter,Inc.,andtheNationalOpinionResearchCenter(NORC)attheUniversityofChicago.CIRCLaimstobrokercollaborationsacrossthefieldofcyberlearning,inadditiontoconductingportfoliomanagementofcurrentCyberlearning-programfundedandCyberlearning-themedawards.

Whilethequestionsregardingdemographicsservedhaschangedovertheiterations,inthetwomostrecent

versions,arethemostsimilar.Generally,projectscatertocommunitypopulationsbetweengrades6-12,

undergraduates,andeducatorswhoworkwiththeseagegroups.Fewprojectstargetinfantsortoddlers.The

lastfourcohortsindicatethatmanyprojectsaredesignedtoserveunderrepresentedminoritiesor

girls/women.Fewtargettheotherspecialpopulations.Themostcommonsiteofprojectareformalschool

settings;verylittleimplementationoccursatotherpossibletypesoflocales.

PROJECTCLASSIFICATION

TheNationalScienceFoundation(NSF)andtheInstituteofEducationSciences(IES)oftheU.S.Departmentof

EducationjointlycreatedtheCommonGuidelinesforEducationResearchandDevelopmentwhichclassifies

projectsinsixcategories.Projectswereaskedtochoosethecategoryorcategoriesthatbestclassifytheir

work.Foreverycohort,thetwomostcommonclassificationswereearly-stage/exploratoryresearchor

designanddevelopment.

PRODUCTDEVELOPMENT

Projectswereaskedtochoosethetypesofproductsthathaveresultedfromtheirworkin10different

categories.Themostcommonproductdevelopedisaweb-basedserviceoronlineapplications.

FIRSTSURVEY:CYBERLEARNINGANDFUTURELEARNINGTECHNOLOGYPROGRAM2014

In2014,CIRCLsentasurveyto99PIsofprojectsfundedbytheCyberlearningprogram,ofwhich73peopleresponded.

PROJECTDISCIPLINES


disciplines(seeFigure3).

0

15

30

45

60

ResopnsePercent

Figure3:DisciplinesaddressedbyProject

• Sciencewasthemostoftenchosen(53%).Computerandinformationsciencewaschosensecond

mostoften(38%).

• Statisticalmethodsandresearchdesignwastheleastoftenchosen(19%).


Projectswereaskedseveralquestionsregardingparticipantdemographicsincludingtheage,educational

levelandnumberofparticipantsservedbytheirwork(seeFigures4-7).

01020304050

Children Teenagers Adults Families Generalpublic

N/A Other

ResponsePercent

Figure4:TargetedCommunityPopulations

0 5 10 15 20 25 30 35 40 45

Infant/toddlerPreschool/pre-Kindergarten

K-grade2Grades3-5Grades6-8Grades9-12

UndergraduateGraduate

N/AOther

ResponsePercent

Figure5:PrimaryAgeofTargetedLearners

• Projectdesignsmostoftentargetedchildren(38%)andteenagers(45%).

• Projectsareexpectedtoreachlargenumbersofparticipants.Surveyresponsesindicatethatthey

expecttoserveatotalofover127,000participantsthroughtheirprojects.Theoverwhelming

majorityofprojectparticipantsarelearnersofallages.

• Noprojectsaddressedinfantsortoddlers.

60,107

66,280

427

466

126

2061,225

Figure6:NumberofParticipantsServedStudentsorlearnersunderage18 Studentsorlearnersoverage18

K-12teachers Collegeanduniversityfacultyorinstructors

Informaleducators Others

051015202530354045

ResponsePercent

Figure7:LocationofProjectImplementation

• Projectimplementationwasmostcommonlyinmiddleschools(40%)orcolleges/universities

(40%),andtargeting6th-8thgraders(43%).



EducationjointlycreatedtheCommonGuildinesforEducationResearchandDevelopmentwhichclassifies


work(seeFigure8).Projectsoftenspannedmultiplecategories.

• Alargemajorityofprojectswereearly-stage/exploratoryresearch(73%)anddesignand

development(70%).

PROJECTDISSEMINATION

Projectswereaskedtoprovideinformationonthemediumstheyusetodiscussanddisseminatetheirwork,

includingformsofsocialmediaandformsofpublicationregardingtheirworkthatoccuredin2013orwill

occurin2014(seeFigure9).

0 20 40 60 80

FoundationalResearch

Early-StageorExploratoryResearch

DesignandDevelopment

EfhicacyStudy

EffectivenessStudy

Scale-upStudy

ResponsePercent

Figure8:NSFCommonGuidelinesforEducationResearchandDevelopment

• Overhalf(53%)publishedaconferencepaperin2013,and74%plantodosoin2014.13%

publishedajournalarticlein2013and58%plantoin2014.Threerespondents(5%)evenplanto

giveaTedtalkin2014.

PRODUCTDEVELOPMENT


categories.Atleast292productsareinsomestageofdevelopmentincludingatleast31phone/tablet

applications,44productsinonlineservices,and30desktopapplications.

SECONDSURVEY:CYBERLEARNINGANDFUTURELEARNINGTECHNOLOGYPROGRAM2015

The2015surveywassenttoPIsofprojectsfundedbytheCyberlearningprogram(NSFProgramElement:8020)andprojectsthathavebeenlabeledascyberlearning-related(NSFProgramReferenceCode:8244).

CYBERLEARNING-RELATEDSURVEYS(8244)

Ofthe242PIsofcyberlearning-relatedprojects,1053peopleresponded.

PROJECTDISCIPLINES



3Therewere106responses;onewasaPIsubmittingasecond.Asitwasunclearwhichresponsewasthecorrectone,butresponseswereincludedinthedataanalysis.

01020304050607080

ResponsePercent

Figure9:PublicationsaboutProject

Occurredin2013

WillOccurin2014


mostoften(40%).

• Readingwastheleastoftenchosen(4%).


Projectswereaskedseveralquestionsregardingparticipantdemographicsincludingtheage,

educationallevelandnumberofparticipantsservedbytheirwork(seeFigures11-14).

0102030405060

ResponsePercent


05101520253035404550

Responsepercent


0510152025303540

Num

berofrespondents

Figure12:Numberofparticipantsservedforeachtargetedcommunitypopulation

0-50 51-100 101-500 501-1000 Morethan1000

0102030405060

Responsepercent

Figure13:Targetedspecialpopulations

• Projectdesignsmostoftentargetedundergraduatestudents(46%).Thesecondmosttargeted

groupswerestudentsingrades9-12andcollegeoruniversityfacultyorinstructors(bothwere

36%).


• Projectsweremostlikelytotargetunderrepresentedminorities(52%).Theotherthreemost

commonlyselectedcategoriesoftargetpopulationsincluded:learnersinschoolswith50%ormore

studentsreceivingfreeorreducedlunch(37%),women/girls(35%),andlearnersinlow-performing

districtsorschools(33%).

• Projectimplementationwasmostinaformalschoolsetting(76%)orsummerprograms(23%).

Programswereleastlikelytotakeplaceinapreschool(1%).


TheNationalScienceFoundation(NSF)andtheInstituteofEducationSciences(IES)oftheU.S.

DepartmentofEducationjointlycreatedtheCommonGuidelinesforEducationResearchandDevelopment

whichclassifiesprojectsinsixcategories.Projectswereaskedtochoosethecategoryorcategoriesthatbest

classifytheirwork(seeFigure8).Projectsoftenspannedmultiplecategories.

0102030405060708090

ResponsePercent


• Alargemajorityofprojectsweredesignanddevelopment(59%)andearly-stage/exploratory

research(44%).


Projectswereaskedtoprovideinformationonthemediatheyusedtodiscussanddisseminatetheirwork,

andwhetherthesepublicationsoccurredin2014andearly2015,orwouldoccurlaterin2015(seeFigure9).

0 10 20 30 40 50 60 70




EfhicacyStudy

EffectivenessStudy

Scale-upStudy

Noneofthese

ResponsePercent


0

10

20

30

40

50

60

ResposnePercent


Occurredin2014/2015

Willoccurlaterin2015

• 49%ofrespondentspublishedaconferencepaperin2014orearly2015,and54%plannedtodoso

laterin2015.34%publishedajournalarticleinin2014orearly2015and46%plannedtoinlater

2015.

• Onlyonerespondent(3%)hadgivenaTEDtalkpriortothesurvey.

PRODUCTDEVELOPMENT

Projectswereaskedtochoosethetypesofproductsthathaveresultedfromtheirworkin8differentcategories.Atleast237productswereinsomestageofdevelopmentincludingatleast43web-basedservicesoronlineapplication,42curriculum/instructionalmateriale.g.,textbook,and40instruments(e.g.,toassessinterest,engagement).Therewere82productsthatweregoingtobedeveloped.

CYBERLEARNINGPROGRAMSURVEYS

Ofthe47PIsofprojectsfundedbytheCyberlearningprogram,19responded.

PROJECTDISCIPLINES



• Sciencewasthemostoftenchosen(56%).AfterScience,Mathematics,ComputerandInformation

Science,andSocialSciences,wereequallylikelytobeaddressedbytheproject(38%).



Projectswereaskedseveralquestionsregardingparticipantdemographicsincludingtheage,

educationallevelandnumberofparticipantsservedbytheirwork(seeFigures18-21).

0102030405060

ResponsePercent


010203040506070

Responsepercent


024681012

Num

berofrespondents

Figure19:Numberofparticipantsservedforeachtargetedcommunitypopulation

0-50 51-100 101-500 501-1000 Morethan1000

• Projectdesignsmostoftentargetedstudentsingrades6-8(67%).Thesecondmosttargetedgroups

werestudentsingrades9-12andK-12teachersin-service(bothwere47%).

• Noprojectsaddressedinfants/toddlersorpreschool/pre-kindergarten.

• Projectsweremostlikelytotargetunderrepresentedminorities(53%)orgirls/women(53%).

• Noprojectstargetedlearnersinspecialeducationorwithspecialneedssuchaslearningdisabilities.

• Projectimplementationwasmostinaformalschoolsetting(73%)orafterschoolprograms(33%).

Programswereleastlikelytotakeplaceinapreschoolorhomes(0%).


0102030405060

Responsepercent

Figure20:Targetedspecialpopulations

01020304050607080

ResponsePercent







research(60%).

• Noprojectswereclassifiedasscale-up,effectiveness,orefficacystudies.


Projectswereaskedtoprovideinformationonthemediatheyusedtodiscussanddisseminatetheirwork,

andwhetherthesepublicationsoccurredin2014andearly2015,orwouldoccurlaterin2015(seeFigure

23).

0 10 20 30 40 50 60 70




EfhicacyStudy

EffectivenessStudy

Scale-upStudy

Noneofthese

ResponsePercent


• 47%ofrespondentspublishedaconferencepaperin2014orearly2015,and53%plannedtodoso

laterin2015.8%publishedajournalarticleinin2014orearly2015and31%plannedtoinlater

2015.

• NoonehadgivenorplannedtogiveaTEDtalk.

PRODUCTDEVELOPMENT

Projectswereaskedtochoosethetypesofproductsthathaveresultedfromtheirworkin8differentcategories.Atleast37productswereinsomestageofdevelopment.Therewere13productsthatweregoingtobedeveloped.

THIRDSURVEY:CYBERLEARNINGANDFUTURELEARNINGTECHNOLOGYPROGRAM,LATE2015

CYBERLEARNING-RELATEDSURVEYS

Ofthe71PIsofcyberlearning-relatedprojects,42peopleresponded

PROJECTDISCIPLINES



0

10

20

30

40

50

60

ResposnePercent


Occurredin2014/2015

Willoccurlaterin2015


mostoften(43%).





010203040506070

ResponsePercent

Figure24:DisciplinesAddressedbyProjects

010203040506070

ResponsePercent


0510152025

Num

berofRespondents

Figure26:NumberofParticipantsServedforEachTargetedPopulation

0-50 51-100 101-500 501-1000 Morethan1000

05101520253035404550

ResponsePercent

Figure27:TargetSpecialPopulations

• Projectdesignsmostoftentargetedundergraduatestudents(58%).Thesecondmosttargeted

groupswerecollegeoruniversityfacultyorinstructors(47%).

• Noprojectsaddressedinfantsortoddlers,preschool/kindergarten,orgradesK-2.

• Projectsweremostlikelytotargetunderrepresentedminorities(33%).44%ofparticipants

indicatedthattheydidnottargetspecialpopulationswiththeirdesign(indicatedN/Aforquestion).

• Projectimplementationwasmostinaformalschoolsetting(69%).






01020304050607080

ResponsePercent

Figure28:LocationofProgramImplementation


research(44%).

PRODUCTDEVELOPMENT


categories(seeFigure30).

• Atleast144productsweregoingtobedevelopedorhadalreadybeendeveloped.• Themostcommonlydevelopedproducttypewasweb-basedservicesoronlineapplications(26

respondents).Oftherespondents,81%plannedtoorhadalreadydevelopedthistypeofproduct.

0 10 20 30 40 50 60 70




EfhicacyStudy

EffectivenessStudy

Scale-upStudy

Noneofthese

ResponsePercent


0%10%20%30%40%50%60%70%80%90%100%

Figure30:ProductDevelopment

Tobedevelopedoralreadydeveloped Notapplicable

CYBERLEARNINGPROGRAMAWARDS

Ofthe37PIswhoreceivedCyberlearningprogramawards,26peopleresponded

PROJECTDISCIPLINES



• Socialsciencewasthemostoftenchosen(48%).Computerandinformationsciencewaschosen

secondmostoften(40%).





0102030405060

ResponsePercent

Figure31:DisciplinesAddressedbyProjects

0102030405060

ResponsePercent


024681012

Num

berofRespondents

Figure33:NumberofParticipantsServedforEachTargetedPopulation

0-50 51-100 101-500 501-1000 Morethan1000

• Projectdesignsmostoftentargetedstudentsingrades6-8(50%).Thesecondmosttargetedgroups

wereundergraduateandcollegeoruniversityfacultyorinstructors(33%).


• Projectsweremostlikelytotargetunderrepresentedminoritiesorwomen/girls(36%).36%of

participantsindicatedthattheydidnottargetspecialpopulationswiththeirdesign(indicatedN/A

forquestion).

• Projectimplementationwasmostinaformalschoolsetting(58%).

• Noprojectswereimplementedatpre-schools.

0510152025303540

ResponsePercent

Figure34:TargetSpecialPopulations

010203040506070

ResponsePercent

Figure35:LocationofProgramImplementation



EducationjointlycreatedtheCommonGuidelinesforEducationResearchandDevelopmentwhichclassifies


work(seeFigure36).Projectsoftenspannedmultiplecategories.

• Alargemajorityofprojectswereearly-stage/exploratoryresearch(71%)anddesign-development

(42%).

PRODUCTDEVELOPMENT


categories(seeFigure37).

0 10 20 30 40 50 60 70 80




EfhicacyStudy

EffectivenessStudy

Scale-upStudy

Noneofthese

ResponsePercent


• Atleast71productsweregoingtobedevelopedorhadalreadybeendeveloped.• Themostcommonlydevelopedproducttypewasweb-basedservicesoronlineapplications(16

respondents).Oftherespondents,70%plannedtoorhadalreadydevelopedthistypeofproduct.

0%10%20%30%40%50%60%70%80%90%100%

Figure37:ProductDevelopment

Tobedevelopedoralreadydeveloped Notapplicable