6
This evaluation brief provides a summary of the first implementation of two Massively Open Online Courses for Educators (MOOC-Eds) developed with support from the Hewlett Foundation. Fraction Foundations and Disciplinary Literacy for Deeper Learning are designed to help educators better understand math and literacy core academic content respectively. By modeling and supporting instructional practices such as inquiry-based learning and reflection with peers, they are also designed to promote collaboration, critical thinking, and problem solving in the classroom. Sections in this brief are directly aligned to the evaluation questions outlined in the grant proposal. They address the impact on educators’ understanding of content and pedagogy, as well as their effectiveness in supporting changes in classroom instruction. Initial findings suggest that MOOC-Eds have indeed been effective in supporting the professional growth of a relatively small (in MOOC terms) and demographically diverse group of educators. Specifically, the vast majority of educators report that, as a result of their participation, they have deepened their understanding of fractions and literacy instruction, and have applied new knowledge and skills to make positive changes to their professional practice. To improve future MOOC-Eds, participants recommended improvements to the discussion forums, more models of practice, and greater differentiation. Major findings reported in this brief are drawn from registration data, surveys, site analytics, interviews, and a pre-post math assessment. These data sources are described at the conclusion of this report and links are provided to online reports for readers interested in a more detailed breakdown of the data. 2 The Friday Institute for Educational Innovation College of Education North Carolina State University Prepared for the Hewlett Foundation by: Shaun Kellogg, PhD Sherry Booth, PhD Isaac Thompson, MS Tamar Avineri, MA Hewlett Foundation Interim Report February 2015 Contents page Summary Participation Participant Demographics, Motivations, and Engagement 3 Educator Impact Professional Goals, Knowledge & Skills, Application to Practice 4 Design Effectiveness Practice-Based, Peer Supported, Self-Directed Learning Data Sources & Analyses Registration, Analytics, MKT, Surveys, & Interviews 6 Recommendations Forum Design, Peer Engagement, Models of Practice, and Differentiation 5

Hewlett Interim Report (Final) - Nc State University · Hewlett Foundation Interim Report February 2015 3 Impact on Educators Educators reported courses were effective in supporting

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Hewlett Interim Report (Final) - Nc State University · Hewlett Foundation Interim Report February 2015 3 Impact on Educators Educators reported courses were effective in supporting

ThisevaluationbriefprovidesasummaryofthefirstimplementationoftwoMassivelyOpenOnlineCoursesforEducators(MOOC-Eds)developedwithsupportfromtheHewlettFoundation.FractionFoundationsandDisciplinaryLiteracyforDeeperLearningaredesignedtohelpeducatorsbetterunderstandmathandliteracycoreacademiccontentrespectively.Bymodelingandsupportinginstructionalpracticessuchasinquiry-basedlearningandreflectionwithpeers,theyarealsodesignedtopromotecollaboration,criticalthinking,andproblemsolvingintheclassroom.

Sectionsinthisbriefaredirectlyalignedtotheevaluationquestionsoutlinedinthegrantproposal.Theyaddresstheimpactoneducators’understandingofcontentandpedagogy,aswellastheireffectivenessinsupportingchangesinclassroominstruction.InitialfindingssuggestthatMOOC-Edshaveindeedbeeneffectiveinsupportingtheprofessionalgrowthofarelativelysmall(inMOOCterms)anddemographicallydiversegroupofeducators.Specifically,thevastmajorityofeducatorsreportthat,asaresultoftheirparticipation,theyhavedeepenedtheirunderstandingoffractionsandliteracyinstruction,andhaveappliednewknowledgeandskillstomakepositivechangestotheirprofessionalpractice.ToimprovefutureMOOC-Eds,participantsrecommendedimprovementstothediscussionforums,moremodelsofpractice,andgreaterdifferentiation.

Majorfindingsreportedinthisbriefaredrawnfromregistrationdata,surveys,siteanalytics,interviews,andapre-postmathassessment.Thesedatasourcesaredescribedattheconclusionofthisreportandlinksareprovidedtoonlinereportsforreadersinterestedinamoredetailedbreakdownofthedata.

2

TheFridayInstituteforEducationalInnovation

CollegeofEducation

NorthCarolinaStateUniversity

PreparedfortheHewlettFoundationby:

ShaunKellogg,PhDSherryBooth,PhDIsaacThompson,MSTamarAvineri,MA

HewlettFoundationInterimReportFebruary2015

Contentspage SummaryParticipationParticipantDemographics,Motivations,andEngagement

3EducatorImpactProfessionalGoals,Knowledge&Skills,ApplicationtoPractice

4DesignEffectivenessPractice-Based,PeerSupported,Self-DirectedLearning

DataSources&AnalysesRegistration,Analytics,MKT,Surveys,&Interviews

6

RecommendationsForumDesign,PeerEngagement,ModelsofPractice,andDifferentiation

5

Page 2: Hewlett Interim Report (Final) - Nc State University · Hewlett Foundation Interim Report February 2015 3 Impact on Educators Educators reported courses were effective in supporting

HewlettFoundationInterimReportFebruary2015

2

ParticipationMOOC-EdsappealedtobothU.S.andinternationaleducators.InFallof2014,FractionFoundationsandDeeperLearninginDisciplinary

Literacyenrolled808and659educatorsrespectively.Internationally,participantsfromthetwocoursesrepresentedover40countries,withthebulkofparticipants(90%)residingintheUnitedStates.Nationally,educatorsrepresented47ofthe50UnitedStatesregisteredforthecoursewithNorthCarolina,California,andGeorgiaasthetopthreestates.Amongthoseregistered,74%fromDisciplinaryLiteracyand80%fromFractionFoundationsactuallyparticipatedinthecoursebyaccessingoneormoreunitsafterthecoursesopened.Educatorsrepresentedvariousroles,andindicatedawiderangeofgoals.Participantswereaskedduringregistrationtoindicatetheirprimaryroleineducation.Themajorityofparticipantswereworkingprofessionals,andconsistedofeducatorsinthefollowingroles:ClassroomTeachers(50%),CurriculumSpecialists(15%),ProfessionalDevelopmentFacilitators(11%)andSpecialEducators(6%).Participantswerealsoaskedtoselectuptothreeprofessionalgoalstheyhopedtoachieveasaresultsoftheirparticipationinthecourse.Themostfrequentlyselectedwere:acquiringnewknowledge/skills(28%),collectingnewresources/tools(21%),andengaginginfunandinspiringactivities(15%).Asaccessbyusersdeclinedovertime,engagementwithcourseincreased.Duringthefirstweekoftheeachcourse,roughlyhalfofregisteredparticipantsaccessedUnit1.FollowingapatterntypicalofMOOCsingeneral,thenumberofparticipantsaccessingthecoursedeclinedovertime.Asthecourseprogressed,however,therewasagradualincreaseinengagementasmeasuredbytheaveragenumberofactionspereducator(e.g.videoviews,discussionposts,resourceaccess).Moreover,educatorsaccessedthecourseforseveralweeksbeyondthefinalweek.Bycourseend,7%ofFractionparticipantsand8%ofLiteracyparticipantssuccessfullycompletedallcourserequirmentsandrequestedacertificatetoobtainCEUs.Uponregistration,only3%hadindicatedthisasaprimarygoal.

IllustrativeQuotesProfessionalRoles“Iamexcitedtogetstartedasthisismy

firstMOOC-Edcourse!Iteach5thgrade

atablendedonlineschool.I'malways

lookingfornewinstructionalstrategies

toteachmath.”

-FractionFoundations

”Hi,mynameisGitafromIndonesia.IwasauniversitylecturerinIndonesia…

Mygoalistodigfurtherintowhat

disciplinaryliteracyapplicationislikein

arealsetting”

-DisciplinaryLiteracy

“IamanAssistantSuperintendentinan

elementaryK-8district.Iworkwith

administratorsandteacherstosupport

teachingandlearningforallstudents.”

-FractionFoundations

PersonalMotivation“Ihavebeenteachingbiology.Honestly,

Idon'tincludeenoughreadingor

writingintomyclasses.However,I

definitelyrecognizetheliteracyskills

thatmostofmystudentsarelacking,I

amheretolearnnewwaystoexpand

myteachingtoincludemoreliteracy.”

-DisciplinaryLiteracy

“Iwasveryinterestedinthiscourseas

ourstudentsstrugglewithfractionsat

alllevels.AsamathcoachIfeltthis

wouldaddmoretoolsinmybeltto

assistouruppergradeteachersinre-

teachingfractionswithmore

conceptualunderstandingratherthan

anothersetofprocedurestomemorize.”

-FractionFoundations

“MyteamandIhavebeenworkingwith

theCommonCoreandwe’relookingfor

materialsthatwillbestmeetthose

standards...Icouldseeveryclearlyhow

DisciplinaryLiteracyandtheCCSSELA

worktogetherwell.Ourteamis

definitelylookingforwaystobringboth

oftheseintoourteaching…Sincethe

CommonCoreissonewtous,we're

strugglingabit.”

-DisciplinaryLiteracy

0%

20%

40%

60%

Fractions Litearcy

353

78

29790

0

100

200

300

400

EducatorsAccessingtheMOOCperWeek

%RegistrantsbyRole

9

158

16

0

10

20

30

AverageEducatorActionsperWeek

Page 3: Hewlett Interim Report (Final) - Nc State University · Hewlett Foundation Interim Report February 2015 3 Impact on Educators Educators reported courses were effective in supporting

February2015HewlettFoundationInterimReport

3

ImpactonEducators

Educatorsreportedcourseswereeffectiveinsupportingtheirlearninggoals.OnUnitFeedbackForms,90%(693)ofeducatorresponsesagreedthatcourseunitshelpedthemprogresstowardtheirpersonallearninggoals,ofwhich76%“agreed”or“stronglyagreed,”and14%“somewhatagreed.”OnEnd-of-Coursesurveys,90%(97)ofeducatorsalsoagreedthat,asawhole,MOOC-Edswereeffectiveinsupportingtheirpersonaland/orprofessionallearninggoals,with84%reportingthemas“effective”or“veryeffective”and6%as“somewhateffective.”Educatorsreportedimprovedunderstandinganddevelopmentofnewskills.Educatorsfrequentlyreported,bothininterviewsandonopen-endedsurveyitems,thatthecontentandactivitieshelpedthem“grow”,“develop”,and“deepen”theirunderstanding,particularlyaroundstudents’commonmisconceptionsaboutfractionsandtheuseofinquiry-basedlearningtosupportliteracy.OnUnitFeedbackForms,90%ofeducatorsagreedthatcourseunitshelpeddeepentheirunderstandingofthecontent.Beyondself-reportmeasures,Fractionparticipantswereadminsteredapre-andpost-courseassessmentadaptedformtheMathematicalKnowledgeforTeaching(MKT)assessment.Preliminaryresultssuggestaslightimprovementinthepercentofcorrectresponses(+3.3%)acrossallitems,withthegreatestgains(+6.9%)onitemsrelatedtofair-sharingconcepts.Amoredetailedstatisticalanalysiswillbeprovidedinthesummativereport.Educatorsreportedapplyingcoursestrategies,projects,andcontentintheirprofessionalpractice.Whenaskedonend-of-coursesurveysiftheyhadattemptedtomakechangesintheirprofessionalpracticeasaresultofparticipation,97%ofeducatorsanswered“Yes.”Duringinterviewsandonopen-endeditems,educatorsoftencitedusinginstructionalstrategiesandcoursecontentfromtheMOOC-Ed,andincorporatinglessonsandprojectstheydeveloped.InLiteracy,educatorsnotedapplyingclosereadingstrategieswithstudentsandincreaseduseoftechnologytosupportinquiry.Participantsalsoreportedimplementingorplanningtoimplementtheinquiry-basedlessonstheydesigned.InFractionFoundations,participantsoftenreportedtheyarenowallowingstudentsmoretimetoexploreproblemswithoutrushingtothealgorithm,takingtimetoexplainthe“why”behindalgorithms,andusing“interviews”tobetterunderstandstudentthinkingandadjustinstruction.Manyindicatedusingfairsharingandnumberlineactivitiesfromthecoursewiththeirstudents,andemployinggreateruseofmanipulativesandfractionmodelsintroducedinthecourse.Finally,professionaldevelopmentleadersnotedusingideasandtoolsfromthecourse,andthattheexperiencehasmadethemmoreeffectiveinsupportingtheirteachers.

IllustrativeQuotes

KnowledgeandSkills“Ihavelearnedtobetterlistentohow

studentsthinkaboutfractionproblems,

andIammoreconfidentinbeingableto

helpstudentstounderstandfractions.”

-FractionFoundations

Ihavebecomemuchmoremindfulof

whatInternetresourcesIusewiththe

studentswithaviewtoincreasingclose

readingpractices”-DisciplinaryLiteracy

“Thiscoursehashelpedmehonemyconceptualskillsandhelpedmefocuson

whatisimportantforteacherstoknow

intheirupcomingprofessional

development.”

-FractionFoundations

LearningGoals“Ifoundthatusingthevideosandthe

coursereadingsweremosthelpfulwith

mypersonalandprofessionallearning

goals.Iwasabletotakealotofthis

informationbackanduseitinmy

classroom…Itwasveryhelpfulwithmy

growthasateacher.”

-DisciplinaryLiteracy

ImpactonPractice“Ihavechangedaboringoldhistorical

fictionunitintoaninquiry-basedunit

revolvingaroundheroes.”

-DisciplinaryLiteracy

“Ihavebeguntofacilitatelearningof

fractionsversusteachingthestudents

aboutfractions.Inowhavestudents

taketheirtimeandexploreconceptsin

differentwaysratherthanrushing

throughtoteachanalgorithm.

-FractionFoundations

“Iconductedatrainingon‘whatis

disciplinaryliteracy’andwealsodid

‘makeandtakes’forpre-reading

strategies,vocabularystrategiesand

during-readingstrategies.Iwilladdress

otherinformationlearnedfromthis

courseatfutureofferingsandthrough

coaching.”

-DisciplinaryLiteracy

94%

93%

86%

90%

helpedmeprogresstowardsmypersonal

learninggoals

deepenedmyunderstandingofthetopic(s)addressed

Fractions Literacy

%EducatorsWhoAgreedCourseUnits…

97%

3%Yes No

Haveyouattemptedtomakechangesinyourpractice?

%CorrectonMKTAssessmentItems

64% 67%67%

74%

Pre-Test Post-Test

AllItems Fair-Sharing

Page 4: Hewlett Interim Report (Final) - Nc State University · Hewlett Foundation Interim Report February 2015 3 Impact on Educators Educators reported courses were effective in supporting

HewlettFoundationInterimReportFebruary2015

4

DesignEffectivenessOpportunitiesforpeercollaborationanddiscussionsupportedtheirlearning.MOOC-Edsaredesignedtofosteracollaborativecommunityoflearners.Discussionforumsaretheprimarytoolforsupportingthisgoalandwereusedtoencourageparticipantstoshareideas,askquestions,andprovideconstructivefeedback.Intheforums,44%(214)ofLiteracyparticipantsand47%(304)ofFractionparticipantscontributedacombined4,355postingsandcommentstotheforums,bothduringandbeyondtheendofthecourse.Onaverage,participantscontributed8.4postsorreplies,andreceivedresponsesfrom3differentpeers.Whenaskedifthediscussionspromotedconstructivepeerinteraction,84%indicatedpositiveagreement,including66%who“agreed”or“stronglyagreed”and18%who“somewhatagreed.”Discussionswithpeerswereoftencitedduringinterviewsandsurveysasoneofthemostvaluableaspectsofthecourse.Specifically,educatorsappreciatedthemultipleperspectivesshared,“hearingfromotherteachers”abouttheirexperienceandpractice,andnetworkingwithotherprofessionalsinsimilarroles.Educatorscitedvideos,resourcesandprojectsasvaluableforsupportingtheirprofessionalpractice.MOOC-edsaredesignedtosupportthedirectapplicationofcontenttopractice.WhenaskedonUnitFeedbackFormsiftheyagreedthecoursewasachievingthisend,91%indicatedpositiveagreement.OnEnd-of-CourseSurveys,participantswerealsoaskedhoweffectivethecoursewasinpreparingthemtomakechangestotheirprofessionalpractice;96%ofparticipantsrespondedpositively.Educatorsspecificallycitedthevalueofthevideos,thecoreresources,andcourseprojects,andhowthesecoursecomponents,aswellastheMOOC-Edexperienceasawhole,contributedtotheirprofessionalgrowth.EducatorsreportedMOOC-Edsarehelpingthemself-directtheirlearning.MOOC-Edsaredesignedtoalloweducatorstotailortheexperiencetotheirprofessionalneedsandinterests,aswellashelpparticipantsgaugeprogresstowardstheirpersonallearninggoals.Atcourseend,over90%ofsurveyparticipantsagreedthattheMOOC-Ed1)enabledthemtopersonalizetheirlearningthroughdifferentiatedresourcesandactivities,2)providedopportunitiestoinvestigateself-identifiedproblemsorareasofinterests,and3)providedactivitiesand/orresources(e.g.self-assessments,peerassessment,feedback)thathelpedthemgaugetheirlearning.

IllustrativeQuotes

SupportingPractice“Themostvaluableaspectofthecourse

formewastheexpertpanelsvideos.

BecauseofwhereweareinMontana,

wejustdon’thearpeople,leading,

cuttingedgepeople,orleadersin

mathematicseducation,sotobeableto

heartheexpertpanelwasreallygood.

-FractionFoundations

“Themostvaluableaspectofthecourse

wasdevelopingmyInquiry-Based

DisciplinaryLiteracylesson,whichI'll

beabletobringbacktomyPLCand

sharewithmycolleagues,whowill

hopefullytweakitandmakeitareally

greatandapplicableproject!”

-DisciplinaryLiteracy

PeerSupport“Themostvaluableaspectofthis

MOOC-Edwaslearningaboutthevalue

ofonlinecoursesandhoweffectivethey

aretoourprofessionallearning

communitiesaswellasbeingableto

makeauthenticconnectionswith

educatorsfromaroundtheworld.”

-DisciplinaryLiteracy

“Iappreciatedreadingthepostingsof

othereducators.Ialsofoundmyself

askingothereducatorshowtheywere

addressingtheneedsoftheirstudents.I

learnedIwasn'ttheonlyteacher

searchingforawaytomakethe

conceptsofcomparingfractionseasier

fortheResourceandELLstudentsto

understand.”

-FractionFoundations

“ThePLCquestionsfordiscussionwere

great-itreallysparkedsomenice

discussionwithinmycohort.”

-DisciplinaryLiteracy

Self-DirectedLearning“Ihaveaclearersenseofwhatthe

expectationsareformypersonal

learninggoalwhilecontinuingto

deepenmyunderstandingof

disciplinaryliteracy.”-DisciplinaryLiteracy

1 2 3 4 5 6 7

Assessments

Forums&Chats

PractitionerTools

CourseReadings

CourseProject

CourseVideos

1=VeryIneffective7=VeryEffective

Avg.RatingofDesignComponentEffectiveness

Fractions Literacy

28%

33%

39%

48%

28%

15%

Literacy

Fractions

VeryEffective Effective

SomewhatEffective NeutralorNegative

HoweffectivewastheMOOCinsupportingpositivechangestoyourpractice?

Page 5: Hewlett Interim Report (Final) - Nc State University · Hewlett Foundation Interim Report February 2015 3 Impact on Educators Educators reported courses were effective in supporting

February2015HewlettFoundationInterimReport

5

ParticipantRecommendationsImprovetheusabilityofthediscussionforums.Whenaskedtoproviderecommendationsforimprovingtheunitorcourse,participantsfrequentlyexpressedfrustrationwiththedesignandlayoutofthediscussionforums.Participantsreportedfeeling“overwhelmed”bythesheervolumeofpostings,and“confused”innavigatingtheforums.Specifically,participantsnoteddifficultiesingettingbacktoparticularconversationsandlocatingrepliestoposts.Theyalsofrequentlynoteddifficultiesinlocatingothereducatorsinsimilarprofessionalrolesandadesiretodiscusswithothersinthesamediscipline.Participantsrecommendedprovidingadditionalsupportinthebeginningsuchasvideosortutorialstohelpbetternavigatethediscussionforums,aswellasbetterwaystofollowdiscussionsandconnectwithpeersinsimilarroles.Promotedeeperengagementbetweeneducators.Specifically,participantsrecommendedthatconsiderationbegiventogroupingpeopleindifferentwaysandfindingwaystoincreaseinteractionwithingroups.Indiscussinghowtomakethediscussionsandgroupsmoreinteractive,participantssuggestedalternategroupingmethodssuchasgroupsbasedoneducationalrolesorformingsmaller“workteams”forthoseinterestedandcommittedtofullyparticipating.Additionally,participantsindicatedthatitwouldbehelpfulifexpectationsforengagementinthediscussionforumsweremademoreexplicitandadditionalguidelinesand/orencouragementforengagementintheforumswereprovided.Forexample,oneparticipantsuggestedemphasizingtheimportanceofengaginginthediscussionsinatimelymannersoasnottodelayordisruptthemomentumoftheconversations.Anothersuggestedthatparticipantsmightneedmoreencouragementforcommentingonpostsratherthansimplyviewingthem,asmanypostsintheforumshadahighnumberofviews,butnocommentsinresponsetotheirpost.Includemoremodelsandconcreteexamplesofeffectivepractice.ParticipantscalledformorepracticalexamplesofhowprinciplesorconceptspresentedinMOOC-Edsmightbeappliedinclassroomsoreducationalsettings.Specifically,someparticipantsinFractionFoundationsfeltthattheywerenotprovided“practicaltips”orstrategiesforaddressingstudentmisconceptions.Participantsalsocalledformoreexamplesofcompletedcourseprojects,aswellasmorerealisticexamplesthantheonesprovided.OneparticipantnotedthattheInquiry-BasedDisciplinaryLiteracy(IDL)projectexamplesfelt“overdone”intermsoftechnologyintegrationand“didnotrepresenttheteachingcontextofmostpublicschooleducators.”Finally,participantssuggestedmorevideosofcurrentclassroomteachersdemonstratingthestrategiesorconceptsintroduced.Includeresourcesformorespecificcontexts,andhelpfilterexisitngones.Toalesserextent,participantsdesiredresourcesthatweremorespecifictotheirgradelevelandsubjectarea,ormoregearedtowardstheirprofessionalrole.InDiscplinaryLitearcy,forexample,oneteachernotedthatreadingsandvideosweresometimes“tootechnical,”whilecoachesandadministratorsrequestedprojectsmorespecifictotheirrole.InFractions,someparticipantsindicatedthatsomeoftheresourcesweretoobroadandwouldhavelikedsomemorespecifictheirgradelevel.However,severalparticipantsfelt“overwhelemed”withtheamountofresourcesalreadyavailableandsuggestedofferingfeweroptions,orbetterwaystofilterthem.

IllustrativeQuotes

PromotingEngagement“Manydiscussionshavenoresponses

despiteseveralviewingsandthatisnot

helpfultothepersonwhoauthoredthe

discussion…mostofthe[peer]feedback

onprojectswasverysuperficial.”

-FractionFoundations“Ithinkthatmaybewecanmakegroups

accordingtotheinterestsordiscipline…

asamiddleschoolspecialed.and

generaled.mathteacherIfoundit

difficulttofindotherstotalkwithwhom

teachsimilarstudents.”

-DisciplinaryLiteracy

ForumDesign“Somehow,thediscussionswere

overwhelmingforme.Iagreethatwe

needthoseforumsbutthereisaLOTof

informationwiththeamountofpeople

takingtheclass…”

-FractionFoundations

“Ilikedthediscussionboardsbeing

brokendownbystate,butfoundit

difficulttofindteachersteachingthe

samegrade/similarclasssettingsto

havediscussionswith.Itwouldbenice

tofocusinonmiddleschoolspecial

education.”

-FractionFoundations

ModelsofPractice“Ifeltlessconfidentinteachingfractions

aftertakingthiscoursethanwhenI

startedbecause…IdidnotfeelIwas

givenpracticaltipsofhowtoteachto

solvetheproblems.Iwouldlikemore

specificbreakdownofhowtoteacheach

concept.”

-FractionFoundations

Differentiation“Therearealotofresources...fora

coachitwasdifficulttodecidewhich

onestoread.Somaybeasetofresources

specificallyforcoaches…andresources

forhelpingadministratorscoach

teachersthroughthedevelopmentofIDL

intheirclassrooms.”

-DisciplinaryLiteracy

Page 6: Hewlett Interim Report (Final) - Nc State University · Hewlett Foundation Interim Report February 2015 3 Impact on Educators Educators reported courses were effective in supporting

HewlettFoundationInterimReportFebruary2015

6

TechnicalBriefMOOC-EdRegistrationAllparticipantscompletearegistrationformforeachMOOC-Edcourse.Theregistrationformconsistsofself-reporteddemographicdata,includinginformationonprofessionalrolesandworksettings,yearofexperience,gender,levelofeducation,andpersonallearninggoals.

GoogleCourseBuilder&VanillaForumsAnalyticsCourseBuilderandVanillaforumanalyticsincludeusereventlogdatausedtotrackindetaileachparticipant’sengagementwiththeunits,unitlessons,lessonresources,anddiscussionviewsandposts.Acombinationofapplicationswereusedtoclean,merge,andformatthedataintousabletables.TheseapplicationsaccessthedatarepositorydirectlytoproducedownloadableExceltables.Tableau,Excel,andNodeXLwereusedtocalculatebasicdescriptivestatisticsandvisualizations.

UnitFeedbackFormsUnitFeedbackformswereembeddedattheendofeachunitinbothMOOC-Eds.Thesesshortformsconsistedoffiveclose-endeditemsalignedtotheevaluationquestionsandtwoopen-endeditemsrequestingparticipantsdescribethemostvaluableaspectoftheunitandrecommendationsforimprovingtheunit.Overall,709embeddedsurveyswererecordedby305uniqueusers.Analysisofsurveyresultsconsistedofitem-leveldescriptivestatisticsusingExcelsoftwareandthematicanalysisoftheopen-endedresponsesusingAtlas.tisoftware.

End-of-CourseSurveysAtthecompletionofeachMOOC-Ed,participantswereaskedtocompleteandendofcoursesurvey.Completionwasarequisiteforparticipantsrequestingacertificate.Thesurveyconsistedofroughly30closed-endeditemsand4open-endeditemsdesignedtosolicitparticipants’perceptionsoftheimpactandeffectivenessoftheMOOC-Eds.Theysurveywascompletedby48Literacyparticipantsand55Fractionparticipants.Analysisofsurveyresultsconsistedofitem-leveldescriptivestatisticsusingExcelsoftwareandthematicanalysisoftheopen-endedresponsesusingAtlas.tisoftware.

FractionsPre-PostAssessmentTheFractionFoundationspre-postassessmentwasadaptedfromtheMathematicalKnowledgeforTeaching(MKT)assessmentdevelopedattheUniversityofMichigan.Itemsincludedinthepre-postassessmentswereselectedfromMKTitemsthatalignedwiththethreemajortopicscoveredintheMOOC-Ed:Fractionsinfair-sharingcontexts;fractionasmeasure;andunderstandingoperationswithfractions.BasicdescriptiveswerecalculatedinSPSSandfurtherstatisticalanalysesareplannedforthesummativereport.

ParticipantInterviewsFollowingamid-courseemailsolicitation,arepresentativesampleof11participants(5and6participantsfromtheLiteracyandFractionMOOC-Edsrespectively)wereselectedbasedontheireducationalroleandextentofengagement.Asemi-structuredphoneinterview(30-60min)wasdesignedtogatherfeedbackregardingmotivations,perceivedvalue,recommendations,andapplicationtopractice.Audiorecordingsweretranscribedandopen-codedbyoneTeammemberusingAtlas.tisoftwarewhothenconsolidatedquotesbythemes.

DataLinks

Demographics&SiteAnalytics:go.ncsu.edu/hewlett_analytics

UnitFeedbackSurveys:go.ncsu.edu/hewlett_unit_all

go.ncsu.edu/hewlett_unit_fractionsgo.ncsu.edu/hewlett_unit_literacy

End-of-CourseSurveys:go.ncsu.edu/hewlett_eoc_all

go.ncsu.edu/hewlett_eoc_fractionsgo.ncsu.edu/hewlett_eoc_literacy