Download pdf - JMU Assessment Progress Template Interdisciplinary Liberal ...1 JMU Assessment Progress Template Interdisciplinary Liberal Studies (IdLS) - BS PART I. Objectives Description of process

1

JMUAssessmentProgressTemplate

InterdisciplinaryLiberalStudies(IdLS)-BS

PARTI.Objectives

Descriptionofprocessfordevelopingobjectives:Morethanadecadeago,asmallgroupofadministratorsassembled31minutelydetailedStudentLearningObjectives.ThesewerelargelydefinedbytheVirginiaSOL’sandteacherlicensurecompetenciesineachofthemajorsubjectareas.Whilefairlydetailed,theseobjectiveswerelargelyunassessable.Assuch,arecommendationwasmadeinthe2008-2009IdLSAssessmentProgressTemplatetorevisethelistofIdLSGoalsandObjectives.InSpringof2010,representativesfromIdLSmetwithDr.KestonFulcherfromCARSanddiscussedthedevelopmentofnew,assessable,programGoalsandObjectivesthatfulfilltheIdLSMission.TheMissionofIdLSis:

• Tosupporttheuniversity’smissiontoproduceeducatedandenlightenedcitizens.• Tohelpstudentsembracewisdom,inspirelearning,andenhanceliving.• TomeetVirginiateachercompetenciesbyprovidingbreadthandintegrationacrossthecontent

areasofEnglishandlanguagearts,history,socialsciences,mathematics,naturalsciences,andtechnology.

• ToworkcollaborativelywiththeEducationUnittoreachitsgoalsasarticulatedinitsConceptualFramework,particularlyastheyrelatetodevelopingadeepunderstandingofcontent.

Startingwiththe2009-2010APTthefollowinglearningobjectivesserveasAPTProgramGoalsandObjectivesfortheIdLSprogram*.

Goals Objectives MeasuresandRationale

IdLSstudentscompletingGeneralEducationwilldemonstrateknowledgecentraltotheuniversity’smission,andrelevanttotheVirginiateachercompetencies.

IdLSstudentswill,asagroup,matchotherJMUstudentsonGeneralEducationlearningoutcomes,specificallyintechnology,informationliteracy,scientificreasoning,quantitativereasoning,andtheglobalandAmericanexperiences.

IdLSvsnon-IdLSdatafromClusters1(MREST*test),3(ScientificReasoning&QuantitativeReasoningsub-scales),and4(Global&AmericanExperiencetests).

GeneralEducationisthebaseuponwhichJMUaimstofulfillitsmissiontoproduceeducatedandenlightenedcitizens,andthespecificcontentareasmeasuredareteachercompetenciesrequiredbytheVirginiaDepartmentofEducation.

IdLSseniorswilldemonstratecontent-areaproficiencyonteacherlicensure

ForallIdLS-relatededucationprogramshavingPRAXISIIcontent-arealicensureexams(i.e.,ElEdandMIED),eachprogram

PRAXISIIscoresandpassrates,reportedbyeducationprogram.

Content-areaproficiencyismeasuredbyPRAXISIItests,andan80%passrateon

2

exams. willhaveatleastan80%passrateonthoseexams.

PRAXISIIisaminimumpassraterequiredbyNCATEforaccreditation.

ForELEDandIECE,requiredcontent-areaknowledgeiscoveredbytheIdLScore,takenbyallELEDandIECEstudents.ForMIEDstudents,requiredcontent-areaiscoveredbytheIdLSMIEDcoreandupper-levelconcentrations.SPEDstudentsdonothaveacontent-relatedexam.

IdLSgraduateswillapplycontent-areaproficiencyinpK-8classroomsettings.

ForcurrentJMUMATstudentswhocompletedtheIdLSmajor,morethan80%willgetconfirmationthattheyappropriatelyappliedcontent-knowledgeduringtheirstudentteachingassignments.

ST-9data(itemA2,“Identifieskeyprinciplesandconceptsofsubjectmatter”)completedbystudentteachersupervisorsandJMUinstructors.

Whilethe80%passrateisnotrequiredbyanyaccreditingbody,thiscontent-relatedpassratemirrorsthePRAXISIIpassraterequiredbyNCATEandisthereforeanappropriateminimumexpectationforcontentproficiency.

Table1.IdLSGoals,Objectives,andMeasures

*Asdiscussedinthefollowingtext,startingwiththe2013-2014reportingyear,theISST(InformationSeekingSkillsTest)testhasbeenreplacedbytheMREST(MadisonResearchEssentialSkillsTest).PartII.Course/LearningExperiencesVirginiarequiresallofitsteachercandidatestobepreparedtoteachthematerialinalloftheSOLfortheareaoflicensure,thereforetheIdLSprogramgoalsandobjectivesmustmeshwiththestateandfederalrequirementsforteachereducation.In2005-06,IdLSfacultyconductedthefollowingalignmentsofourcurriculum. Math/Science Humanities/SocialScience Core Concentration Core ConcentrationVA–SOLElementary X X VALicensureStandards–ElementaryEducation

MathOnly MathOnly X X

VALicensureStandards–MiddleEducation

X X X X

SPAStandards ScienceOnly ScienceOnly Table2.AlignmentsconductedforIdLScurriculumandaccreditation/licensurestandards,2005-06.

Resultsofthesealignmentstudiesrevealedthatourcorecurriculuminbothmath/scienceandhumanities/socialsciencesincludesnearlyalloftheessentialcomponentsforteacherlicensure.Afew

3

specificsubjectareasinsciencehavelittleornocoverage(weather,plants,soil,technologyforexample)andinlanguageartsstudentsareexposedtooneoratmosttwoofthe4literatureareas(American,British,World,Ethnic)butoverallthecorecurriculumprovidesanexcellentfoundationinall4subjectareas.Theconcentrationcurriculumwasevaluatedintwoways.First,transcriptsofallrecentMiddleEducationgraduateswereanalyzed.Sincestudentshavemanychoicesintheirconcentrationcoursework,itwasfeltthatdirectevaluationoftranscriptswouldgivethebestinformationofwhatisactuallycoveredinstudents’programs.ThesedataarefoundintheAppendicesofthe2009-2010report.Second,thecoursesthemselveswereanalyzedfortheSOLorlicensureareasthattheinstructorscoverinthecourse.ThesedataarealsofoundintheAppendicesofthe2009-2010report.TranscriptevaluationsshowedthatmostoftheMIEDhumanities/socialsciencesstudentschoosecoursesthatcoverlessthanhalfoftherequiredlicensurecompetencies.Particularweaknesseswereincivics/economicsandworldhistory.Worldhistoryiscoveredextensivelyinthecore,butcivic/economicscoverageappearsweakinbothcoreandconcentration.Math/scienceMIEDconcentrators’transcriptswerenotevaluatedinthesameway,becausethesciencecomponentofthisconcentrationhaschangedsignificantlyinthepastseveralyears.Thisevaluationshowedthatstudentsarechoosingcourseswhichfallintooneortwosciencedisciplines(astheoldguidelinesrecommended).Thenewconcentrationguidelinesaremorerestrictiveofcourseselectionsandrequireabroaderchoiceofdisciplineareas.Futureevaluationsneedtobedonetodetermineifcoverageisimproved.Theindividualalignmentsarefoundintheappendicesassociatedwiththe2005-06report.

4

PartIII.Evaluation/AssessmentMethodsIdLSassessmentisverycomplex.Evaluatingstudentswithtwodistinctupperdivisionconcentrations,fortheirmasteryofknowledge,skills/attitudesineachof4subjectareasposesachallenge.Thankfullyseveralfacultyanddepartmentshavebeenextraordinarilyhelpfulinassemblingdataforourevaluation.CARSstaffhavedoneanalysesofGeneralEducationdata(Clusters1,3,and4)thatidentifyIdLSstudentsandcalculatetheirscoresseparately.TheEducationalSupportCenterintheCoEhasprovideddatabasequeriesandprovidedstudentinformationregardingPRAXISIItestresultsandST-9results.(SeeTable1foradescriptionofeachoftheinstrumentsused(ClustersinGenED,PRAXISII,andST-9)andwhyIdLSchosetousethem).Table3belowindicatesthecurrentstatusofassessmentsforcandidates’knowledgeandskills/attitudesineachofthefourcoresubjectareas.

SubjectArea InstrumentsUsedtoEvaluateCandidates’Knowledge Skills/Attitudes

Science Cluster3,PRAXISII ST-9Math Cluster3,PRAXISII ST-9LanguageArts Cluster1,PRAXISII ST-9SocialStudies Cluster4,PRAXISII ST-9Table3.IdLSassessmentmethodsgroupedbysubjectareaversusknowledgeorskill/attitude

GeneralEducationInstrumentsTheCorecomponentoftheIdLScurriculumincludesallGenEdrequirementsandallowsustocompareperformanceofIdLSstudentstonon-IdLSstudentsusingthefollowingGeneralEducationassessmenttests:MadisonResearchEssentialSkillsTest(MREST),NaturalWorld(NAW)quantitativereasoning,NaturalWorldscientificreasoning(NAW),GlobalExperience(GLEX),andAmericanExperience(AMEX).Aswasthecaseforacademicyears2008-2009,2009-2010,2010-2011,2011-2012,2012-2013,2013-2014,and2014-2015,dataforthisyear(2015-2016)wereevaluatedontheperformanceofIdLSstudentsfortheMREST,theNaturalWorldQR&SR,theGlobalExperience,andAmericanExperiencetests.Generaldescription,datacollectioninformation,anddesiredresultsareprovidedforeachofthesegeneraleducationtestsbelow.MadisonResearchEssentialsTest(MREST)(oldInformationSeekingSkillsTest(ISST))Startingwiththe2013-2014year’sAPTtheMadisonResearchEssentialsTest(MREST)hasbeenusedtoassessthestudentsabilityto:(http://www.jmu.edu/gened/infor_lit_general.shtml)

1. Recognizethatinformationisavailableinavarietyofformsincluding,butnotlimitedto,text,images,andvisualmedia.

2. Determinewheninformationisneededandfinditefficientlyusingavarietyofreferencesources.

3. Evaluatethequalityoftheinformation.4. Useinformationeffectivelyforapurpose.5. Employappropriatetechnologiestocreateaninformation-basedproduct.

5

6. Useinformationethicallyandlegally.Priorto2013-2014,theexamusedforthisassessmentwastheInformationSeekingSkillsTest(ISST).AccordingtoDeMars,Cameron,andErwin(2003),“theISSTisaweb-basedtestof53multiple-choiceitems.Fourcontentareas(BasicReference,DatabaseSearching,InternetSkills,Ethics)arecrossedwithtwoprocessareas(Knowledge,Application).Applicationquestionsrequirestudentstoapplyknowledgebyfindinganswersincatalogsanddatabasesandbyevaluatingwebsites.Proctorsadministerthetestinacomputerlab”.(http://muse.jhu.edu/journals/journal_of_general_education/v052/52.4demars.html)Becausefirst-yearstudentsmustpassthetestbeforeenrollinginsophomorecourses,studentstypicallygiveagoodeffortonthistest.PracticallyallIdLSstudentstakethistest(i.e.,acensus).TheexactnumberofIdLSstudentswhotookthetestisprovidedintheresultssection.Reliabilityanalysesoverthepastseveralyears(viaitemresponsetheory)revealthatthereliabilityfortheentiretestisinthelowtomid.70’s,areasonablelevelformakinggroupdecisionsinhighereducation.LibrariansdevelopedthistestandstudiesbyCARShaveindicatedthatstudentswhohavehadmoreexposuretoinformationliteracycurriculum(e.g.,inclassworkorpracticewithwebmodules)performbetteronthetest.Thesefactorsprovidevalidityevidencethatthescoresonthistestrepresentinformationliteracy.ThedesiredoutcomeisthatIdLSstudentsexhibitthesamedegreeofcompetenceasnon-IdLSstudentsontheMREST.NaturalWorldTestVersion9,ScientificReasoningandQuantitativeReasoningScoresTheNW-9testconsistsof66items,allofwhichcontributetothescientificreasoning(SR)score.Twenty-sixofthoseitemsalsocontributetoquantitativereasoningandaretotaledfora“QR”subscore.Thistestisdeliveredviapaperandpencilandcomputer-basedversions,bothinthecontextofAssessmentDay.Itistypicalthatapproximatelyonequarterofenteringfreshmenarerandomlyassigned(viathelasttwodigitsofastudent’sID)totaketheNAW-9duringafallAssessmentDay.Unfortunately,theSpring2015NAW-9hadanextremelysmallsamplesizeduetosomeaflawedexperimentaldatadesignswhichresultedinonly4IDLSstudentsbeingmatchedfrompretesttoposttest.Thisunfortunatelymakesstatisticaltestsinappropriate.Asaresultthisyear’sreportwillnotpresentNAW-9testresultsforthisyearbutwillinsteadpresentthepreviousyear’stestresultswhereastatisticallysignificantnumberofIdLSstudentstooktheNAW-9asenteringfreshmenandretookthetestinthespringof2014.PrioryearsstudentsselfreportthattheyaremotivatedtotakethisexamandgiveareasonableeffortontheNAW-9.ThereliabilityoftheSRandQRscoresaretypicallyinthe.70sand.60s(Cronbach’salphas)respectively.Thislevelofprecisionisrespectableforhighereducationtestsforgroup-leveldecisions.Thetestwasdesignedbyfacultycontentexpertsandthesescoresrelatetobothcourseexposureandcoursegradesinscienceandmath.Thesefactorscontributetovalidityevidencethatthescoresdoindeedreflectquantitativeandscientificreasoning.Intermsofdesiredresults,theIdLSprogramwouldlikeIdLSsophomores(post-test)toscorethesameasotherJMUstudents.Additionally,theIdLSprogramwouldlikeIdLSstudentstomakesimilargainsfrompre-testtopost-testasnon-IDLSstudents.ThesecriteriafordesiredresultsarebaseduponpreviousdataprovidedbyCARS.Global(GLEX)andAmericanExperience(AMEX)TestsTheGLEXinstrumentconsistsof31multiplechoiceitems,AMEXconsistsof81multiplechoiceitems.ThetestsareadministeredtoincomingFreshmenduringtheAugustassessmentday,andtostudents

6

with45-70credithoursduringtheSpringassessmentday.Testsweredevelopedbycontentareafaculty.Scoresonbothtestsarestandardizedtoameanof500andstandarddeviationof100,setsotheymatchthemeansofthenorminggroupsforthetests(freshmenin2000or2001).ThereliabilityoftheAMEXtestisconsistentlyintherangeof0.87,theGLEXistypicallyintherangeor0.75(Cronbach’salpha).Thesereliabilitiesaresufficienttomakegroup-leveldecisionsbasedonaggregatedscores.PRAXISIIAllteacherlicensurecandidatesmustpasstherelevantPRAXISIIexam(s)inordertobelicensed.TheseexamsaredevelopedatEducationalTestingServices(ETS)inconsultationwithteachingexpertsacrossthenation.Inessence,thetestsaredesignedtocorresponddirectlywithteachinglicensureobjectives.ETSprovidesreliabilityandvalidityevidenceforthistest:http://www.ets.org/s/praxis/pdf/validity.pdf.Thereliabilitiesofthese5testsrangefrom0.88to0.90nationally.Becauseapassingscoreisrequiredforlicensure,studentsareassumedtoprovideagoodeffortonthistest.Forthepastseveralyears,scorereportsandinstitutionalsummariesofJMUdatahavebeenavailablefromETS.FortheElementaryEducation(ElEd)ContentKnowledgetest,scoresareprovidedforeachofthe4subjectareasubscales.Startingwiththisyears’APTreport,IdLSstudentsarerequiredbytheVirginiaDepartmentofEducation(VDOE)topasseachofthefoursubjectareas(Science,Math,LanguageArts,andSocialStudies)tests.InallpriorreportsIdLSstudentspassed(orfailed)thePRAXISIIexambasedonanaggregationofscoresforallfoursubjectareas(Science,Math,LanguageArts,andSocialStudies)whereeachareacontributedequally(25%)tothetotalscore.ElEdPRAXISIIdatareportedforthisyears’APTissomewhatconfusingasaresultoftheentireexaminebeingrewrittenandscoresbeingrecalibrated.Foreachofthefoursubjectareastherearetwoversionsoftheexamthatwillbereported.Thebreakdownofcontentontheexamsisasfollows.From9/14/14to6/30/15ElEdReadingandLanguageArtsContentCategories(Test5032)

ApproximatePercentageofExamination

I.Reading 49%II.Language,Writing,andCommunication 51%(source:https://www.ets.org/s/praxis/pdf/5031.pdf)After7/1/15ElEdReadingandLanguageArtsContentCategories(Test5002)


I.ReadingII.Writing,Speaking,andListening

47%53%

(source:https://www.ets.org/s/praxis/pdf/5001.pdf)From9/14/14to6/30/15ElEdMathematicsContentCategories(Test5033)


I.Number,Operations,andAlgebraicThinking 65%

7

II.Geometry,Measurement,Data,andInterpretation 35%(source:https://www.ets.org/s/praxis/pdf/5031.pdf)After7/1/15ElEdMathematicsContentCategories(Test5003)


I.NumbersandOperationsII.AlgebraicThinking

40%30%

III.GeometryandMeasurement,Data,Statistics,andProbability

30%

(source:https://www.ets.org/s/praxis/pdf/5001.pdf)From9/14/14to6/30/15ElEdSocialStudiesContentCategories(Test5034)


I.UnitedStatesHistory,Government,andCitizenship 45%II.Geography,Anthropology,andSociologyIII.WorldHistoryandEconomics

30%25%

(source:https://www.ets.org/s/praxis/pdf/5031.pdf)After7/1/15ElEdSocialStudiesContentCategories(Test5004)


I.UnitedStatesHistory,Government,andCitizenshipII.Geography,Anthropology,andSociology

40%30%

III.WorldHistoryandEconomics 30%(source:https://www.ets.org/s/praxis/pdf/5001.pdf)From9/14/14to6/30/15ElEdScienceContentCategories(Test5035)


I.EarthScience 32%II.LifeScienceIII.PhysicalScience

34%34%

(source:https://www.ets.org/s/praxis/pdf/5031.pdf)After7/1/15ElEdScienceContentCategories(Test5005)


I.EarthScienceII.LifeScience

32%33%

III.PhysicalScience 34%(source:https://www.ets.org/s/praxis/pdf/5001.pdf)TableSet4A.ContentareacoverageandexambreakdownforthefourElementaryEducationPraxisIIcontentexams.

8

Eachofthe4MiddleSchoolsubjectareatestscontainsseveraldiscipline-relatedscales(seebelow).ETSpublishesthelistofcontentknowledgethatisusedtodevelopthetest;thisappearstomatchtheIdLSlearningobjectivesfairlywell.ETSrecommendsthatPRAXIScontentbealignedwithcurriculumandlearningoutcomesbeforeusingittomakedecisionsaboutprograms.Thebreakdownofcontentontheexamsisasfollows(NOTE:VDOErequiredstudentstakingtheMathematicsandLanguageArtscontentexamstotakedifferentexamsafterJan.1,2014,thereforebreakdownsareprovidedforbothoftheseexams):PriortoDec31,2013MiddleSchoolMathematicsContentCategories


I.ArithmeticandBasicAlgebra 20%II.GeometryandMeasurement 17%III.FunctionsandTheirGraphs 13%IV.Data,Probability,andStatisticalConcepts;DiscreteMathematics

17%V.Problem-SolvingExercises 33%ProcessCategories(DistributedAcrossContentCategories)MathematicalProblemSolving,MathematicalReasoningandProof,MathematicalConnections,MathematicalRepresentation,UseofTechnology(source:http://www.ets.org/s/praxis/pdf/0069.pdf)AfterJan1,2014MiddleSchoolMathematicsContentCategories


I.ArithmeticandAlgebra 62%II.GeometryandData 38%(source:http://www.ets.org/s/praxis/pdf/5169.pdf)PriortoDec31,2013MiddleSchoolLanguageArtsContentCategories ApproximatePercentageofExaminationI.ReadingandLiteratureStudy 37%II.LanguageStudy 13%III.CompositionandRhetoric 25%IV.ShortEssays

1.TextualInterpretation,2.TeachingReading/Writing

25%(source:http://www.ets.org/s/praxis/pdf/5049.pdf)

AfterJan1,2014MiddleSchoolLanguageArtsContentCategories ApproximatePercentageofExaminationI.Reading 46%II.LanguageUseandVocabulary 11%

9

III.Writing,Speaking,andListening 18%IV.EnglishLanguageArtsInstruction 25%

(source:http://www.ets.org/s/praxis/pdf/5049.pdf)

MiddleSchoolScienceContentCategories ApproximatePercentageofTotalScoreI.ScientificMethodology,Techniques,andHistory 8%II.BasicPrinciples 11%III.PhysicalSciences 18%IV.LifeSciences 15%V.Earth/SpaceSciences 15%VI.Science,Technology,andSociety 8%VII.ShortContentEssays:

1.PhysicalSciences,2.LifeSciences,3.Earth/SpaceSciences

25%(source:http://www.ets.org/s/praxis/pdf/0349.pdf)

MiddleSchoolSocialStudiesContentCategories ApproximatePercentageofExaminationI.UnitedStatesHistory 19%II.WorldHistory 15%III.Government/Civics 14%IV.Geography 14%V.Economics 13%VI.ShortContentEssays 25%(source:http://www.ets.org/s/praxis/pdf/0089.pdf)TableSet4B.ContentareacoverageandexambreakdownforfourMiddleSchoolPraxisIIcontentexams.ST-9ST-9ispartofthe“AssessmentofStudentTeaching”conductedbytheCOEatJMU.Thisform(seeAppendix1),titled“PROFILEOFSTUDENTTEACHINGPERFORMANCE”isfilledoutbythecooperatingteacheranduniversitysupervisorwhiletheIdLSstudentisStudentTeaching.BoxA2ofthisformpertainstotheabilityoftheSTUDENTTEACHERtoIDENTIFYKEYPRINCIPLESANDCONCEPTSOFSUBJECTMATTER.Ascoreof:

• 3.0meansthatthestudentteacherexplicitlyreferencesANDclearlyalignsappropriatecontentstandardswithplannedactivitiesandassessments,

• 2.0meansthatthestudentteacherexplicitlyreferencesappropriatecontentstandardsindailyplans.

• 1.0meansthatthestudentteacherinaccuratelyandvaguelyreferencesORdoesnotreferenceappropriatecontentstandards.

10

ThemostrecentdatathatisavailablefromtheCOEisforthe2013-2014AcademicYear,andiswhatwillbepresentedhere.

11

PARTIV.ObjectiveAccomplishments/ResultsGENERALEDUCATIONCluster3NW-9TestResults:AsstatedinPartIIItheSpring2015NAW-9hadanextremelysmallsamplesizeduetosomeexperimentsmissingdatadesignswhichresultedinonly4IDLSstudentsbeingmatchedfrompretesttoposttest.Thisunfortunatelymakesstatisticaltestsinappropriate.ThereforethefollowingpresentationofresultsdonotincludedatafortheSpring2015IdLScohort.TheNaturalWorld(NW-9)instrumentmeasuresgeneralscientificreasoningandanalysisskills,independentofspecificcontent.Assuch,itisagoodtestofstudents’overallscienceabilityorskill,butnotoftheirspecificsubjectareaknowledge.AsshowninthelastcolumnofTable4A,IdLSstudents(n=21),onaverageanswered73.88%(SD=12.40)ofNW-9itemscorrectly;whereasnon-IdLSstudents(n-384)answeredonaverage72.81%(SD=13.53)correctly.Thesedifferences(forSpring2016)failedtobestatisticallysignificantt(403)=0.356,p=.700,d=.08.However,thedifferencesinSpring2014werestatisticallysignificantt(973)=3.12,p=.002,d=.20.Stateddifferently,theaverageIdLSNW-9scoreisapproximately.20standarddeviationsbelownon-IdLSstudents.Note:Table4Apresentsdataforthelast4NW-9testresults.Startingwiththemostrecentreportingperiod(Spring2014),CARSispresentingdataas“Mean%”correct.Inpreviousyears,dataispresentedas“Meancorrectresponses”.

NW-9DescriptiveStatisticsforTotalScore Spring2012 Spring2013 Spring2014 Spring2016

MeanCorrect

SD N MeanCorrect

SD N Mean%Correct

SD N Mean%Correct

SD N

Non-IdLS 49.21 7.62 943 49.08 7.63 1231 74.76 11.84 919 73.88 12.40 384

IdLS 48.03 6.73 66 46.49 7.08 74 69.59 11.61 56 72.81 13.53 21

Table4A.ComparisonofNW-9Scoresofnon-IdLSandIdLSstudentsSpring2014andthethreepreviousyears.

AsimpliedbydatashowninthelastcolumnofTable4B,similarresultsareobtainedwithQuantitativeReasoning(QR)scorest(404)=.049,p=.001,d=.01.

Note:Table4Bpresentsdataforthelast4QRtestresults.Startingwiththemostrecentreportingperiod(Spring2014),CARSispresentingdataas“Mean%”correct.Inpreviousyears,dataispresentedas“Meancorrectresponses”.

NW-9DescriptiveStatisticsforQuantitativeReasoning(QR) Spring2012 Spring2013 Spring2014 Spring2016

Mean SD N Mean SD N Mean% SD N Mean% SD N

12

Correct Correct Correct CorrectNon-IdLS

18.55 3.73 943 18.49 3.79 1231

71.45 14.53 919 69.59 0.78 384

IdLS 17.92 3.43 66 17.31 3.72 74 64.22 14.67 56 69.43 3.08 21

Table4B.ComparisonofQRScoresofnon-IdLSandIdLSstudentsforSpring2014andthethreepreviousyears.

InordertodetermineifthesedifferenceswereafunctionCluster3coursework,thisanalysiswasreplicatedwhenholdingCluster3courseworkrequirementsconstant.Table4CprovidesdescriptivestatisticsforperformanceontheNW-9andQRonlyforstudentswhocompletedtheirCluster3requirements.

IdLSstudentswhocompletedrelevantcourseworkonaverageanswered70.48%(SD=14.04)oftheNW-9correctly;whereassimilarnon-IdLSstudentsanswered77.01%(SD=11.10)oftheNW-9correctly.Thesedifferenceswerestatisticallysignificantt(295)=2.65,p=.009,d=.33.ThesevaluesindicatethattheaverageIdLSstudent,whocompletedtheircoursework,isapproximately1/3ofastandarddeviationbelowtheaveragenon-IDLSstudentontheNW-9.Onceagain,similarresultswereobtainedforQRt(295)=2.38,p=.02,d=.29.

Note:DatapresentedinTables4C-4Earenotavailableforpreviousyears.

Table4C.Cluster3MeanScoresandStandardDeviationsforStudentswhoCompletedRelevantCourseworkforSpring2014

Ofthe56IdLSstudentswhocompletedtheNW-9inFebruary2014,therewere23whocompletedtheirCluster3requirements.Ofthese,18completedapre-testasenteringfreshmen.Descriptivestatisticsforthesestudents,aswellassimilarnon-IdLSstudents(n=192)arereportedinTable4DfortheNW-9.BothIdLSandnon-IdLSstudentsincreasedinNW-9atpost-testF(1,208)=15.33,p<.001.Thereis

Spring2014 Spring2015andSpring2016

NW-9 QR

Mean%

Correct

SD N Mean%Correct

SD Mean%Correct

SD N Mean%Correct

SD

Non-IdLSstudents

77.01 11.10 274

73.72 14.08

NoData NoData

NoData

NoData NoData

IdLSstudents

70.48 14.04 23

66.38 15.65

NoData NoData

NoData

NoData NoData

13

currentlyinsufficientevidencetoindicatethatthisincreaseisafunctionofwhetherthestudentisanIdLSornon-IdLSstudentF(1,208)=0.19,p=.67.Stateddifferently,itappearsthatbothIdLSandnon-IdLSwhocompletedrelevantCluster3courseworkshowsimilargainsatpost-test.

Pretest Posttest Difference

Non-IdLSstudents(N=192) 48.33(6.16) 51.31(6.89) 2.98

IdLSstudents(N=18) 44.83(6.88) 47.22(7.35) 2.39

Table4D.NW-9Pre-PostComparisonsforStudentswhoCompletedrelevantCluster3CourseworkforSpring2014(Note.Valuesreportedarebasedupon66totalitems)

AsimilaranalysiswascompletedforQRscores(seeTable4E).Onceagain,IdLSstudentsandnon-IdLSstudentsshowedsimilar,thoughstatisticallyinsignificant,gainsatpost-testF(1,208)=0.48,p=.49.

Pretest Posttest Difference

Non-IdLSstudents(N=192) 18.50(3.40) 19.37(3.40) 0.87

IdLSstudents(N=18) 16.94(3.87) 17.28(3.41) 0.34

Table4E.QRPre-PostComparisonsforStudentswhoCompletedrelevantCluster3CourseworkforSpring2014(Note.Valuesreportedarebasedupon26totalitems)

Cluster4GlobalExperienceandAmericanExperienceTests

TheseinstrumentsareusedtoassessperformanceinClusterFourofGeneralEducation.Ofthisyear’sIDLSstudents,27tooktheAmericanExperiencetestandtheGlobalExperiencetestinFebruary2016.MeanstandardizedscoresareshowninTable5A.Thestandardizedscoresweredefinedtohaveameanof500andastandarddeviationof100inthenorminggroupofenteringfreshmen,soa10-pointdifferenceisabout.10standarddeviationunits(comparabletoCohen’sd,butwithaconstantdenominatorthatdoesn’tchangefromyeartoyearorgrouptogroup).

Spring2015 Spring2016

AmericanExperience

GlobalExperience

AmericanExperience

GlobalExperience

N Mean(sd) N Mean(sd) N Mean(sd) N Mean(sd)

IdLSstudents 45 526.9(89.8)42

572.7(94.8)

27 520.6(100.0)27 517.4(109.4)

Non-IdLS 864 529.1 897583.9

604 520.5(115.4) 463 578.5(104.7)

14

students (118.0) (109.7)

Table5A.StandardizedscoresontheAMEXandGLEXforIdLSstudentsandNon-IdLSstudents(StandardDeviation).

AhigherproportionofIDLSstudentscompletedtheAmericanrequirements(85%ofIDLSvs.69%ofnon-IDLS).InGlobal,aboutthesameproportionsofIDLS(70%)andnon-IDLS(68%)studentscompletedtherequirement.Amorereasonablecomparisoncanbemadebetweenstudentswhohadcompletedtherelevantrequirement.ThesevaluesareshowninTable5B.

Spring2015 Spring2016

AmericanExperience

GlobalExperience

AmericanExperience

GlobalExperience

N Mean(sd) N Mean(sd) N Mean(sd) N Mean(sd)

IdLSstudents 40 532.7(90.3)24

580.6(98.9)

23 532.3(99.8)19

524.9(102.3)

Non-IdLSstudents

608 543.7(117.7) 503

596.3(108.4)

417 535.6(114.2)317

586.0(102.3)

Table5B.AMEXandGLEXscoresforstudentswhohadcompletedtheAmericanorGlobalExperiencerequirement.

FromTable5B,thenon-IDLSstudentsscored3points(approximately0.03standarddeviationunits)higher.Thisdifferencewasnotstatisticallysignificant(t438=0.14,p=.893),whichmeansthatadifferenceofthismagnitudecouldbeduetochancevariation.Theplausiblerangeforthedifferencerangedfromabout-44.5to51.1,onthestandardscale.InGlobalExperience,thenon-IDLSstudentsscored0.60standarddeviationunitshigher.Althoughthisseemslikealargedifference,duetothesmallsamplesizewehavelittleconfidenceintheestimateofthedifference;theplausiblerangeofthedifferenceisabout13.5to108.6onthestandardscale.Thus,althoughthedifferencebetweentheIDLSstudentscoresandthenon-IDLSstudentscoreswasstatisticallysignificant(t334=2.53,p=.012),thedifferencecouldbequitesmallorverylarge.

Ofthe27IDLSstudentstestedontheAmericanExperience,18hadscoresfromthefall1.5or2.5yearsbefore.Similarly,21oftheIDLSstudentshadearlierscoresontheGlobalExperiencetest.15ofthesestudentshadcompletedoneoftheAmericanExperiencecoursesatJMU,and15hadcompletedoneormoreoftheGlobalExperiencecourses(another5hadtransferorAPcredit).Fifteenstudentsisaverysmallnumber,sounlikepreviousyearstheCARSprogramdidnofurtheranalysesofpretopostdifferences.

Tables5Cand5Dshowpre-andpost-testdifferenceanalysisfortheprevioustwoyears.For2015(themostrecentyearwherethisdataisavailable)ofthe45IDLSstudentstestedontheAmericanExperience,34hadscoresfromthefall1.5or2.5yearsbefore.Similarly,34oftheIDLSstudentshadearlierscoresontheGlobalExperiencetest.26ofthesestudentshadcompletedoneoftheAmerican

15

ExperiencecoursesatJMU(another5hadtransferorAPcredit),and21hadcompletedoneormoreoftheGlobalExperiencecourses(another4hadtransferorAPcredit).Thespringtestservedasaposttestforthesestudents.UnlikeTables2and3,thepre-postcomparisondoesnotincludestudentswhocompletedtherequirementbyAPortransfercreditbeforethepretestanddidnottakeanadditionalcourseatJMU(thesestudentswouldnotbeexpectedtoincreasetheirscores).MeanscoresareshowninTables5Cand5D.

AmericanExperiencePre-PostComparisons

2014Data 2015Data

N Pretest(sd)

Posttest(sd)

Difference N Pretest(sd)

Posttest(sd)

Difference

IdLSstudents 24 468.0

(107.7)516.4(103.0) 48.4 26 471.7

(90.3)527.0(92.6) 55.3

Non-IdLSstudents 273 491.9

(99.3)522.5(102.5) 30.6 307 495.1

(103.6)532.1(101.1) 37.0

Table5C.Pre-andPost-testcomparisonsforAmericanExperience(StandardDeviation).

GlobalExperiencePre-PostComparisons

2014Data 2015Data

N Pretest(sd)

Posttest(sd)

Difference N Pretest(sd)

Posttest(sd)

Difference

IdLSstudents 23 522.8(137.2)

587.9(123.6) 65.1 21 520.8

(86.0)584.3(97.1) 63.5

Non-IdLSstudents 560 543.3

(109.3)586.3(117.5) 43.0 387 542.2

(106.1)604.8(103.8) 62.6

Table5D.Pre-andPost-testcomparisonsforGlobalExperience(StandardDeviation).

OntheAmericanExperiencetest,theinteractionbetweenIdLS/notIdLSandpre/posttestwasnotstatisticallysignificantthisyear(F1,331=1.25,p=.264).Inotherwords,thenon-IdLSincreasewasnotsignificantlydifferentfromtheIdLSincrease.Withsuchasmallsample,themeandifferencesareunstable.

OntheGlobalExperiencetest,therewasnotasignificantinteractionbetweenIdLS/non-IdLSandpre/posttest(F1,406=0.00,p=.965).Inotherwords,thenon-IdLSincreasewasnotsignificantlydifferentfromtheIdLSincrease,asonewouldexpectgiventhatthedifferenceswerenearlyequal.

16

Cluster1(MRESTtest)

Startingin2014,theCluster1examiscalledtheMadisonResearchEssentialSkillsTest(MREST).TherearetwoformsoftheMREST,soscoresareagainreportedonlyonthestandardizedscale.Scoresrangefrom100-200.TheMRESTcontainsitemsthatwereadministeredwiththeISSTin2012-2013,sothescalewassetwithameanof150andstandarddeviationof15forthe2012-2013students.Thepassingscorewassetbyafacultycommitteeat148.Scoresof166orgreaterreceiveanAdvancedtranscriptnotation.Studentsmayrepeatthetestanunlimitednumberoftimes,andtutorialsareavailable.Nearlyallstudentspassbytheendofthe1styear(thosewhodonotprobablydidnotbotherrepeatingthetestiftheydidnotintendtoremainatJMU).Thisyear(2016Data)IdLSandnon-IdLSstudentsperformedapproximatelythesameontheMRESTexam(seeTable6A).100%ofIdLSstudentspassand99%ofnon-IdLSstudentspass,and22%ofIdLSstudentspassadvancewhile21%onnon-IdLSstudentspassadvance.PercentPassingMREST(ISST)(ofthosewhoattemptedthetestatleastonce)

2015Data(MREST) 2016Data(MREST)

N

#Pass

%Pass

#Advanced

%Advanced

N#

Pass%

Pass#

Advanced%

Advanced

IdLS 186 186 100% 49 26% 184 184 100% 41 22%

NonIdLS

3997 3949 98.8% 964 24% 4016 3980 99% 836 21%

Table6A.PercentPassingeithertheISSTortheMREST(ofthosewhoattemptedthetestatleastonce)

IdLSstudentsattemptedthetestanaverageof1.83times,almostthesameasthenon-IDLSstudentswith1.86attemptsonaverage.Scoresfromthe1stattemptandfinalattemptareinthetablebelow(seeTable6B).Formanystudents,the1stattemptwasalsothefinalattempt;onlythosewhodidnotpassrepeatedthetest.Thus,scoresincreaseandthestandarddeviationdecreasesforthefinalattempt.

2015Data(MREST) 2016Data(MREST)

1stAttempt FinalAttempt 1stAttempt FinalAttempt

Mean SD Mean SD Mean SD Mean SD

IdLS 153.1 13.6 159.9 10.2 149.2 13.1 158.9 8.52

Non-IdLS 152.8 14.6 159.6 10.1 150.4 14.3 158.8 9.43

Table6B.Meanscoresfor1standfinalattemptsforeithertheISSTortheMRESTforIdLSstudentsandothers.

17

IdLSstudentsscorednearlythesameasnon-IdLSstudents.Thedifferencebetweennon-IdLSandIdLSstudentscoreswasvirtuallyzeroandnotstatisticallysignificant[firstattempt:t4198=1.13,p=.259,finalattempt:t4198=-0.10,p=.917].

18

PRAXISIIElementaryEducation(ElEd)ContentKnowledgeAsofthewritingofthisreport,theElEdContentKnowledgetestsarehigh-stakesassessmentsinwhichstudentsmustpassallfoursubjectareaexams(Reading&LanguageArts,Mathematics,SocialStudies,andScience)inordertoreceivelicensuretoteachinVirginia.ThetestgenerallymatchestheIdLSprogramcorecurriculumsincethisiscontentthatallelementaryteachersmustteach.

ForthisreportingperiodtherearetwoexamsthatwereadministeredbyPRAXISandbothoftheseexamsaredifferentthanthePRAXISIIteststhatwerereportedinpreviousAPT’sforIdLS.Theresultofthisisthatitisdifficulttomakedirectcomparisonsbetweenthisandpreviousyears’data,howeverforcompletenessdataforthreepreviousyearsofPRAXISIItestresults(TablesIandJ)anddiscussionofdataareincludedattheendofthissectionforcomparison.

-ElEdReading&LanguageArts

Therearetwoversionsofthisexamthatstudentsmayhavetakenduringthisreportingperiod.TableSet4Aofthisreportdetailsthecontentareascoveredforbothperiodsexams.Forexam#5032thepassingscoreis165andforexam#5002thepassingscoreis157.Atotalof168studentstookexam#5032andtheirscoresrangedfrom200-100withamedianscoreof185.Moststudentstakingthisexampassed.Atotalof168studentstookexam#5032andtheirscoresrangedfrom200-100withamedianscoreof185(Table7A).Moststudentstakingthisexampassed.Atotalof27studentstookexam#5002andtheirscoresrangedfrom190-157withamedianscoreof174(Table7A).Allstudentstakingthisexampassed.Forbothexams,themedianscoreishigherthanthenationalaverage.

ElementaryEducation:Reading&LanguageArts5032

9/1/2014to8/31/2015

ALL JMU

N 9,069 168

High 200 200

Low 100 100

Median 178 185

AverageRange 170-187 180-191


9/1/2014to8/31/2015

ALL JMU

N 8,831 27

High 200 190

Low 100 157

Median 168 174


19

Tables7A.PRAXISIIscoresforalltesttakersandJMUcohortforElementaryEducationReading&LanguageArtsTests

Tables7Bshowthequartilescoresforbothversionsofthisexam(Reading,andLanguage,Writing,andCommunicationorWriting,Speaking,andListening).Theweakestareaoftheexamistheportiondealingwithwritingforeitherversionoftheexam,with37-53%ofJMUstudentsscoringinthelowest2quartiles.


9/1/2014to8/31/2015

Subscale

Number(Percent)ofScoresineachquartile

1st(low) 2nd 3rd 4th(high)

Reading 11(7%) 36(21%) 56(33%) 65(39%)

Language,Writing,andCommunication 12(7%) 51(30%) 55(33%) 50(30%)

N=168

Table7B.JMUquartileresultsforElementaryEducationPraxisIIReading&LanguageArtsTest.

-ElEdMathematics

Therearetwoversionsofthisexamthatstudentsmayhavetakenduringthisreportingperiod.TableSet4Aofthisreportdetailsthecontentareascoveredforbothperiodsexams.Forexam#5033thepassingscoreis164andforexam#5003thepassingscoreis157.Atotalof168studentstookexam#5032andtheirscoresrangedfrom200-179withamedianscoreof179(Table7C).Moststudentstakingthisexampassed.Atotalof30studentstookexam#5002andtheirscoresrangedfrom200-146withamedianscoreof184(Table7A).Moststudentstakingthisexampassed.Forbothexams,themedianscoreismorethan10pointshigherthanthenationalaverage.


9/1/2014to8/31/2015

Subscale



Reading 0(0%) 7(26%) 10(37%) 10(37%)

Writing,SpeakingandListening 2(7%) 13(48%) 5(19%) 7(26%)

N=27

20

Tables7C.PRAXISIIscoresforalltesttakersandJMUcohortforElementaryEducationMathematicsExam.

Tables7Dshowthecontentareaquartilescoresforbothversionsofthisexam.Forallcontentareasandtestversions,morethan70%oftheJMUcohortscoredinthetop2quartilessoidentifyingtheweakestcontentareaisdifficulty.However,testversion#5003indicatesthat27%oftheJMUcohortscoredinthesecondlowestquartile.However,withonly30studentstakingthisexamitwouldbedifficulttomakeaprogrammaticchangebasedonthisresult.

ElementaryEducation:MathematicsSubtest5033

9/1/2014to8/31/2015

ALL JMU

N 11,598 168

High 200 200

Low 100 146

Median 168 179


ElementaryEducation:Mathematics5003

9/1/2014to8/31/2015

ALL JMU

N 9,428 30

High 200 200

Low 100 146

Median 171 184


21

Tables7D.JMUquartileresultsforElementaryEducationPraxisIIMathematicsExam.

-ElEdSocialStudies

Therearetwoversionsofthisexamthatstudentsmayhavetakenduringthisreportingperiod.TableSet4Aofthisreportdetailsthecontentareascoveredforbothperiodsexams.Forbothexams#5034and#5004thepassingscoreis155.Atotalof168studentstookexam#5034andtheirscoresrangedfrom200-144withamedianscoreof179(Tables7E).Moststudentstakingthisexampassed.Atotalof29studentstookexam#5004andtheirscoresrangedfrom200-150withamedianscoreof179(Tables7E).Moststudentstakingthisexampassed.Forbothversionsofthisexamthemedianscoreismorethan10pointshigherthanthenationalaverage.

ElementaryEducation:Mathematics5003

9/1/2014to8/31/2015

Subscale


1st(low) 2nd 3rd4th

(high)

NumbersandOperations 1(3%)

7(23%)

12(40%)

10(33%)

AlgebraicThinking 0(0%)

8(27%) 5(17%)

17(57%)

GeometryandMeasurement,Data,Statistics,andProbability 4(13%)

4(13%)

16(53%) 6(20%)

N=30

ElementaryEducation:MathematicsSubtest5033

9/1/2014to8/31/2015

Subscale



Number,Operations,andAlgebraicThinking 1(1%)

25(15%)

50(30%) 92(55%)

Geometry,Measurement,DataandInterpretation 5(3%)

26(15%)

81(48%) 56(33%)

N=168

22

Tables7E.PRAXISIIscoresforalltesttakersandJMUcohortforElementaryEducationSocialStudiesExam.

Tables7Fshowthecontentareaquartilescoresforbothversionsofthisexam.BothtestversionsindicatethatJMUstudentsstrugglewiththeareas“Geography,Anthropology,andSociology”and“WorldHistoryandEconomics”withapproximately35%and40%ofscoresinthebottomtwoquartilesrespectively.

ElementaryEducation:SocialStudiesSubtest5034

9/1/2014to8/31/2015

ALL JMU

N 10,071 168

High 200 200

Low 100 144

Median 163 176


ElementaryEducation:SocialStudies5004

9/1/2014to8/31/2015

ALL JMU

N 8,927 29

High 200 200

Low 100 150

Median 163 179


23

Tables7F.JMUquartileresultsforElementaryEducationPraxisIISocialStudiesExam.

-ElEdScience

Therearetwoversionsofthisexamthatstudentsmayhavetakenduringthisreportingperiod.TableSet4Aofthisreportdetailsthecontentareascoveredforbothperiodsexams.Forbothexams#5035and#5005thepassingscoreis159.Atotalof166studentstookexam#5035andtheirscoresrangedfrom200-121withamedianscoreof173(Tables7G).Moststudentstakingthisexampassed.Atotalof30studentstookexam#5005andtheirscoresrangedfrom191-150withamedianscoreof170(Tables7E).Moststudentstakingthisexampassed.Forbothversionsofthisexamthemedianscorehigherthanthenationalaverage.

ElementaryEducation:SocialStudies5004

9/1/2014to8/31/2015

Subscale



UnitedStatesHistory,Government,andCitizenship 2(7%) 5(17%)

12(41%) 10(34%)

Geography,Anthropology,andSociology 2(7%) 9(31%)

10(34%) 8(28%)

WorldHistoryandEconomics

3(10%) 7(24%) 6(21%) 13(45%)

N=29

ElementaryEducation:SocialStudiesSubtest5034

9/1/2014to8/31/2015

Subscale


1st(low) 2nd 3rd

4th(high)

UnitedStatesHistory,Government,andCitizenship 6(4%)

18(11%)

66(39%)

78(46%)

Geography,Anthropology,andSociology 6(4%)

51(30%)

76(45%)

35(21%)

WorldHistoryandEconomics

14(8%)

55(33%)

55(33%)

44(26%)

N=168

24

Tables7G.PRAXISIIscoresforalltesttakersandJMUcohortforElementaryEducationSocialStudiesExam.

Tables7Hshowthecontentareaquartilescoresforbothversionsofthisexam.BothtestversionsindicatethatJMUstudentsstrugglewiththeEarthScienceareawithapproximately45%ofscoresinthebottomtwoquartiles.

ElementaryEducation:ScienceSubtest5035

9/1/2014to8/31/2015

ALL JMU

N 10,036 166

High 200 200

Low 100 121

Median 167 173


ElementaryEducation:Science5005

9/1/2014to8/31/2015

ALL JMU

N 8,905 30

High 200 191

Low 100 150

Median 167 170


25

ElementaryEducation:ScienceSubtest5035

9/1/2014to8/31/2015

Subscale



EarthScience

16(10%) 58(35%)

38(23%) 54(33%)

LifeScience 15(9%) 37(22%)60

(36%) 54(33%)

PhysicalScience

17(10%) 34(20%)

61(37%) 54(33%)

N=166

Tables7H.JMUquartileresultsforElementaryEducationPraxisIIScienceExam.

-PreviousPRAXISIITestResultsforElementaryEducation(ElEd)ContentKnowledgeThepreviousversionsoftheElementaryContentKnowledgeexamcoversbasiccontentknowledgeacrossall4subjectareasinIdLS.Inthe2013-2014report,ETSprovideddataforboththepaperexamversionofthistestaswellastheComputerBasedTestsorElectronicversionoftheexam.Inthefollowingtext,scoresthatappearinblackrepresentthepaperexamscoresandthosethatappearasdarkorangerepresenttheElectronicexamscores.JMUstudentscontinuetodoextremelywellontheelementaryeducationcontentknowledgePraxisIItest.Themedianscorefortheprevioustestperiod(9/1/2013to8/31/2014)is177(178),whichis17(18)pointshigherthanthenationalaverage(Table7A).ThisscoreisalsohigherthanthepassscoreforVAlicensurewhichis143(143).ThelowestscoreamongallJMUstudentswhotookthetestduringthisyearwas148(147),indicatingthatallscoresforthePRAXISIItestarepassingscoresforthepreviousreportingperiod.

ElementaryEducation:Science5005

9/1/2014to8/31/2015

Subscale



EarthScience 2(7%)12

(40%) 8(27%) 8(27%)

LifeScience 1(3%) 9(30%) 13(43%) 7(23%)

PhysicalScience 1(3%)

11(37%) 8(27%) 10(33%)

N=30

26

ElementaryEducationPraxis2results

9/1/10to8/31/11 9/1/11to8/31/12 9/1/12to8/31/13 9/1/2013to8/31/2014

ALL JMU ALL JMU ALL JMU ALLJMU

(Electronic)

N 22,833 164 14,589 163 5,477 124 2,199 79(115)

High 200 198 200 199 200 200 200 195(198)

Low 100 143 100 134 100 143 100 148(147)

Median 162 177 163 174 163 177.5 160 177(178)

AverageRange

149-175 168-185 150-176 168-182 149-176 168-184 146-173170-181(169-183)

Table7I.PRAXISIIscoresforalltesttakersandJMUcohortforElementaryEducation

ETSreportsthedistributionofscoresforeachinstitutionrelativetothenationalquartiles.Forthepreviousreportingperiod,ScienceandMathematicshave47%(27%)and61%(43%)ofscoresinthetopquartile,respectively,whileLanguageArtsandSocialStudieshave38%(23%)and49%(28%)ofthescoresinthetopquartile,respectively(Table7B).ItisinterestingtonoteherethatthenumberofstudentsplacinginthetopquartileforallsubjectareasisSIGNFICANTLYlowerfortheComputerBasedTests.

Equallyimpressivearetheverylownumbersofstudentswhoscoredinthelowestquartiles:3%(10%)forScience,3%(4%)forMathematics,6%(2%)forSocialStudies,and5%(11%)LanguageArts(Table7B).Againitisinterestingtonotethatthenumberofstudentsscoringinthelowest(4th)quartileishigherfortheComputerBasedTestsforallareasotherthanSocialStudies.

ElementaryEducationPraxis2results

9/1/2012to8/31/2013 9/1/2013to8/31/2014

Subscale

Number(Percent)ofScoresineachquartile Number(Percent)ofScoresineachquartile

1st(low) 2nd 3rd 4th(high) 1st(low) 2nd 3rd

4th(high)

LanguageArts 8(6%) 39(31%) 39(31%) 38(31%)

4(5%)13(11%)

16(20%)18(16%)

29(37%)57(50%)

30(38%)27(23%)

27

Mathematics 4(3%) 9(7%) 39(31%) 72(58%)

2(3%)5(4%)

7(9%)21(18%)

22(28%)39(34%)

48(61%)50(43%)

SocialStudies 7(6%) 25(20%) 45(36%) 47(38%)

5(6%)2(2%)

10(13%)18(16%)

25(32%)63(55%)

39(49%)32(28%)

Science15

(12%) 24(19%) 34(27%) 51(41%)2(3%)

11(10%)15(19%)29(25%)

25(32%)44(38%)

37(47%)31(27%)

N=124 N=79N=115

Table7J.JMUquartileresultsforElementaryEducationPraxisII.

ComparingthesumofthetoptwoquartilesforeachSubscaleforthe2012-2013dataandthe2013-2014data,weseethatstudentscoresstayedtheapproximatelythesamefortheareasofLanguageArts,andSocialStudies.Therewasanimprovementof7%forMathematics,andasharpdeclinein(10%)fortheSciencearea.

28

MiddleSchoolContentAreas

TheMiddleSchoolContentAreatestsareahigh-stakesassessmentoftheconcentrationcurriculum.Studentsmustpasstwoofthesetests,usuallychosentomatchtheirtwoareasofconcentration.

MiddleSchoolLanguageArts

Therearetwoversionsofthisexamthatstudentsmayhavetakenduringthisreportingperiod.Table4BofthisreportdetailsthecontentareascoveredonthisexamforbothperiodsuptoDec.31,2013andafterJan1,2014.Forbothexams,thepassingscoreis164.Twelve(12)JMUstudentstooktheMiddleEdLanguageArtsPRAXIS2examthisyearandtheirscoresrangedfrom159to184(Table7K)andamedianscoreof169.Virginia’spassscoreforthistestis164.Moststudentstakingthistestpassed.

MiddleEdLanguageArtsPraxis2results

9/1/2012to8/31/2013 9/1/2013to8/31/2014 9/1/2014to8/31/2015

ALL JMU ALL JMU ALL JMU

N 3,177 8 725 12 3,028 12

High 200 200 200 196 195 184

Low 100 169 120 156 114 159

Median 173 179.5 171 183 164 169

AverageRange 160-184 172-188 158-184 166-190 153-171 163-178

Table7K.PRAXISIIscoresforalltesttakersandJMUcohortforMiddleEdLanguageArts.

Tables7Lshowsthequartilescoresforthesubgroupsofthisexam.Thesubscorescategorieschangedinthisyears’ETSreport,sotwotablesareincluded,onetableforthisyearandaseparatetablefortheprevioustwoyearsdata.

Table7Lshowsthat83%ofstudentsscoredinthetoptwoquartilesinLanguageUseandVocabulary.TheweakestareaisWriting,Speaking,andListeningwith50%ofthestudentsscoringinthelowest2quartilesrespectively.However,withonly12studentstakingthistest,thenumberofstudentsistoosmalltodrawreliableprogrammaticconclusions.

29

MiddleSchoolEnglishLanguageArts5047

9/1/2014to8/31/2015

Number(Percent)ofscoresineachquartile


Reading 1(8%) 4(33%) 3(25%) 4(33%)

LanguageuseandVocabulary 0(0%) 2(17%) 3(25%) 7(58%)

Writing,Speaking,andListening

1(8%) 5(42%) 1(8%) 5(42%)

EnglishLanguageArtsandInstruction

2(17%) 2(17%) 7(58%) 1(8%)

ConstructedResponse 0(0%) 5(42%) 5(42%) 2(17%)

N=12

MiddleEdLanguageArtsPraxis2Results

9/1/2012to8/31/2013 9/1/2013to8/31/2014



1st(low)

2nd 3rd4th

(high)1st(low) 2nd 3rd

4th(high)

ReadingandLiteratureStudy

0(%) 4(50%) 1(13%)3

(38%)NR NR NR NR

LanguageStudy1

(13%)2

(25%)3(38%) 2(25%) 0(0%)

5(42%)

4(33%) 3(25%)

CompositionandRhetoric

0(%) 2(25%) 2(25%) 4(50%) 1(8%) 3(25%) 3(25%) 5(42%)

ShortEssays 0(%) 3(38%) 2(25%) 3(38%) NR NR NR NR

30

LiteratureandUnderstandingText

NR NR NR NR 1(8%) 4(33%) 3(25%) 4(33%)

N=8 N=12

Tables7L.TwotablesshowingthreeyearsofJMUquartileresultsfortheMiddleSchoolLanguageArtsPraxisIIexam.Notethatsubscorereportingchangedthisyear.NRdenotesthatETSdidNotReportthesescorestoJMU.

MiddleSchoolSocialStudies

ThisexamcoverscontentinUSHistory,WorldHistory,GovernmentandCivics,Geography,Economics,andSociology/Anthropology.PassingscoreinVirginiais160.Sixstudentstookthetestin2014-15,withscoresrangingfrom155to184(Table7M).Moststudentstakingthistestpassedontheirfirstattempt.

ItisunclearwhattheBehavioralSciencescategoryrepresentedinlastyears’reportthatwasgeneratedbyETSsinceitisnotacontentcategorythatislistedforthisexam.Likewise,itisunclearwhynoShortAnswerresultsweregeneratedinthereportprovidedtoJMUasthiscontentareacountsfor25%ofthetotalscorestudentsreceiveforthisexam.InallsubscalesexceptforGovernment/Civics,atleast50%ofscoreswereinthehighesttwoquartilescomparedtothenationalaverage(Table7N).USHistoryandGovernment/CivicsaretheweakestareasforIdLSstudentswith50%ormoreplacinginthebottomtwoquartilescomparetothenationalaverage.However,withonly6studentstakingthistest,thenumberofstudentsistoosmalltodrawreliableprogrammaticconclusions.

Table7M.JMUversusUSresultsforMiddleEdSocialStudiesPraxisII.NRdenotesthatNoReportwasgeneratedforthisreportingperiod.

MiddleEdSocialStudiesPraxis2results

9/1/2012to8/31/2013 9/1/2013to8/31/2014

9/1/2014to8/31/2015


N NR <5 1,615 12 2,325 6

High NR NR 200 194 200 184

Low NR NR 122 146 122 155

Median NR NR 161 168 164 166

AverageRange NR NR 152-172 158-177 152-177 162-181

31

MiddleEdSocialStudiesPraxis2Results

9/1/2014to8/31/2015 9/1/2014to8/31/2015

Number(Percent)ofscoresineachquartile Number(Percent)ofscoresineachquartile

1st(low) 2nd 3rd 4th(high)1st

(low)2nd 3rd 4th(high)

USHistory 0(0%) 7(58%) 3(25%) 2(17%) 1(17%) 2(33%) 1(17%) 2(33%)

WorldHistory 2(17%) 3(25%) 4(33%) 3(25%) 0(0%) 2(33%) 2(33%) 2(33%)

Government/Civics 0(0%) 6(50%) 4(33%) 2(17%) 2(33%) 2(33%) 2(33%) 0(0%)

Geography 3(25%) 2(17%) 6(50%) 1(8%) 2(33%) 1(17%) 2(33%) 1(17%)

Economics 2(17%) 3(25%) 4(33%) 3(25%) 0(0%) 2(33%) 3(50%) 1(17%)

ShortEssays NR NR NR NR 1(17%) 0(0%) 3(50%) 2(33%)

BehavioralSciences 0(0%) 2(17%) 5(42%) 5(42%) NA NA NA NA

N=12 N=6

Table7N.JMUquartileresultsforMiddleSchoolSocialStudiesPraxisII.NRdenotesthatNoReportwasgeneratedforthisreportingperiodorContentArea.NAdenotesthatNoAnalysiswasperformedonthiscontentareabecauseitdidnotexistduringthereportingperiod.

MiddleSchoolMathematics

AsshowninTable4Bthemathematicsexamchangedthisyear.Thirty-twostudentstookthemiddleschoolmathematicsexamduringthemostrecentreportingperiod.Theirscoresrangedfrom119to20.Themedianscorewas178,whichis9pointshigherthanthenationalaverage(Table7O).ThepassingscoreforthisexaminVirginiais165.Withtheaveragerangebeing169-187forthisyears’datamoststudentspassedthisexam.

Tables7Pshowsthequartilescoresforthesubgroupsofthisexam.Thesubscorescategorieschangedinthisyears’ETSreport,sotwotablesareincluded,onetableforthisyearandaseparatetablefortheprevioustwoyearsdata.

32

Forthecurrentreportingperiod,approximately65%ofscoreswereinthehighesttwoquartiles.(Table7H).

MiddleEdMathematicsPraxis2Results

9/1/2012to8/31/2013 9/1/2013to8/31/2014 9/1/2014to8/31/2015


N 9,294 20 1,768 6 6,151 32

High 200 199 200 191 200 200

Low 100 152 118 164 118 119

Median 164 178.5 166 185 169 178

AverageRange 151-178 165-184 153-180 170-189 154-179 169-187

Table7O.PRAXISIIscoresforalltesttakersandJMUcohortforMiddleEdMathematics.

MiddleSchoolMathematics5169

9/1/2014to8/31/2015



ArithmeticandAlgebra 5(16%) 6(19%) 9(28%) 12(38%)

GeometryandData 2(6%) 38(25%) 11(34%) 11(34%)

N=32

MiddleEdMathematicsPraxis2Results

33

9/1/2012to8/31/2013 9/1/2013to8/31/2014



1st(low)

2nd 3rd4th

(high)1st

(low)2nd 3rd

4th(high)

ArithmeticandBasicAlgebra 3(15%) 3(15%) 6(30%)

8(40%) 0(0%) 1(17%) 2(33%) 3(50%)

GeometryandMeasurement 1(5%) 4(20%) 9(45%)

6(30%) 0(0%) 3(50%) 3(50%) 0(0%)

Functionsandtheirgraphs 1(5%) 2(10%) 9(45%)

8(40%)

1(17%) 0(0%) 3(50%) 2(33%)

Data,probability,statisticalconcepts,discretemath 2(10%) 5(25%) 6(30%)

7(35%)

1(17%) 1(17%) 3(50%) 1(17%)

Problemsolvingexercises 1(5%) 6(30%) 5(25%)

8(40%) 0(0%) 1(17%) 1(17%) 4(67%)

N=20 N=6

Table7P.TwotablesshowingthreeyearsofJMUquartileresultsfortheMiddleSchoolMathematicsPraxisIIexam.Notethatsubscorereportingchangedthisyear.

Tables7O-PcontinuetoconfirmthatstudentscontinuetobeverywellpreparedtotaketheMiddleSchoolMathematicsPRAXIS2exam.

MiddleSchoolScience

Seventeenstudentstookthistestduringtheyear.Thescoresrangedfrom141to184.ThemedianscoreforJMUstudentstakingthetestwas158comparedtothenationalaverageof156(Table7Q).ThepassingscoreforthistestinVirginiais150,whichisadropfromlastyearwherethepassingscorewas163.Moststudentswhotookthisexampassed.

In6ofthe7subscales,themajorityofscoreswereinthehighesttwoquartilescomparedtothenationalaverage(Table7J).“PhysicalSciences”and“BasicPrinciples”arethebestperformingsubscalesintheSciences.“Science,technology,society”istheweakestperformingsubscaleswith64%ofstudentsscoringinthelowest2quartiles.Ofparticularconcernisthatstudentshaveperformedpoorlyonthe“Science,technology,society”areafor7consecutiveyearsanditisbelievedthatthisismeaningful

34

resultfortheprogram.“Earth/SpaceSciences”performedpoorly3outofthelast5years,butfortheprevioustworeportingperiodsJMUstudentsscoredbetterthanthenationalaverage.

MiddleEdSciencePraxis2Results

9/1/2012to8/31/2013 9/1/2013to8/31/2014 9/1/2014to8/31/2015

ALL JMU All JMU All JMU

N 4,171 14 2,551 24 3,092 17

High 200 178 200 189 200 184

Low 100 140 109 133 100 141

Median 156 163 157 169 156 158

AverageRange 146-169 155-173 146-170 163-174 141-170 154-169

Table7Q.PRAXISIIscoresforalltesttakersandJMUcohortforMiddleSchoolSciencePraxisII.

MiddleEdSciencePraxis2Results

9/1/2013to8/31/2014 9/1/2014to8/31/2015



1st(low) 2nd 3rd4th

(high)1st

(low)2nd 3rd

4th(high)

Scientificmethodology,techniques,history 2(8%) 7(29%)

11(46%)

4(17%) 1(6%) 6(35%)

5(29%)

5(29%)

Basicprinciples 0(0%) 8(33%)9

(38%)7

(29%) 1(6%) 4(24%)6

(35%)6

(35%)

Physicalsciences 1(4%) 6(25%) 2(8%)15

(63%) 1(6%) 4(24%)7

(41%)5

(29%)

Lifesciences 1(4%) 5(21%)9 9

1(6%) 8(47%)5 3

35

(38%) (38%) (29%) (18%)

Earth/spacesciences 3(13%) 8(33%)9

(38%)4

(17%)2

(12%) 7(41%)4

(24%)4

(24%)

Science,technology,society 4(17%) 9(38%)6

(25%)5

(21%)6

(35%) 5(29%)5

(29%)1

(6%)

Shortessays 1(4%) 7(29%) 9(38)7

(29%) NA NA NA NA

N=24 N=17

Table7J.JMUquartileresultsforMiddleSchoolSciencePraxisII.

UsingDatafromTable7I,thereisperhapslittledifferencebetweentheperformanceofthe2011-2014groupsofstudentsasMedian,Low,andHighscoresareallapproximatelythesame(althoughthereisperhapsasignificantdifferenceintheLowscores).

AnalysisofPRAXISIIData

-ElementaryEducation

Inpreviousyears’reportsIdLSboasteda100%passrateforallElEdIdLSmajorsforthePRAXISIItest.ThiswasanartifactofthefactthatpreviouslytheoverallPRAXISIIscorewasanaggregationofscoresforallfoursubjectareas(Science,Math,LanguageArts,andSocialStudies)whereeachareacontributedequally(25%)tothetotalscore.Thisallowedstudentstoperformpoorlyonasingleportionoftheexamandstillpass.

Forthecurrentyears’reporttheElEdContentKnowledgetestsarehigh-stakesassessmentsinwhichstudentsmustpassallfoursubjectareaexams(Reading&LanguageArts,Mathematics,SocialStudies,andScience)inordertoreceivelicensuretoteachinVirginia.Thefollowingtable(Table7K)showspassratesofIdLSstudentsforeachofsubjectareasaftertheirfirsttestattemptandtheirfinalattempt.Additionallyfinalnotesaremadeastohowmanytimesstudentshavetakenthetest.

Section Passrateafter1sttry

FinalrateasofJune302015

Notes

LanguageArts 99%

99%

1studentfailedonce

1studentfailedtwice

36

Mathematics 93% 97% 6studentspassedonsecondtry

3studentsfailedonce

2studentsfailedtwice

SocialStudies 97%

99% 3studentspassedonsecondtry

2studentsfailedonce

Science

95%

98% 5studentspassedonsecondtry

2studentsfailedonce

1studentfailedtwice

Table7K.PassinformationfortheELEdPRAXISII.

ItisimportanttonotethatwhileeachoftheElEdpreparationareas(Science,Math,LanguageArts,andSocialStudies)mayhavesub-areaweaknesses,thefactremainsthatallareahaveatleasta97%passrate.ThereforeourIdLSstudentsareverywellpreparedtotakeeachsubjectareaofthishighstakesexamandqualifytobecomefutureteachers.

-MiddleEducation

InpreviousyearsETSsupplieddatasummarizingthenumberofattemptsattakingthePRAXISIIteststudentsneededinordertopass,howeverthisdatawasnotprovidedinthisyears’reporttous.ThefollowingtablesummarizesthenumberofattemptsthatstudentsneededtotakeindividualPRAXISIItestsinordertopassfrompreviousyear’sdata.Whilethisdataisforthevalidforthecurrentyear,wefeelitisstillaccurateinthetrendsthatitshows.

2012-2013Data 9/1/2013-8/31/2014

ContentArea Passedon1st

Attempt

Passedon2nd

Attempt

Passedon3rd(or

more)attempt

NotPassed<number

ofattempts>

Passedon1st

Attempt

Passedon2nd

Attempt

Passedon3rd(or

more)attempt

NotPassed<number

ofattempts>

Mathematics 12 -- -- -- 6 -- -- --

37

Science 6 1 2 1<2>

2<3>

14 3 3 2<1>

1<2>

1<3>

English/LanguageArts

5 -- -- -- 11(?) ?-- -?- 1<1>

SocialStudies 2 1 -- -- 5 -- -- --

Table7K.PassinformationfortheMiddleEdPRAXISII.ThereissomeuncertaintywiththisyearsreportingoftheLanguageArtscontentexam.Allthatisknownfromthedatathatwasreceivedisthatatotalof12studentstookthisexamand1studenthasyettopassthisexam.

Thisdatashowsthatinthe2012-2013cohort,91%ofthestudentswhohadtakenthePRAXISIItestshadeventuallypassedtheexams.Inthecurrentreportingperiod,89%ofmiddleeducationstudentseventuallypassedtheirPRAXISIIexams.DatasuggestthatSciencecontinuestobetheareathatstudentshavethemostdifficultypassing.However,alargemajorityofIdLSstudentstakingPRAXISIIexamspassontheirfirstattempt(74%in2008-2009,89%in2009-2010,83%in2010-2011,82%in2011-2012,and78%in2012-2013).Thereissomeuncertaintywiththisyear’sreportingoftheLanguageArtscontentexam.Allthatisknownfromthedatathatwasreceivedisthatatotalof12studentstookthisexamand1studenthasyettopassthisexam.ThereforenocalculationcanbemadeonhowmanyIdLSstudentspassedthePRAXISIIexamonfirstattempt(excludinglanguageartsfromthiscalculationwecancalculatethat72%ofstudentsfromallotherareaspassedthisexamonfirstattempt).

ST-9DATA(ItemA2,“Identifieskeyprinciplesandconceptsofsubjectmatter”)ST-9ispartofthe“AssessmentofStudentTeaching”conductedbytheCOEatJMU.Thisform(seeAppendix1),titled“PROFILEOFSTUDENTTEACHINGPERFORMANCE”isfilledoutbythecooperatingteacheranduniversitysupervisorwhiletheIdLSstudentisStudentTeaching.BoxA2ofthisform,pertainstotheabilityoftheSTUDENTTEACHERtoIDENTIFYKEYPRINCIPLESANDCONCEPTSOFSUBJECTMATTER.Ascoreof:

• 3.0meansthatthestudentteacherexplicitlyreferencesANDclearlyalignsappropriatecontentstandardswithplannedactivitiesandassessments,

• 2.0meansthatthestudentteacherexplicitlyreferencesappropriatecontentstandardsindailyplans.

• 1.0meansthatthestudentteacherinaccuratelyandvaguelyreferencesORdoesnotreferenceappropriatecontentstandards.

38

ForSpring2015,atotalof294evaluationsweremadeusingtheST-9instrumentwiththefollowingstatisticalresults(unfortunatelydataforFall2014wherenotsenttoustoincludeinthisanalysis).Duringthisreportingperiod,candidateswereratedbyasupervisorandaclassroomteacher,andtypicallydotwoplacementsperyear,thereforetheremaybeuptofourratingsforanyonecandidate.

ST-9Analysis 2009-10 2010-11 2011-12 2012-13 Fall2013 Spring2014 Spring

2015AverageScore

2.9 2.94 2.93 2.95 2.91 2.88 2.95

High 3.0 3 3 3 3 3 3

Low 1.5 1.5 1 1 2 1.5 1.5

StandardDeviation

0.26 0.22 0.22 0.17 .13 .18 .18

N 345 208 190 751 189 258 294

Table8A.ST-9scoresfor2008-2009to2013-2014.

In2008-200984%ofstudents(ElEDandMiddleSchool)scoreda3.0onitemA-2oftheST-9,meaningthat84%ofthestudentsdemonstratedthehighestlevelofmasteryofcontentknowledgeintheirclassrooms.In2009-2010,86%ofElEDandMSEDstudentsachievedthishighestlevelofmastery.FortheSpringandFallsemestersof2011,dataisavailablethatsplitstheElEDandMSEDstudentsintoseparategroups.Forthetimeperiodof2011to2012,95%ElEDmethighestlevelofmastery,while81%ofMSEDalsometthishighestlevelofmastery.Forthereportingperiod(2012-13)93%ELEDmetthehighestlevelofmastery,while78%ofMSEDalsometthishighestlevelofmasteryasdemonstratedintheclassroom.Forthisreportingperiod(Spring2015)93%ELEDmetthehighestlevelofmastery,and100%ofMSEDalsometthishighestlevelofmasteryasdemonstratedintheclassroom.

39

RESULTSFromthedatapresentedhereforthe2015-2016reportingperiod,itappearstheIdLShasmeteachofitsprogramgoals.

• FromtheCluster1,3and4dataanalysisitappearsthatthereisnosignificantdifferencebetweenIdLSandnon-IdLSstudents.Asaresult,IdLSstudentshave,asagroup,matchedotherJMUstudentsonGeneralEducationlearningoutcomes(specificallyintechnology,informationliteracy,andtheglobalandAmericanexperiences).

• FromPRAXISIIdata,itappearsthateacharea(ELED,MiddleSchoolMath,MiddleSchoolScience,MiddleSchoolEnglish,andMiddleSchoolSocialStudies)isperformingbetterthanthenationalaverages.Forthisyears’cohort,IdLSachievedapassrateof97%orbetterforElEdinall4subjectareas(Science,Math,LanguageArts,andSocialStudies).Whiledatawasnotavailabletoassessthepassrateforthisyears’cohortfortheMiddleSchoolareas,weareconfidentthatitisnotsignificantlydifferentfromthepreviousyearsforwhichdataisavailable.Asaresult,webelievethatwehavepassrateof~90%orbetterforallMiddleSchoolareas.Bothscoresarebetterthantheprogramtargetof80%.

• FromST-9data,100%ofstudentsachievedanadequatelevelofcontentproficiencyasdemonstratedintheclassroom.Thisisagainbetterthanthetargetof80%.

Whilemeetingtheseassessmentgoalsismeaningful,thereareotherrecommendationsthatwecanmakebasedonthecombinedresultsoftheseassessments.ThefollowingisalistofrecommendationstobedisseminatedtothevariousconstituenciesinIdLS.

1. AllPRAXISIIdataforthisreportingperiodneedstobecautiouslyexaminedandresultsneedtobecarefullyinterpreted.Thereare3differentPRAXISIIexamsusedforElEdassessmentinthepast2years,withthecurrenttestonlybeingusedsince7/1/15.Theexamisstillbeingnormedmeaningthatspecificresultsforthisyears’testmaynotgiveanaccurateindicationofabsolutestrengthsandweaknessesoftheprogram.

2. Sciences:ElementaryEducationPRAXISIItestresultsindicatethatstudentsarebeingadequatelypreparedtopassthisexam(Tables7Gand7H).Infactforthecurrentreportingperiod98%ofstudentspassedthisexam(Table7K)andmorethan60%ofJMUstudentswhotookthisexamplacedinthehighest2quartilesinforLifeandPhysicalSciencescomparisontothenationalaverage(Table7H).However,barelyover50%scoredinthehighest2quartilesforEarthScience.ForMiddleEducation,thecontentareaofScience,Technology,andSociety(STS)hascontinuallyshowntobeanareaofpoorperformance(Table7J).ThisistheseventhyearinarowthatSTShasbeenalowperformeronPRAXISII.Itwashopedthatanewclass(ISAT495)thatwasdevelopedsixyearsagowasgoingtohelpimprovethisarea,butwearestillseeinglowperformancenumbers(51%ofJMUstudentswhotookthistestplaceinthelowest2quartiles).InFall2013,theIdLSSteeringCommitteeconsideredthisissueintheFallof2013andbegandiscussionswithfacultyandadministratorswhodevelopedthisclassinhopesoffindingwaystoimprovestudentperformanceinthisPRAXISIIarea.Thesediscussionscontinuedintosummerof2014withtheDeanandDepartmentHeadofCISEandISAT.AsaresultitwasdeterminedthattheScience,Technology,andSocietycontentareawouldbeacoursewhosecontentwouldbedistributedacrosstheIdLSCore.

40

StudentperformanceinEarthandSpaceSciencehasvariedconsiderablyoverthelastfewyears.Thisyearstudentperformanceinthisarealookstobeweakerthanyearspast.WebelievethatthecontinueddevelopmentofanewastronomycoursestartedinAY2012-2013byDr.GearyAlbright(whichtooktheplaceofASTR301(SearchingforLifeintheUniverse))willcontinuetoimprovethecurriculumfortheMiddleEdSpaceandPlanetarySciencerequirements.Also,Dr.JenniferMangancontinuestodevelopanewWeatherandClimatecourseforIdLScoursethatwilltaketheplaceoftheexistingmeteorologycourse,againaddresstheMiddleEdlearningobjectivesforthisarea.Additionally,duringAY2013-2014,discussionswerestartedwithfacultyteachingtheEarthScienceforTeachersclasstore-examinethecontentofthiscourseanditsalignmentwithPRAXISIIlearningobjectives.WearehopefulthatthesechangeswillconcludeinAY2015-16andwillultimatelyimprovethisarea.AcrossallscienceareastheIdLSSteeringCommitteehasactivelymettodiscusseffortstore-developtheIdLScorescienceclasses(ISCI171-173).AquestionnairewascreatedanddeployedinSpring2016andiscurrentlybeinganalyzed.ResultswillbeusedinAY2015-2016torealigntheCoreSciencecurriculumwithnewNextGenerationScienceStandards.

3. SocialStudies:ItappearsstudentsarebeingwellpreparedforElementaryEducationinSocialStudieswith99%ofstudentspassingthisexam(Table7K)andmorethan60%ofstudentsplacinginthetop2quartilesnationallyforallsubareasofthisexam(Table7F).Toimprovethecurriculumevenmoreinthisarea,theIdLSsteeringcommitteeisworkingwithfacultyintheSocialStudiesareatopotentiallymodifythestructureoftheconcentrationtoinclude“tracks”whichwouldallowforamorefocusedconcentrationofcurriculumforstudents.

4. LanguageArts:ItappearsstudentsarebeingwellpreparedforElementaryEducationin

LanguageArtswith99%ofstudentspassingthisexam(Table7K)andmorethan60%oftop2quartilesnationallyforallsubareas(exceptfortest#5002areaofWriting,Speaking,andListening).BaseduponPRAXISIIdatatherehaveneverbeenanymultipleyeartrendsinthedatathatwouldindicateaconsistentlyweakpartoftheMiddleEducationprogramforLanguageArts.Toimprovethecurriculumevenmoreinthisarea,theIdLSsteeringcommitteeisworkingwithfacultyintheLanguageArtsareatodevelopmorecoursesfocusedonwriting.

5. Mathematics:Accordingtoourassessmentcriteria,themathcurriculuminIdLShasbeenthe

strongestcontentareas.AllcoursesweredesignedfromtheNCTMstandards,andthestudentsalltakethesamecoreandconcentrationcourses.ItappearsstudentscontinuetobewellpreparedforElementaryEducationinMathematicswith97%ofstudentspassingthisexam(Table7K)andthatmorethan70%ofstudentswhotookthePraxisIIElementaryContenttestscoreinthetoptwoquartilesnationally(Table7D).TherearenomultipleyeartrendsinthedatathatwouldindicateaconsistentlyweakportionoftheMiddleEducationprogramforMathematics.

PartV.Dissemination

41

TheAnnualAssessmentReportisprovidedtotheprogramdirector(FletcherLinder)anddiscussedwithbothrepresentativesfromMath/Science/TechnologyandHumanities/LanguageArtswhoserveontheIdLSExecutiveCommittee.Substantialprogresshasbeenmadeoverthepastseveralyearsandthisisanticipatedtocontinueuntilamatureassessmentprogramhasbeendeveloped.TheIdLSExecutiveCommitteereceiveanddiscussthisassessmentinformationandspecificinstrumentresultsaresharedwithrelevantareacoordinatorsandfaculty.ResultsarealsosharedwiththeCoEunitassessmentcommitteeandtheCoEAssessmentDirector(AmyThelk)aswellasseveralotherjointIdLS/CoEgroups.Finally,IdLSprovidesthisreport(orpartsofthereport)toanyotherprogramorindividualwhoexpressesinterestintheseassessmentresults.ForinstanceinthepastyearwehaveprovidedthisreporttotheDean’sofficeofCSMandCISE,DepartmentHeadsofISATandGeology,andseveralfacultyteachingintheprogram.PARTVI.UsesofEvaluation/AssessmentResultsandActionsTakenSeveral specific actions have been taken as a result of assessment results. Most of these are discussed in the previoussections.Afewofthemostsignificantactionsaresummarizedhere.

1. DatafromallpreviousyearsAPT’swereextensivelyusedinpreparingtheIdLSAcademicProgramReviewmaterials.TheIdLSexternalteamvisitedthisacademicyearandwereimpressedwithourcurriculumandourassessmentprogram.

2. TheIdLSsteeringcommitteeisworkingwithfacultyintheLanguageArtsareatomodifythecurriculumtoincludemorecoursesfocusedonwriting.

3. TheIdLSsteeringcommitteeisworkingwithfacultyintheSocialStudiesareatopotentiallymodifythestructureoftheconcentrationtoinclude“tracks”whichwouldallowforamorefocusedconcentrationofcurriculumforstudents.

4. InresponsetomultipleyearsofpoorstudentperformanceonthePRAXISIIMiddleEducationScience,sub-areaScience,technology,society,TheDirectorofIdLSandtheCoordinatorofIdLSMathematics/Science/TechnologymetwiththeDeanoftheCollegeofIntegratedScienceandEngineeringandtheDepartmentHeadofISATinSummerof2014todiscussIdLSassessmentanditimplicationsforcourserigorandimprovementofcontentfocusforthecourseISAT495.

5. WithattentionpaidtoIdLSassessment,theDirectorofIdLSandtheCoordinatorofIdLSMathematics/Science/TechnologymetwiththefacultyteachingtheScienceCorecourses(ISCI171,172,173)duringthepasttwoyears.Thesemeetingswereaimedatredesigningandre-aligningthesecoursesinpreparationfornewsciencelearningobjectivesbeingdevelopedbytheVDOE.WeanticipatethatthisworkwillconcludeduringAY2015-2016.

6. TheDirectorofIdLSandtheCoordinatorofIdLSMathematics/Science/TechnologymetwiththeDeansoftheCollegeofScienceandMathematicsinFallof2013todiscussIdLSassessmentanditimplicationsforcourserigorandimprovementofcontentfocusforScienceandMathclasses.

7. MiddleGradescurriculumwas,andcontinuestobe,revised.Specifically,inresponsetolowPRAXISIIscoresforMiddleEducationScience,sub-areaEarthandSpaceScience,anewAstronomycoursewaspilotedinFall2012.AdditionallyinFall2011,anewcourseentitledOceanographyforTeacherswastaughtinplaceofanon-teachersOceanographycourse.In2013anewcourseinWeatherandClimatewasdevelopedandtaught.FacultyteachingEarthScienceforTeachersstartedevaluatingcurriculumandwillrevisecoursecontentduringthe2015-2016academicyear.

42

8. OngoingimprovementinIdLS400basedonannualfacultydiscussionofcoursedesignandimplementation.Thisisespeciallyusefultonewfacultyandguaranteesconsistencyacrosssectionsandyears.

9. Increasedtransparencyofadvisingandscheduling,andenhancedcooperationbetweenCOEandIDLStofacilitateschedulingandsequencingofconcentrationcoursesbasedonformalandinformalsurveysofstudentsandfaculty.

10. Chemistry,STSclasses,worldhistorycourses,andmiddleeducationsciencerequirementswereallchangedinresponsetoassessmentresults.

11. IdLS400pilotedasectionwhichincludesscienceandmathematicscontentin2009-10andagainin2011-12toprovideamorefullyintegrativecontentcourseforstudents.

12. Goals,ObjectivesandMeasuresweremodifiedin2009-2010basedonpreviousyearsAPTreports.

13. ImprovedcooperationbetweenCARSandIdLStoassuredataanalysisinatimelymanner.

43

APPENDIX1.ST-9