Funcons of Standardized Tests - NC State: WWW4 Serverjlnietfe/EDP560_Notes_files/Achievem… ·...

12/7/09

1

Func,onsofStandardizedTests

•  StudentAssessment•  Diagnosis

•  PlacementandSelec3on

•  Accountability•  Predic3veValidity

Whatisastandardizedtest?

•  Atestthathasstandardproceduresforadministra3on,scoring,andinterpreta3on.

12/7/09

2

AdvantagesofStandardizedTests

•  Evalua3ngstudents’generaleduca3onaldevelopmentinthebasicskills&inlearningoutcomescommontomanycoursesofstudy

•  Evalua3ngstudentprogressduringtheschoolyearoroveraperiodofyears

•  Determiningstrengths&weaknesses

WeaknessesofStandardizedTests

•  Evalua3ngthelearningoutcomesandcontentuniquetoapar3cularclassorschool

•  Evalua3ngstudents’day‐to‐dayprogress•  Evalua3ngknowledgeofcurrentdevelopmentsinrapidlychangingcontentareassuchasscienceandsocialstudies

12/7/09

3

TypesofAchievementTests

•  StateDevelopedTests N.C.EOG’s,FLFCAT’s,VA’sSOL’s

•  Publisher‐DevelopedNorm‐referencedba:eries ITBS,CAT,GRE,Stanf.Ach.Test

•  Publisher‐DevelopedNorm‐referencedcontentareatests

Nelson‐Denny,Gates‐McGini3e•  Publisher‐DevelopedCriterion‐referencedtests

Somepopularteststobeawareof:

•  CaliforniaAchievementTest(CAT)•  IowaTestsofBasicSkills(ITBS)•  MetropolitanAchievementTests(MAT)

•  StanfordAchievementTests

•  TerraNova

12/7/09

4

Whydoweusethese?

•  Hightechnicalquality•  Standarddirec3onsforadministra3onandscoring

•  Normsbaseduponna3onallargena3onalsamples

•  Equivalentforms

•  Comprehensivemanuals

UsingAchievementTests:

•  Bewaryofusingsubtestsfordiagnos3cpurposesunlessenoughitemsareincluded

•  Whatisanormgroupandwhatisthebenefitofhavinganormgroup?

•  Normgroupsprovideastandardframeofreference

•  Equivalentforms

12/7/09

5

JudgingtheAdequacyofNorms

•  Shouldberelevant(Whodoyouwanttocompareyourscoresto?)

•  Shouldberepresenta3ve•  Shouldbeuptodate•  Shouldbecomparable

•  Shouldbeadequatelydescribed

AchievementBaOeries

•  Consistsofaseriesofindividualtestsallstandardizedonthesamena3onalsample

12/7/09

6

Wheretofindinfoontests:

•  MentalMeasurementYearbook(MMY)•  MMYpublishedbyBurosCenterforTes3nghOp://www.unl.edu/buros/

SomeAp,tude/Intelligenceteststobeawareof:

•  WechslerIntelligenceTests(WISC,WAIS)•  Stanford‐Binet•  Raven’sAdvancedProgressiveMatrices•  Cogni3veAbili3esTest(CogAT)•  GraduateRecordExamina3on(GRE)•  O3s‐LennonSchoolAbilityTest(OLSAT)•  CaYellCulture‐FairIntelligenceTests•  ArmedServicesVoca3onalAp3tudeBaYery(ASVAB)

•  Differen3alAp3tudeTest(DAT)

12/7/09

7

Ap,tudeTests

•  Donotmeasurefixedcapacitybutratheradifferenttypeofabilityusedtopredictfutureperformance

•  Commondis3nc3on:achievementtestsmeasurewhatastudenthaslearnedandthatap3tudetestsmeasuretheabilitytolearnnewtasks

Whyuseap,tudetestswhenyouhaveachievementtests?

•  Canbeadministeredinarela3velyshort3me•  Canbeusedwithstudentsofmorewidelyvaryingeduca3onalbackgrounds

•  Canbeusedbeforeanytrainingorinstruc3on

12/7/09

8

Specifictheoriesofimportance:

•  Spearmanvs.Thurstone•  Guilford’s120abili3es•  Crystallized&Fluidintelligence•  Gardner’smul3pleintelligences

•  OthersincludeDavidPerkins&RobertSternberg

Whatisgeneralintelligence?

•  Generalabilitytypicallymeasuredviastandardizedtests‐‐symbolizedasg

•  Predic3vepowerstrongestwhenfacingnoveltasksorbeginningcompetence

•  Consideredtobereasoningability(typicallyinduc3ve)thatishighlydependentuponworkingmemory‐‐makingtransforma3onsinyourhead

12/7/09

9

Normal distributions (bell shaped) are a family of distributions that have the same general shape. They are symmetric (the left side is an exact mirror of the right side) with scores more concentrated in the middle than in the tails. Examples of normal distributions are shown to the right. Notice that they differ in how spread out they are. The area under each curve is the same.

If your data fits a normal distribution, approximately 68% of your subjects will fall within one standard deviation of the mean.

Approximately 95% of your subjects will fall within two standard deviations of the mean.

Over 99% of your subjects will fall within three standard deviations of the mean.

12/7/09

10

The mean and standard deviation are useful ways to describe a set of scores. If the scores are grouped closely together, they will have a smaller standard deviation than if they are spread farther apart.

Small Standard Deviation Large Standard Deviation

Different Means Different Standard Deviations

Different Means Same Standard Deviations

Same Means Different Standard Deviations

When you have a subject’s raw score, you can use the mean and standard deviation to calculate his or her standardized score if the distribution of scores is normal. Standardized scores are useful when comparing a student’s performance across different tests, or when comparing students with each other.

z-score -3 -2 -1 0 1 2 3

T-score 20 30 40 50 60 70 80

IQ-score 65 70 85 100 115 130 145

SAT-score 200 300 400 500 600 700 800

12/7/09

11

The number of points that one standard deviation equals varies from distribution to distribution. On one math test, a standard deviation may be 7 points. If the mean were 45, then we would know that 68% of the students scored from 38 to 52.

24  31 38 45 52 59 63 Points on Math Test

30 35 40 45 50 55 60 Points on a Different Test

On another test, a standard deviation may equal 5 points. If the mean were 45, then 68% of the students would score from 40 to 50 points.

Length of Right Foot

8 7 6 5 4 3 2 1

4 5 6 7 8 9 10 11 12 13 14

Data do not always form a normal distribution. When most of the scores are high, the distributions is not normal, but negatively (left) skewed.

Num

ber o

f Peo

ple

with

th

at S

hoe

Size

Skew refers to the tail of the distribution.

Because the tail is on the negative (left) side of the graph, the distribution has a negative (left) skew.

12/7/09

12

Length of Right Foot

8 7 6 5 4 3 2 1

4 5 6 7 8 9 10 11 12 13 14

Num

ber o

f Peo

ple

with

th

at S

hoe

Size

When most of the scores are low, the distributions is not normal, but positively (right) skewed.

Because the tail is on the positive (right) side of the graph, the distribution has a positive (right) skew.

When data are skewed, they do not possess the characteristics of the normal curve (distribution). For example, 68% of the subjects do not fall within one standard deviation above or below the mean. The mean, mode, and median do not fall on the same score. The mode will still be represented by the highest point of the distribution, but the mean will be toward the side with the tail and the median will fall between the mode and mean.

Negative or Left Skew Distribution Positive or Right Skew Distribution

12/7/09

13

StandardScores

•  Acalculatedscorethatenablesaresearchertocomparescoresfromdifferentscales

•  Z‐scoremostpopular(meanofzeroandSDofone)

•  z=(X‐M)/SD

12/7/09

14

SomePopularStandardScores

TScores

•  Anothertypeofstandardscore‐‐some3mespreferredbecauseallnumbersareposi3ve

•  Tscore=50+10(z)•  Example:Ifyouscored2SD’sabovethemeanonareadingtestyourTscorewouldbe50+10(2)=70

12/7/09

15

Stanines

•  Anothertypeofstandardscore‐‐preferredforinterpretability

•  Scorebetween1and9witheachstaninecovering1/2standarddevia3onunit(e.g.,stanineof5=40%ile‐59%ile)

AvoidMisconcep,onswithGradeEquivalentScores

•  Don’tconfusenormswithstandardsofwhatshouldbe

•  Don’tinterpretagradeequivalentasanes3mateofthegradewhereastudentshouldbeplaced

•  Don’texpectthatallstudentsshouldgain1.0gradeequivalenteachyear

•  Don’tassumethattheunitsareequalatdifferentpartsofthescale

12/7/09

16

AvoidMisconcep,onswithGradeEquivalentScores

•  Don’tassumethatscoresondifferenttestsarecomparable

•  Don’tinterpretextremescoresasdependablees3matesofstudent’performancelevel

Na,onalTes,ngProgram:NAEP

•  NAEP–Na3onalAssessmentofEduca3onalProgress

•  Formulatedinthe1960stoprovidebenchmarksofeduca3onalaYainment

•  Nowtestsatgrades4,8,and12•  Subjectareaschangedependingonyear•  Includesmul3ple‐choiceandopen‐endedtestitems