Day 2, Evening: UnivariateTests of Statistical Inference · Day 2, Evening: UnivariateTests of Statistical Inference Instructor: Marshall A. Taylor 844 Flanner Hall [email protected]

Day2,Evening:Univariate TestsofStatistical

InferenceInstructor:MarshallA.Taylor

[email protected]

Recap• Thismorningwetalkedabouthowtheasymptotictheoryofprobabilitydistributionsallowsustoassesshowrepresentativeoursampledistributionisofthepopulationdistribution.

• Wespecificallylookedathowwecanquantifythestandarddeviationofthesamplingdistributionofsamplemeans(thestandarderror),whichissmallerwhenwehavelargersamplessizes.Thesmallerthestandarderror,themorereliableourestimates—assumingthesamplingdistributionisnormalpertheCLT.

Recap

• Wethenusedthestandarderrorandourassumptionsofthesamplingdistributiontocreateconfidenceintervalsaroundthepopulationmean.

• Finally,wetalkedaboutz-scores—whichtransformanobservation’svalueonx sowecanseehowmanystandarddeviationsitisfromthevariablemean.

Butwhataboutrandomchance?• Standarderrorsandconfidenceintervalsaroundthemeangiveusagenerallyideaofhowefficientoursamplemeanmightbe.

• Buthowdewereallyknowwhetherornotoursamplemeanisduetosamplingerror?

• Forexample,ifoursamplemeanisdifferentfromaknownpopulationmean,howdoweknowwhetherornotthisdifferenceispotentiallyrealorjustaproductofrandomchance?

HypothesisTesting• Answeringaquestionsuchasthisnecessitateshypothesistesting.

• Ahypothesis isastatementabouttheexpectedpropertyofavariableortherelationshipbetweenmultiplevariables.

• Wetest ahypothesisbycomparingtheseexpectationstoobserveddatatodeterminewhetherornottheseexpectedpropertiesorrelationshipsaregeneralizabletothetargetpopulation.

HypothesisTesting

• Suchatestfirstrequiresstatingbothanullhypothesisandanalternativehypothesis.

• Anullhypothesis (denotedH0) isastatementthatistestedtodeterminewhetheritspremiseshouldbeacceptedorrejected.Instatistics,H0 istypicallythatthereisnostatisticallysignificantdifferencebetweentwonumbers.

HypothesisTesting

• Suchatestfirstrequiresstatingbothanullhypothesisandanalternativehypothesis.

• Analternativehypothesis (denotedHA)isthehypothesisthatweeitherfindsupportforornotdependingonwhetherH0 isrejectedoraccepted.Thealternativehypothesisisoftenassimpleasthepropositionthatthereisastatisticallysignificantdifferencebetweentwonumbers.

HypothesisTesting

• Forexample,ifwearecuriousifthehypothesizedmeanheightofNDundergrads(x) findssupportgivenameanfromarandomsampleofNDundergrads,thenournullhypothesiswouldbe:

H0 : µ = x

HypothesisTesting• Ifwethinkthatthepopulationaverageisactuallytallerthantheonepreviouslyhypothesized,thenouralternativehypothesiswouldbe:

HA : µ > x

• Ifwethinkitisshorter,thenHA wouldbe:

HA : µ < x

HypothesisTesting

• OrwemightbesatisfiedwiththeHA thatthepopulationaverageissimplydifferentformtheonehypothesized—regardlessoftheparticulardirectionofthedifference.Inthiscase,HA wouldbe:

HA : µ ≠ x

HypothesisTesting

• Inhypothesistesting,wetestH0—not HA!– IfH0 isaccepted,thenwedonotfindsupportfor

HA assomethingthatishappeninginthepopulation.

– IfH0 isrejected,thenwedofindsupportforHA assomethingthatishappeninginthepopulation.

HypothesisTesting

• SoweunderstandthisH0 andHA business.ButhowdowegoaboutactuallytestingH0?

• Forexample,howdoweknowwhetherornotthepopulationheightissignificantlydifferentfromtheonehypothesized?

Backtothetheory!

• Luckily,wehavetheCLTandthosecriticalvaluestohelpusoutagain!

• WecanusetheCLTtohelpusassesshowlikelywearetoobserveHA assumingthatH0 istrue.IfwearenotveryliketoobserveHA whenassumingH0 istrue,butweobserveitanyway,thenwerejectthepremiseofH0 andinsteadfindsupportforHA assomethingthatmaybehappeninginthepopulation.

Backtothetheory!

Thoughwehavetalkedaboutthesamplingdistributionofsamplemeansuptothispoint,theCLTactuallyholdsforanyrandomvariable—standarddeviations,differences,whatever!

Backtothetheory!

• Ifwewanttoknowwhetherornotasamplemeanisdifferentfromapopulationmean,thenweareinterestedinthedistributionofmeandifferences.

So,ifweknowthat95%ofsamplemeandifferencesfallwithin±1.96standarddeviationsofthe“real”meandifference(whichwewillassumetobe0),then,bywayofthecomplementsrule,wealsoknowthat5%donot.Soifourobservedsamplemeandifferencefallsabove1.96orbelow–1.96,thenwerejectH0 becausethereislessthana5%chancethatwewouldobserveoursamplemeandifferenceifH0 weretrue.

AnExample:TheOneSamplet-test• Thinkbacktoourheightexample.Let’ssaythemeanheightinoursampleis67inches(about5’6”).Nowlet’shypothesizethemeanheightofpopulationtobe63inches(about5’4”).Thereareelevenofyou,andlet’ssaythesamplestandarddeviationis3inches.

• OurH0 wouldbethatthereisnodifference betweenthetwomeans(H0 : µ = 63).

• OurHA wouldbethat thepopulationheightistallerthantheonehypothesized(HA : µ > 63).

AnExample:TheOneSamplet-test• Weonlyhaveonesamplestatisticthatweareworkingwith:themeanheightinthesample.Assuch,wecantestournullhypothesisusingwhatisreferredtoasaonesamplet-test.

• Further,sincewearehypothesizingthatthedifferenceisinaparticulardirection,wesaythatthisisaone-tailedtest.Ifwewerenothypothesizingaparticulardirectionforthedifference,thiswouldbeatwo-tailedtest.

AnExample:TheOneSamplet-test

• Theonesamplet-testdirectlyassessesourH0thatthereisnostatisticallysignificantdifferencebytakingthedifferencebetweenthesampleandpopulationmeanandscalingthisvaluebythestandarderror:

• Theresultiswhatisreferredtoasthetstatistic.


• Thisstatisticisourteststatistic.Butwhatdowedowithitoncewe’vegotit?

AnExample:TheOneSamplet-test• Wehavetocompareourteststatistic(at-statistic,inthiscase)to

thecriticalvalueassociatedwithourdesiredconfidencelevelinwhatiscalledthet-distribution (whichisverysimilartoanormaldistribution,but“penalizes”youforhavingsmallsamplesizesandthereforefewerdegreesoffreedom).

• Solet’ssaythatweonlywanttohavea.05probabilityofbeingwrong.Thecriticalt-valueassociatedwiththis(one-tailed)probability,giventhatwehave10degreesoffreedom(11– 1),is1.81.

• BTW:This“.05probabilityofbeingwrong”isknownasoursignificancelevel,alsoknownasthealphalevel(α).


Ifourteststatisticfallsabove1.81,thenwecanrejectH0 becauseourstatisticdoesnotfallwithinthebottom95%ofthenormaldistributionifweassumethe“real”meandifferenceis0!

AnExample:TheOneSamplet-test• IfwecouldrejectH0,thenourteststatistic(thet statistic,in

thiscase)wouldfallintotherighttailofdistribution,asshownbelow(withthedifferencebeingthatthecriticalvalueis1.64asopposedto1.81).

*ImagefromPsychStatisticswebsite(http://www.psychstatistics.com/2010/11/24/stata-graphing-distributions/).


• Solet’scrunchthenumbersforourheightexample:

AnExample:TheOneSamplet-test• Ourteststatisticis4.42.Thisismuchlargerthan1.81.

• Assuch,wecanrejectthenullhypothesis—atthe.05level—thatthehypothesizedpopulationmeanheightis63inches.Maybeweshouldthereforereconsiderthe“real”valueofthepopulationmean(i.e.,allNDundergrads).

• Ifweareabsolutelysurethatthepopulationmeanheightis63inches,thenwecanatleastsaythatitisveryunlikelythatoursamplemeancamefromapopulationwithameanof63.Perhapsoursampleisnotentirelyrandom,ormaybewejustgotabadsample.


• ConfirmwithStata:

.

Pr(T < t) = 0.9994 Pr(|T| > |t|) = 0.0013 Pr(T > t) = 0.0006 Ha: mean < 63 Ha: mean != 63 Ha: mean > 63

Ho: mean = 63 degrees of freedom = 10 mean = mean(x) t = 4.4222 x 11 67 .904534 3 64.98457 69.01543 Obs Mean Std. Err. Std. Dev. [95% Conf. Interval] One-sample t test

. ttesti 11 67 3 63, level(95)

Let’stryanotherone• Arandomsampleoffivecongressionalcandidatesfrom

2006raised,onaverage,$1,964,018.Nowlet’ssaythattheaveragefundsraisedacrossallcongressionalcandidatesinthecountryin2006washypothesizedtobe$5,000,000.*Further,thesamplecandidatesvaryfromtheirmeanof$1,964,018byabout$925,149.Atα =.05,howlikelyisitthatourhypothesizedmeanvaluereflectsthepopulationparameter(regardlessofdirection)?

• Tip:thecriticalt-valuewithfourdegreesoffreedom(5candidates– 1)is±2.776.

*Madeupnumber.

Let’stryanotherone• Firststateyournullhypothesis:

H0 : µ = $5,000,000

• Thenthealternativehypothesisthattheyaredifferent:

HA : µ ≠ $5,000,000

• Thisisatwo-tailedtest,sincewearenotassumingaparticulardirection ofthedifference.Thusthe≠ ratherthanthe< or>!

Let’stryanotherone

• Specifically,werejectH0 ifourteststatisticfallsabove2.776orbelow–2.776,asillustratedbytheshadedregionsinthedistributionbelow:

FigurecreatedusingStudent’st-distributionappletattheUniversityofIowa(http://homepage.divms.uiowa.edu/~mbognar/applets/t.html).


• Rememberourformula:

• So:

Let’stryanotherone• Ourteststatisticis–7.338.Thisissmallerthanourcriticalvalue,–

2.776.

• So,atthe.05significancelevel,werejectthenullhypothesis thatthepopulationmeanfundsraisedis$5,000,000.

• Instead,wefindstatisticalsupportforouralternativehypothesisthatthepopulationmeanfundsraisedisnot $5,000,000.

• Ifwearesurethatthepopulationmeanis$5,000,000(i.e.,itismorethanahypothesizedmean),thenwecanatleastsaythatitisveryunlikelythatoursamplemeancouldhavecomefromapopulationwithameanof$5,000,000.


• Nowlet’ssaywewantedtoknowifthedifferenceissignificantinaparticulardirection.Specifically,wewanttoknowifthepopulationmeanissmallerthantheonehypothesized.

• Inthiscase:HA : µ < $5,000,000

Let’stryanotherone• Ournewcriticalvalueis–2.132.Thisvalueissmallerthanthe

onefromourtwo-tailedtest,meaningthatwehavemorepowertorejecttheH0 (becausetheshadedregionisbigger).However,itcomesattheexpenseofnotbeingabletodetectsignificantteststatisticsattheotherendofthedistribution.

FigurecreatedusingStudent’st-distributionappletattheUniversityofIowa(http://homepage.divms.uiowa.edu/~mbognar/applets/t.html).


• So–7.338issmallerthanourcriticalvalueof–2.132.

• WecanonceagainrejecttheH0.

• However,wecannowsaythatitislikely(atthe.05significancelevel)thattherealpopulationmeansislessthanthehypothesizedpopulationmean.

ButletStata dothework!

• Luckily,Stata canhandlethis.

. univar FRAISED if Y2006==1 & STATE2==14


Ho: mean = 5000000 degrees of freedom = 4 mean = mean(FRAISED) t = -7.3379 FRAISED 5 1964018 413739.2 925149 815293.9 3112742 Variable Obs Mean Std. Err. Std. Dev. [95% Conf. Interval] One-sample t test

. ttest FRAISED==5000000 if Y2006==1 & STATE2==14







Wegotourteststatisticcorrect.







Andbothourtwo-tailedandone-tailedtestswerecorrect.Thep-valuefortheone-tailedtesttellsusthatthereisonlya.09%chancethatoursamplemeanwoulddifferbyatleastthismuchfromthepopulationwhenweassumethatthe“real”meandifferenceiszero.







Butnoticeourone-tailedtestintheotherdirectionwouldhavegivenusanon-significantresult.

Othertests• Butsometimesweneedtoteststatisticsotherthanmeans.Remember,meansaren’talwaysanappropriatemeasureofcentraltendency!

• Sometimeswehaveavariablethatcanonlytakeontwocategories.Inthiscase,wecanusetheonesampledifference-of-proportionstest.

• Othertimesthemedianisabettermeasureofcentraltendency.Inthiscase,wecanusetheonesample signtest.

OneSampleDifference-of-ProportionsTest

• Verysimilartotheonesamplet-test,butforadichotomousvariableinsteadofamean.

• Theteststatisticcanbefoundwith:

• Noticetheteststatisticisaz-statisticinsteadofat-statistic.Thissimplymeansthatweareassumingasamplingdistributionthatfollowsanormaldistributioninsteadofat-distribution(which,withalargeenoughsample,convergetoessentiallybesamething).

AnExample:TheOneSampleDifference-of-ProportionsTest

• Alandlordatabusydowntownapartmentbuildingtookapollandfoundthatonly3%ofresidentsareinfavorofrequiringparkingpasses.Youquestionthelandlord’sconclusionandtakearandomsampleof50residents.Youfindthat,amongtheseresidents,23areinfavorofrequiringparkingpasses—46%ofyoursample.Howvalidisthelandlord’sconclusion,assumingthatyouareonlywillingtobewrong5%ofthetime?


• Ournullhypothesis:

H0 : µ = .03

• Ifwetakeonaone-tailedtest,ouralternativehypothesiswouldbe:

HA : µ > .03

.


• Ourcriticalz-valueis1.645.Wechoosethisvalueinsteadof±1.96becauseweareperformingaone-tailedtest.So,insteadofrejectingH0 ifourteststatisticfallswithinthetopandbottom2.5%ofthenormaldistribution(bottomleftplot),welookatthetop5%only(bottomrightplot).

.

*FiguremadewithNormalDistributionCalculatoratOnlineStatsbook(http://onlinestatbook.com/2/calculators/normal_dist.html).


• Nowlet’sdothemath:

• Ourteststatisticismuchlargerthanourcriticalvalueof1.645.Assuch,thereislessthana5%chancethatwewouldfind46%of50residentsinfavorofparkingpassesifitweretruethatonly3%oftotalresidentswereinfavorofparkingpasses.Weconcludethatmoreresidentsareinfavorofthispolicyatthe.05significancelevel.

.

AStata Example• Weknowthat52%ofoursampleofHousecandidatesin2006“wentnegative”—thatis,theyengagedinnegativecampaigningagainsttheiropponent.Let’ssay(hypothetically)thatpreviousstudiessuggestthat,onaverageandacrosstime,Housecandidatesgonegativeabout55%ofthetime.Ifoursampleconsistsof227candidatesandisindeedarepresentativerandomsampleofHousecandidatesacrosstime,canwesaythatHousecandidatesactuallygonegativelessthan55%ofthetime?

AStata Example

• Wecannotrejectthenullhypothesisatthe.05levelthatHousecandidates,onaverageandacrosstime,gonegative55%ofthetime.

. display (.5242291-.55)/sqrt((.55(1-.55))/227)

Pr(Z < z) = 0.2176 Pr(|Z| > |z|) = 0.4351 Pr(Z > z) = 0.7824 Ha: p < 0.55 Ha: p != 0.55 Ha: p > 0.55

Ho: p = 0.55 p = proportion(GONEG) z = -0.7805 GONEG .5242291 .0331472 .4592618 .5891964 Variable Mean Std. Err. [95% Conf. Interval] One-sample test of proportion GONEG: Number of obs = 227

. prtest GONEG==.55 if Y2006==1 & SENATE==0

OneSampleSignTest

• Sometimesthemedianisthemostefficientmeasureofcentraltendency.– Perhapsthevariableisordinal.– Ormaybeitisacontinuousvariablebutskewedbecauseofoutliers.

• Inthiscaseourbestunivariate testofstatisticalinferenceistheonesamplesigntest.

OneSampleSignTest

• Theteststatisticforthistestis(1)inthecaseofone-sidedtests,thenumberofpositiveornegativedifferencesbetweentheobservedvaluesandthehypothesizedmedianvalue,or(2)inthecaseoftwo-sidedtests,thesmallerofthesetwodifferences.Ineithercase,thenumberofpositive(P)differencesandnegative(N) differencesiscalculatedwithΣxi– m0.Theyarethensortedbasedonthesignofthedifference.

OneSampleSignTest• Thenullhypothesisforthistestisthatthereisno

statisticallysignificantdifferencebetweenthesamplemedianandthehypothesizedmedian(H0 : m = m0).

• Thetwo-tailedalternativehypothesiswouldbe:

HA : m ≠ m0

• Withone-tailedtests,theHA canbeoneof:

HA : m > m0HA : m < m0

OneSampleSignTest• Wethencalculatetheprobabilityofobservingthedifferences(d) thatwedoundertheassumptionthatthe“real”differenceis0:

– Wheren isthesamplesizeminus“ties”andp istheprobabilityofobservingeitherapositiveornegativedifferencewhenthedifferencebetweenthesamplemedianandhypothesizedmedianisassumedtobe0—thatis,p =.5,sincewearejustaslikelytoobserveeitherkindofdifference.

AnExample:OneSampleSignTest

• Ninestudentsaresurveyedtoassesshowmuchtheyenjoythequalityoftheirschoolfood.TheyaregivenaLikert scalethatrangesbetween1(verybad)to5(verygood).Theobservedscoresare1,3,5,4,2,3,3,4,and5.Previoussurveyssuggestthatthemedianratingforfoodqualityamongstudentsis4(fairlygood).Dowehavesufficientevidencetoinferthatstudentsnowratethefoodqualitymorepoorly?


• First,wefindthenumberofpositiveandnegativedifferences:

Observed m0 Difference1 4 -33 4 -15 4 14 4 02 4 -23 4 -13 4 -14 4 05 4 1

5negatives

2positives

2ties(0s)


• Weareinterestedifstudentsratefoodqualitymorepoorly,sothisisaone-tailedtestandwefocusonthenegativedifferences(d =5):

• Thenumber.164representstheprobabilitythatwewouldobservethismanynegativedifferencesifthemedianswereequal.

AnExample:OneSampleSignTest• Wethenrepeatthisproceduretoobtaintheprobabilitiesforgetting6,7,8,and9negativedifferences.

• Afterdoingthisweadduptheprobabilities:.227.

• Thisismuchlargerthanourcut-offvalueof.05,sowecannotrejectthenullhypothesisthatthereisnostatisticallysignificantdifferencebetweenoursamplemedianandthehypothesizedmedian.Giventhesedata,wecannotsaythatstudentsnowratetheirschoolfoodqualityanylowerthantheyusedto.

YayforStata!

min(1, 2*Binomial(n = 7, x >= 5, p = 0.5)) = 0.4531 Pr(#positive >= 5 or #negative >= 5) = Ha: median of var1 - 4 != 0 Ho: median of var1 - 4 = 0 vs.Two-sided test:

Binomial(n = 7, x >= 5, p = 0.5) = 0.2266 Pr(#negative >= 5) = Ha: median of var1 - 4 < 0 Ho: median of var1 - 4 = 0 vs.

Binomial(n = 7, x >= 2, p = 0.5) = 0.9375 Pr(#positive >= 2) = Ha: median of var1 - 4 > 0 Ho: median of var1 - 4 = 0 vs.One-sided tests:

all 9 9 zero 2 2 negative 5 3.5 positive 2 3.5 sign observed expected

Sign test

. signtest var1=4

Conclusion• Ourknowledgeofsamplingdistributionsiswhatallowsus

togetagrasponthelikelihoodthatoursampleestimatesare“generalizable”tothetargetpopulationorduetosamplingerror(randomchance).

• Wemaketheseinferencesthroughhypothesistesting.

• Weexploredavarietyofunivariate inferentialtests.Theyareunivariate becausewecomparejustonesampleestimatetohypothesizedpopulationparameters.

• Tomorrowwestartlookingatbivariateinferentialtests;i.e.,comparisonsbetweentwosampleestimates.

DatasetsUsed• Druckman,James,MichaelParkin,andMartinKifer.2013.CongressionalCandidateWebsites.ICPSR-34895-v1.AnnArbor,MI:Inter-UniversityConsortiumforPoliticalandSocialResearch.RetrievedFebruary10,2015(http://doi.org/10.3886/ICPSR34895.v1).

• TheStata surveydocumentationdata,nhanes2f,fromtheStata Presswebsite.RetrievedJuly24,2016(http://www.stata-press.com/data/r11/svy.html).

Documents

Day 2, Evening: UnivariateTests of Statistical Inference · Day 2, Evening: UnivariateTests of Statistical Inference Instructor: Marshall A. Taylor 844 Flanner Hall [email protected]