• Home

  • Documents

  • DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a...
64
DRAWING INFERENCES FROM GROUP DATA: Construc)ng 2 nd level models, working with ROIs, and avoiding sta)s)cal circularity

DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

DRAWINGINFERENCESFROMGROUPDATA:Construc)ng2ndlevelmodels,workingwithROIs,

andavoidingsta)s)calcircularity

Page 2: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Soyouhavegroupdata…Whatnow?

¨  Whole-brainvoxelwiseanalysis¤ ConstrucJngappropriate2ndlevelmodels

n Characterizingwithin-subject(repeatedmeasures)effects

n Characterizingbetween-subject(group)effects

n Usingcovariatesatthe2ndlevel

¨  Region-of-interest(ROI)analysis¤ ChoosingyourROIs¤ Avoiding“voodoo”

n a.k.a.inferenJalcircularity;non-independenceerrors

Page 3: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Workingwithrepeatedmeasuresdata

¨  YouhavedatafromagroupofsubjectsandyouwanttolookforacJvitychangesacrosscondiJons

¨  Thesimplestapproachistocreatecontrastimagesforeachsubjectandthenconductaone-samplet-testacrosssubjects

¨  ContrastimagesaresimplyweightedlinearcombinaJonsofcondiJon-specificbetaimages

¤  e.g.,CondA-CondB:[1-1]

¤  ReferredtoasCOPEimagesinFSLand.conimagesinSPM

Page 4: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Workingwithrepeatedmeasuresdata

¨  ContrastswithmorethantwocondiJons:

¤  CondAandCondBvs.CondC:[11-2]

n NOTE:IfCondA&Bhavevastlydifferent#’softrials,itmaybebecertorunanewGLMthatcombinesthesecondiJons,oralternaJvelyyoucouldweighteachcondiJonbythe#oftrials

¤  (CondA–CondB)vs.(CondC–CondD):[1-1-11]

n  TheinteracJonterm!

¨  Nullhypothesis:

¤  acrosssubjectmeanofcontrastvariable=0

Page 5: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Workingwithrepeatedmeasuresdata

¨  Probingforparametriceffectswithlinearcontrastcoefficients

¤  3condiJons:[-101]

¤  4condiJons:[-3-113]

¤  5condiJons:[-2-1012]

¤  6condiJons:[-5-3-1135]

¨  Canalsoprobefor2ndorder(quadraJc)and3rdorder(cubic)effects

¨  Anysetofcontrastcoefficientscanwork,solongastheysumtozero

¤  WriteyourcontrastcoefficientsasposiJvenumbersandthensubtractthemeanfromeachnumber

¤  Butnotethatcustomcontrastsmightnotbeorthogonaltoeachother

Page 6: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Example:MappinglinearandquadraJccontrasts

Page 7: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Example:Mappingcustomcontrasts

Todd&Marois(2004)Nature

Page 8: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

AlternaJveapproach:Separatepairwisetests

Chenetal.(2013)AJNRAmJNeuroradiol

Page 9: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

AlternaJveapproach:Separatepairwisetests

Chenetal.(2013)AJNRAmJNeuroradiol

Page 10: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Workingwithrepeatedmeasuresdata

¨  Probingforparametriceffectswithlinearcontrastcoefficients

¤  3condiJons:[-101]

¤  4condiJons:[-3-113]

¤  5condiJons:[-2-1012]

¤  6condiJons:[-5-3-1135]

¨  Canalsoprobefor2ndorder(quadraJc)and3rdorder(cubic)effects

¨  Anysetofcontrastcoefficientscanwork,solongastheysumtozero

¤  WriteyourcontrastcoefficientsasposiJvenumbersandthensubtractthemeanfromeachnumber

¤  Butnotethatcustomcontrastsmightnotbeorthogonaltoeachother

Page 11: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Theone-samplet-test(inFSL)

TheimagesinpucedintoEV1canbe:Ø  thebetaesJmatesfromasinglecondiJonØ  thedifferencebetweentwocondiJonsØ  anylinearcontrastbetweencondiJons

Take-homepoint:Aone-samplet-testcanaccomplishalot!

Page 12: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

AnalternaJve(slightlymoreinvolved)waytotestforadifferencebetween2condiJons

¨  FSLrepresentaJonof2ndlevelmodelforpairedt-test

8subjects2condiJons(A&B)

Page 13: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Whythetwoapproachesareequivalent

PLoSBiology(2015)

Page 14: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

WhatifonesubjectismissingdatafromagivencondiJon?

¨  RelaJvelycommonsituaJontobefacedwith¤ E.g.,AnalysisofCorrectvs.Incorrecttrials

n  subjectmayhavenoincorrecttrials

¨  PotenJalsoluJons:¤ Analysiscanberunomipngthatsubject’sdata

¤ Or,analysiscanberunasanunpairedt-test(essenJallyconsideringthetwocondiJonsastwoseparategroupsofsubjects)n MoreconservaJve,butmaysomeJmesbeuseful,especiallyifmulJplesubjectsaremissingdata

Page 15: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Between-subjectmodels

¨  Wehavetwogroupsofsubjects(e.g.,9paJentsand7controls)withpotenJallydifferentcross-subjectvariance.¤ Two-SampleT-Test(equalvariancenotassumed)

¨  SpecifytwogroupmembershipssothatFSL’sFEATesJmateseachgroup'scross-subjectvarianceseparately.¤ SPM’sdefaultisalsotoassumeunequalvariance

¨  WewanttotestwhetherornotacJvitybetweenthetwogroupsisequivalentforagivencondiJon

Page 16: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Two-sampleunpairedt-test

Page 17: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Howabouttwogroupswithtwocondi)ons?

¨  E.g.,Comparingthemagnitudeofpre-vs.post-trainingacJvaJonchangesinpaJentsandcontrols¤ Within-subjectfactor:pre-vs.post-training¤  Across-subjectfactor:paJentsvs.controls

¨  Oneeasy-to-implementapproach:¤  FirstcomputeA-Bdifferenceimageforeachsubject¤  Thenperformsimpletwo-sample(unpaired)t-testtofindgroupdifferencesn  i.e.,groupxcondiJoninteracJon

¨  ThisbasicapproachcangeneralizetoanylinearcombinaAonofrepeatedmeasuresfactors

Page 18: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Howabouttwogroupswithtwocondi)ons?

¨  AnalternaJveapproach(slightlymoreinvolved):

Page 19: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

SpecificaJonofmorecomplexgroup-levelmodelscangettrickyinFSL

¨  SeeFSLFEATuserguideforexamplesofmulJ-factorANOVAmodels

¨  hCp://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FEAT/UserGuide

Page 20: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Introducingcovariatesintoyourmodels

¨  Singlegroupwithcovariate¤ Youhaveasinglegroupofsubjectsandyoualsohavemeasuredage.YouwouldliketoseeifthereisanageeffectonbrainacJvity.

n Whatwouldthemodellooklike?

n Whatcontrastswouldyouspecifyfortheageeffect?

Page 21: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Use age Use demeaned age

•  Both models will give exactly the same result for C2, but C1 will be different.

Singlegroupwithcovariate

SlidefromJeaneCeMumford

Page 22: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Simulateddata

Takehome:MeancenteringisonlynecessaryifyouwantyourPEofcolumnof1stobetheoverallmean

SlidefromJeaneCeMumford

Page 23: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

TwogroupswithaconJnuouscovariate

¨  Wehavetwogroupsandaconfoundingcovariate(e.g.,depressionscore).

¨  OurprimaryinterestisinthedifferenceofmeanbrainacJvaJonbetweenthetwogroups.

¨  Wewanttomakesurethisdifferencewasn’tduetobetweengroupdifferencesindepression.

¨  Whatshouldthemodellooklike?

SlidefromJeaneCeMumford

Page 24: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

¨  Okaytodemeantheconfoundingmeasureacrossallsubjects

¤  Butonlynecessaryifyouintendtolookateachgroupvs.baseline(e.g.,[100])

¨  ButdoNOTdemeantheconfoundingmeasurewithingroup

¤  Thiscouldremoveanyconfoundingeffectthemeasuremighthave

TwogroupswithaconJnuouscovariate

SlidefromJeaneCeMumford

Page 25: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Whyyoushouldn’tdemeanwithingroup

¨  Whatifthisiswhatyourdatalooklike?¤ ApparentdifferenceinmeansisclearlyduetorangeofXsampled,notthegroupmembership

SlidefromJeaneCeMumfordFormorediscussionofissuessurroundingmean-centering:hCp://mumford.fmripower.org/mean_centering/

Page 26: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

FormoreinformaJononmean-centering

hCp://mumfordbrainstats.tumblr.com

Page 27: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

TesJngtheinteracJon

¨  WhatifyouwanttotestwhethertherelaJonshipbetweenbrainacJvityanddepressionscorediffersbetweenyourgroups?¤ mean-centeringisnotnecessaryhere

010-1SlidefromJeaneCeMumford

Page 28: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

2ndlevelanalysesinSPM

Page 29: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

2ndlevelanalysesinSPM

Page 30: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

2ndlevelanalysesinSPM

Page 31: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

2ndlevelanalysesinSPM

Page 32: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

2ndlevelanalysesinSPM

Page 33: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

2ndlevelanalysesinSPM

Page 34: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

AlternaJvestovoxelwiseanalysis

¨  ConvenJonalfMRIstaJsJcscomputeonestaJsJcalcomparisonpervoxel.¤  Advantage:candiscovereffectsanywhereinbrain.¤  Disadvantage:lowstaJsJcalpowerduetomulJplecomparisons.

¨  SmallVolumeCorrecAon:OnlyruntestsonasmallproporJonofvoxels(byreducingthesearchspace,allowsformorelenientvoxel-levelstats)

¨  Region-of-interest:PooldataacrossaregionforsinglestaJsJcaltest.

SVC ROI SPM

Example: how many comparisons on this slice?

• Voxelwise: 1600

• SVC: 57

• ROI: 1

SlidefromChrisRorden

Page 35: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

WhyuseROIs?

¨  ConvenientwaytoalleviatethemulJplecomparisonsproblemsthatariseinwhole-brainanalyses¤  ShouldsAlladjuststatsfor#ofROIstested!

¨  Allowsformorehypothesis-drivenanalyses¤  ExploringtheenJredatasetcanbeunwieldyandsomeJmesleadstounfocusedandhighly-speculaJvepapers

¤  ROIresultsareeasiertopresentanddiscuss

¤ HemodynamicJmecourseplotscanbeinformaJve

¤  ROIsdon’trequiresubjectstoacJvatetheexactsamevoxeln  ButanROI-onlyanalysisisvulnerabletoTypeIIerrors!

Page 36: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Region-of-interestanalysis

¨  ChoosingtherightROI(s)¤ Anatomically-defined

n Atlas-based(e.g.,AALatlas)

Page 37: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Region-of-interestanalysis

¨  ChoosingtherightROI(s)¤ Anatomically-defined

n AutomatedsegmentaJon-based(e.g.,Freesurfer)

Page 38: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Region-of-interestanalysis

¨  ChoosingtherightROI(s)¤ Anatomically-defined

n AutomatedsegmentaJon-based(e.g.,Freesurfer)

Page 39: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Region-of-interestanalysis

¨  ChoosingtherightROI(s)¤ Anatomically-defined

n Hand-tracedusinganatomicallandmarks

Page 40: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Region-of-interestanalysis

¨  ChoosingtherightROI(s)¤ Anatomically-defined

n Coordinate-based(e.g.,basedonapreviousstudy)

Page 41: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Region-of-interestanalysis

¨  ChoosingtherightROI(s)¤ Anatomically-defined

n Coordinate-based(e.g.,basedonapreviousstudy)

Page 42: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Region-of-interestanalysis

¨  ChoosingtherightROI(s)¤ Anatomically-defined

n Coordinate-based(e.g.,basedonameta-analysis)

Page 43: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Region-of-interestanalysis

¨  ChoosingtherightROI(s)¤ Anatomically-defined

n Network-based

!

Page 44: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Region-of-interestanalysis

¨  ChoosingtherightROI(s)¤ Anatomically-defined

n Network-based

Poweretal.(2013)CurrOpinNeurobio

Page 45: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Region-of-interestanalysis

¨  ChoosingtherightROI(s)¤ Anatomically-defined

n Network-based

Gordonetal.(2014)CerebralCortex

Page 46: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Region-of-interestanalysis

¨  ChoosingtherightROI(s)¤ Anatomically-defined

n Network-based

Gordonetal.(2014)CerebralCortex

Page 47: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Region-of-interestanalysis

¨  ChoosingtherightROI(s)¤ FuncAonally-defined

n BasedonaparJcularacJvaJoneffectinyourdatan ROI-definingcontrastneedstobeorthogonaltostaJsJcaltestsconductedonextracteddata

n  Independent“localizer”taskscanbeusefulhere

n AVOIDDOUBLE-DIPPINGATALLCOSTS!

Page 48: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Isthisokay?

Ø  IdenAfyaclusterthatshowsabovesignificantacJvityforbothCatsandDogs(viaconjunc)onanalysis)

Ø  TestwhetherthisregionshowssignificantacJvityforDogs>Snakes

Ø  TestwhetherthisregionshowssignificantacJvityforHorses>Snakes

Ø  IdenAfyaclusterthatshowsCats>Dogseffect

Ø  TestwhetherthisregionshowssignificantacJvityforCats>Snakes

Ø  TestwhetherthisregionshowssignificantacJvityforSnakes>Dogs

Ø  TestwhetherthisregionshowssignificantacJvityforDogs>Snakes

Page 49: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Isthisokay?

Ø  IdenAfyaclusterthatshowssignificanteffectofsJmulustypeinone-wayANOVA(F-test)thatincludesalltaskcondiJons(Cats,Dogs,Snakes,Horses)

Ø  TestwhetherthisregionshowssignificantacJvityforCats>Dogs

Ø  TestwhetherthisregionshowssignificantacJvityfor(Cats&Dogs&Horses)>Snakes

Page 50: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Isthisokay?

StrongMemory

ModerateMemory

WeakMemory

***

StrongMemory

ModerateMemory

WeakMemory

*

*

IdenJfyROIsbasedon2contrasts:StrongMemory>WeakMemoryStrongMemory>ModerateMemory

Okaytolookatplotstomakesurethatnothingweirdisgoingon,buteffectsizeswillbeinflatedandsignificancebracketsarehighlymisleading

Page 51: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Isthisokay?

StrongMemory

ModerateMemory

WeakMemory

***

StrongMemory

ModerateMemory

WeakMemory

*

*

IdenJfyROIsbasedonanatomicalboundaries

NotappropriatetodrawanyconclusionsaboutthesetworegionsshowingdissociableprofilesofacJvityunlessyoudirectlytestforaregionxcondiJoninteracJon

Page 52: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

The“implied”interacJon

Nieuwenhuisetal.(2011)NatureNeuroscience

Page 53: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

The“implied”interacJon

Nieuwenhuisetal.(2011)NatureNeuroscience

Page 54: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

The“implied”interacJon

Nieuwenhuisetal.(2011)NatureNeuroscience

¨  “…theerrorofcomparingsignificancelevelsisespeciallycommonintheneuroimagingliterature,inwhichresultsaretypicallypresentedincolor-codedstaJsJcalmapsindicaJngthesignificancelevelofaparJcularcontrastforeach(visible)voxel.

¨  Avisualcomparisonbetweenmapsfortwogroupsmighttempttheresearchertostate,forexample,that“thehippocampuswassignificantlyacJvatedinyoungeradults,butnotinolderadults”.

¨  However,theimpliedclaimisthatthehippocampusisacJvatedmorestronglyinyoungeradultsthaninolderadults,andsuchaclaimrequiresadirectstaJsJcalcomparisonoftheeffects.

¨  Similarly,claimsaboutdifferencesinacJvaJonacrossbrainregionsmustbesupportedbyasignificantinteracJonbetweenbrainregionandthefactorunderlyingthecontrastofinterest.”

Page 55: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

ROIdefiniJonisaffectedbynoise

SlidefromNikoKriegeskorte

true region

overfitted ROI

RO

I-ave

rage

ac

tivat

ion

overestimated effect

independent ROI

Page 56: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Baker, Hutchison, & Kanwisher

(2007)

High selectivity from pure

noise. SlidefromEdVul

ROIdefiniJonisaffectedbynoise

Page 57: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

VoodoocorrelaJons

EdVul

Page 58: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

VoodoocorrelaJons

Page 59: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

VoodooPuzzlinglyhighcorrelaJons

Page 60: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

VoodoocorrelaJons

r = 0.82

Neuralcorrelatesofhumanvirtuejudgment

r = -0.83

Page 61: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

VoodoocorrelaJons

Thisfigureistherealdata

Page 62: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

VoodoocorrelaJons

“Tosumup,then,weareledtoconcludethatadisturbinglylarge,andquite

prominent,segmentoffMRIresearchonemoJon,personality,andsocialcogniJon

isusingseriouslydefecJveresearchmethodsandproducingaprofusionofnumbersthatshouldnotbebelieved.Althoughwehavefocusedhereon

studiesrelaJngtoemoJon,personality,andsocialcogniJon,wesuspectthatthequesJonableanalysismethodsdiscussedherearealsowidespreadinotherfields

thatusefMRItostudyindividualdifferences,suchascogniJve

neuroscience,clinicalneuroscience,andneurogeneJcs.”

Vuletal.(2009)

Page 63: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

ToavoidselecJonbias,wecan...

...performanonselecAveanalysisOR...makesurethatselecJonandresultsstaJsJcsareindependentunderthenullhypothesis,becausetheyareeither:¨  inherentlyindependent¨  orcomputedonindependentdata

¤ e.g.,separatestudy;funcAonallocalizer;cross-validaAon

e.g.independentcontrasts

e.g.whole-brainmapping(noROIanalysis)

SlidefromNikoKriegeskorte

Page 64: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

FormoreinformaJon…

¨  PuzzlinglyHighCorrelaAonsinfMRIStudiesofEmoAon,Personality,andSocialCogniAon.Vul,E.,HarrisC.,Winkielman,P.,&Pashler,H.(2009)PerspecJvesonPsychologicalScience,4,274-290.[FormerlyJtled:VoodooCorrelaJonsinSocialNeuroscience].

¨  Circularanalysisinsystemsneuroscience–thedangersofdoubledipping.KriegeskorteN,SimmonsWK,BellgowanPSF,BakerCI.(2009)NatureNeuroscience12(5):535-40.

¨  Everythingyouneverwantedtoknowaboutcircularanalysis,butwereafraidtoask.KriegeskorteN,LindquistMA,NicholsTE,PoldrackRA,VulE.(2010)JCerebBloodFlowMetab.30(9):1551-7.

¨  Voodooandcircularityerrors.Vul,E.andPashler,H.(2012).NeuroImage,62,945-948.


Reports, Inferences, and Judgments in Deceptive and ... · Keywords: News, Deception, Inferences, Ethics, Content analysis . Reports, Inferences and Judgments 3 Reports, Inferences,

Reports, Inferences, and Judgments in Deceptive and ... · Keywords: News, Deception, Inferences, Ethics, Content analysis . Reports, Inferences and Judgments 3 Reports, Inferences,

Documents
Work in India: Exploring Survey Data to make Inferences ...icrier-jobs.org/wp-content/uploads/2015/12/Jaivir_Singh-jaivir.pdf · Work in India: Exploring Survey Data to make Inferences

Work in India: Exploring Survey Data to make Inferences ...icrier-jobs.org/wp-content/uploads/2015/12/Jaivir_Singh-jaivir.pdf · Work in India: Exploring Survey Data to make Inferences

Documents
M Internal Tide off Oregon: Inferences from Data Assimilationpoa/ · The M2 Internal Tide off Oregon: Inferences from Data Assimilation ALEXANDER L. KURAPOV,GARY D. EGBERT,J.S.A

M Internal Tide off Oregon: Inferences from Data Assimilationpoa/ · The M2 Internal Tide off Oregon: Inferences from Data Assimilation ALEXANDER L. KURAPOV,GARY D. EGBERT,J.S.A

Documents
Inferences for Proportions and Count Data

Inferences for Proportions and Count Data

Documents
1 Single Sample Inferences Greg C Elvers. 2 Single Samples A single sample implies that you have collected data from one group of people or objects You

1 Single Sample Inferences Greg C Elvers. 2 Single Samples A single sample implies that you have collected data from one group of people or objects You

Documents
4th IMPRS Astronomy Summer School Drawing Astrophysical Inferences from Data Sets

4th IMPRS Astronomy Summer School Drawing Astrophysical Inferences from Data Sets

Documents
On Making Valid Inferences by Integrating Data …...On Making Valid Inferences by Integrating Data from Surveys and Other Sources J. N. K. Rao Carleton University, Ottawa, Canada

On Making Valid Inferences by Integrating Data …...On Making Valid Inferences by Integrating Data from Surveys and Other Sources J. N. K. Rao Carleton University, Ottawa, Canada

Documents
DRAFT-Algebra II Unit 4: Inferences and Conclusions from Datamdk12.msde.maryland.gov/share/ccsc/uos/resources/ma…  · Web viewUnit 4. Inferences and Conclusions from Data★

DRAFT-Algebra II Unit 4: Inferences and Conclusions from Datamdk12.msde.maryland.gov/share/ccsc/uos/resources/ma…  · Web viewUnit 4. Inferences and Conclusions from Data★

Documents
Introductory Statistics with R: Simple Inferences for ... · Introductory Statistics with R: Simple Inferences for continuous data Statistical Packages STAT 1301 / 2300, Fall 2014

Introductory Statistics with R: Simple Inferences for ... · Introductory Statistics with R: Simple Inferences for continuous data Statistical Packages STAT 1301 / 2300, Fall 2014

Documents
Observations & Inferences

Observations & Inferences

Documents
STATISTICS FOR MANAGERS LECTURE 3: LOOKING AT DATA AND MAKING INFERENCES

STATISTICS FOR MANAGERS LECTURE 3: LOOKING AT DATA AND MAKING INFERENCES

Documents
Making Inferences. Let’s practice making inferences!

Making Inferences. Let’s practice making inferences!

Documents
Ir inferences

Ir inferences

Documents
OBSERVATIONS vs. INFERENCES OBSERVATIONS vs. INFERENCES

OBSERVATIONS vs. INFERENCES OBSERVATIONS vs. INFERENCES

Documents
Agenda Group Hypotheses Validity of Inferences from Research Inferences and Errors Types of Validity Threats to Validity

Agenda Group Hypotheses Validity of Inferences from Research Inferences and Errors Types of Validity Threats to Validity

Documents
M Internal Tide off Oregon: Inferences from Data Assimilation · The M2 Internal Tide off Oregon: Inferences from Data Assimilation ALEXANDER L. KURAPOV,GARY D. EGBERT,J.S.A ... plication

M Internal Tide off Oregon: Inferences from Data Assimilation · The M2 Internal Tide off Oregon: Inferences from Data Assimilation ALEXANDER L. KURAPOV,GARY D. EGBERT,J.S.A ... plication

Documents
Variance Estimation: Drawing Statistical Inferences from IPUMS-International Census Data

Variance Estimation: Drawing Statistical Inferences from IPUMS-International Census Data

Documents
Drawing Statistical Inferences from Historical Census Data - ipums usa

Drawing Statistical Inferences from Historical Census Data - ipums usa

Documents
Chapter 12 Inferences on Categorical Data and Qualitative Data

Chapter 12 Inferences on Categorical Data and Qualitative Data

Documents
Unit 7: Inferences & Conclusions from Data

Unit 7: Inferences & Conclusions from Data

Documents
Organizing Topic - augusta.k12.va.us€¦  · Web viewCompare maps and make inferences. Draw conclusions and make inferences about geographic data. Identify and interpret regional

Organizing Topic - augusta.k12.va.us€¦  · Web viewCompare maps and make inferences. Draw conclusions and make inferences about geographic data. Identify and interpret regional

Documents
African Lions: Modeling Population Growth Date: · PDF fileWhat inferences can we make about ... • Apply models to real data sets • Make inferences about new ... Lion population

African Lions: Modeling Population Growth Date: · PDF fileWhat inferences can we make about ... • Apply models to real data sets • Make inferences about new ... Lion population

Documents
DRAWING INFERENCES FROM DATA THE CHI SQUARE TEST

DRAWING INFERENCES FROM DATA THE CHI SQUARE TEST

Documents
Who Are The “Expressive Attentives?” Identifying Social ......poll data with respect to this group, whom we label “expressive attentives,” in order to draw inferences about

Who Are The “Expressive Attentives?” Identifying Social ......poll data with respect to this group, whom we label “expressive attentives,” in order to draw inferences about

Documents
CCGPS ADVANCED ALGEBRA UNIT 1- Inferences & Conclusions from Data

CCGPS ADVANCED ALGEBRA UNIT 1- Inferences & Conclusions from Data

Documents
Differential Expression and Network Inferences through ......Differential Expression and Network Inferences through Functional Data Modeling ... ,4 Ruth Etzioni,3 Martin Gleave,4

Differential Expression and Network Inferences through ......Differential Expression and Network Inferences through Functional Data Modeling ... ,4 Ruth Etzioni,3 Martin Gleave,4

Documents
Protecting Sensory Data against Sensitive Inferences · 2018. 6. 21. · Protecting Sensory Data against Sensitive Inferences W-P2DS’18, April 23–26, 2018, Porto, Portugal MobiAct

Protecting Sensory Data against Sensitive Inferences · 2018. 6. 21. · Protecting Sensory Data against Sensitive Inferences W-P2DS’18, April 23–26, 2018, Porto, Portugal MobiAct

Documents
DRAWING INFERENCES FROM DATA

DRAWING INFERENCES FROM DATA

Documents
Unit 9: Inferences for Proportions and Count Dataweb.utk.edu/~leon/stat571/2004SummerPDFs/571Unit9H… ·  · 2010-03-05Unit 9: Inferences for Proportions and Count Data Statistics

Unit 9: Inferences for Proportions and Count Dataweb.utk.edu/~leon/stat571/2004SummerPDFs/571Unit9H… ·  · 2010-03-05Unit 9: Inferences for Proportions and Count Data Statistics

Documents
Quasi-experimental methods enable stronger inferences from ... · Quasi-experimental methods enable stronger inferences from observational data in ecology Van b Butsica,b, David J

Quasi-experimental methods enable stronger inferences from ... · Quasi-experimental methods enable stronger inferences from observational data in ecology Van b Butsica,b, David J

Documents