Overview of Meta-Analysis Approaches

OverviewofMeta-AnalysisApproaches

ThomasE.NicholsUniversityofOxford

NeuroimagingMeta-AnalysisOHBMEducationalCourse

25June,2017slides&posters@http://warwick.ac.uk/tenichols/ohbm

Overview

• Non-imagingmeta-analysis• Menuofmeta-analysismethods– ROI’s,IBMA,CBMA

• CBMAdetails– Kernel-basedmethods– What’sincommon– m/ALE,M/KDA– What’sdifferent

• Limitations&Thoughts

Stagesof(non-imaging) Meta-Analysis

1. Definereview'sspecificobjectives.2. Specifyeligibilitycriteria.3. Identifyalleligiblestudies.4. Collectandvalidatedatarigorously.5. Displayeffectsforeachstudy,withmeasuresof

precision.6. Computeaverageeffect,randomeffectsstd err7. Checkforpublicationbias,conductsensitivity

analyses.

Jones,D.R.(1995).Meta-analysis:weighingtheevidence.StatisticsinMedicine,14(2),137–49.

Methodsfor(non-imaging) Meta-Analysis(1)• P-value(orZ-value)combining– Fishers(≈average–logP)– Stouffers(≈averageZ)– Usedonlyasmethodoflastresort

• Basedonsignificance,noteffectsinrealunits• Differingn willinduceheterogeneity (Cummings,2004)

• Fixedeffectsmodel– Requireseffectestimatesandstandarderrors

• E.g.Meansurvival(days),andstandarderrorofmean– Givesweightedaverageofeffects

• Weightsbasedonper-studystandarderrors– Neglectsinter-studyvariation

Cummings(2004).Meta-analysisbasedonstandardizedeffectsisunreliable.ArchivesofPediatrics&AdolescentMedicine,158(6),595–7.

Methodsfor(non-imaging) Meta-Analysis(2)• Randomeffectsmodel– Requireseffectestimatesandstandarderrors– Givesweightedaverageofeffect• Weightsbasedonper-studystandarderrorsandinter-studyvariation

– Accountsforinter-studyvariation• Metaregression– Accountforstudy-levelregressors– Fixedorrandomeffects

NeuroimagingMeta-AnalysisApproaches(1)

• RegionofInterest– TraditionalMeta-Analysis,onmean%BOLD&stderr– Almostimpossibletodo• ROI-basedresultsrare(exception:PET)• DifferentROIsusedbydifferentauthors• Peak%BOLDuseless,duetovoodoobias

– Peakisoverly-optimisticestimateof%BOLDinROI

MNIx-axis

True%BOLD

Estimated%BOLD

NeuroimagingMeta-AnalysisApproaches(2)

• Intensity-BasedMeta-Analysis(IBMA)–WithP/T/ZImagesonly• OnlyallowsFishers/Stouffers

–WithCOPE’sonly• Onlyallowsrandom-effectsmodelwithoutweights

– Can’tweightbysamplesize!

–WithCOPE’s&VARCOPES• FSL’sFEAT/FLAME is therandomeffectmetamodel!

– 2nd-levelFLAME:Combiningsubjects– 3rd-levelFLAME:Combiningstudies

• Allowsmeta-regression– Butimagedatararelyshared

BestpracticeJ

NotbestpracticeL

NotbestpracticeL

BadpracticeL

NeuroimagingMeta-AnalysisApproaches(3)

• Coordinate-BasedMeta-Analysis(CBMA)– x,y,z locationsonly• ActivationLikelihoodEstimation(ALE)

• MultilevelKernelDensityAnalysis(MKDA)

– x,y,z andZ-value• SignedDifferenceMapping(SDM)

Turkeltaub etal.(2002).Meta-analysisofthefunctionalneuroanatomyofsingle-wordreading:methodandvalidation.NeuroImage,16(3),765–780.Eickhoff etal.(2009). Coordinate-basedactivationlikelihoodestimationmeta-analysisofneuroimagingdata:arandom-effectsapproachbasedonempiricalestimatesofspatialuncertainty.HumanBrainMapping,30(9),2907-26.Eickhoff etal.(2012). Activationlikelihoodestimationmeta-analysisrevisited.NeuroImage,59(3),2349–61

Wageretal.(2004). Neuroimagingstudiesofshiftingattention:ameta-analysis.NeuroImage 22(4),1679–1693.Kober etal.(2008). Functionalgroupingandcortical-subcorticalinteractionsinemotion:ameta-analysisofneuroimagingstudies.NeuroImage,42(2),998–1031.

Radua &Mataix-Cols(2009). Voxel-wisemeta-analysisofgreymatterchangesinobsessive-compulsivedisorder.BritishJournalofPsychiatry,195:391-400.Costafreda etal.(2009). Aparametricapproachtovoxel- basedmeta-analysis.NeuroImage,46(1):115-122.

CMBAKernelMethods• Createstudymaps– Eachfocusisreplacedwithkernel

• Importantdetailsonkerneloverlap

• Createmetamaps– Studymapscombined

• Inference– Traditionalvoxel-wiseorcluster-wise

• Voxel-wise– FDRorFWE• Cluster-wise– FWE

– MonteCarlotest• H0:noconsistencyoverstudies• Randomlyplaceeachstudy’sfoci,recreatemetamaps• Notactuallyapermutationtest(seeBesag&Diggle (1977))

Besag &Diggle (1977).SimpleMonteCarlotestsforspatialpattern.JRSSC(AppliedStatistics),26(3),327–333.

Wager etal.(2007).SCAN,2(2),150–8.

Study 1Study 1Study 2Study 3

KernelMethodsHistory– m/ALE

Study 2Study 3

ALE– ActivationLikelihoodEstimation(Turkeltaub etal.,2002)

ALEper-studymap

ALEmapkernelFHWMf

ALEinterpretationforsinglefocus()Probabilityofobservingafocusatthatlocation()

ALEcombiningProbabilityofunionofevents…ALE(p1,p2)=p1+p2−p1×p2ALE(p1,p2,p3)=p1+p2+p3−p1×p2−p1×p3−p2×p3+ p1×p2×p3

ALEinterpretation:Probabilityofobservingoneormorefociatagivenlocationbased onamodelofGaussianspreadwithFWHMf

Study 1Study 1Study 2Study 3

KernelMethodsHistory– m/ALE

Study 2Study 3

ALE– ActivationLikelihoodEstimation(Turkeltaub etal.,2002)

ALEper-studymap

ALEmapkernelFHWMf

ProblemwithfirstALESinglestudycoulddominate,iflotsonehaslotsofpoints

ModifiedALE(Eickhoff etal.,2009;Eickhoff etal.,2012)RevisedMonteCarlotestaccountsforstudies

Fixfoci,randomlysampleeachmapAdaptkernelsizef tostudysamplesizeVoxel-wisetest– noMonteCarlo!Cluster-wisetest– stillrequiresMonteCarlo

Study 1Study 2Study 3

Study 1

MKDAmap – weightedaverageofstudymaps

Study 1Study 2Study 3

KernelMethodsHistory– M/KDA

SameproblemwithindividualprofligatestudiesMKDA(Kober etal.,2008)

TruncatedkernelMonteCarlotest

Movesclusters,notindividualfoci

MKDA(unweighted) interpretation:Proportionofstudieshavingoneormorefociwithindistancer

Study 2Study 3

KDA– KernelDensityAnalysis(Wageretal.,2004)

KDAper-studymap

KDAmap– averageofstudymaps

MKDA

MKDA– MultilevelKernelDensityAnalysisper-studymap

kernelradiusr

CBMALimitations• Effectsize– Non-imagingMAisallabouteffectsize,CI’s–Whatistheeffectsize?• MKDA– Proportionofstudyresultinneighborhood• ALE– Probabilityatindividualvoxeloneorfoci

– Standarderrors?CI’s?– Power/sensitivity• 5/10studies– Great!• 5/100studies– Notgreat?Orsubtleevidence?

• Fixedvs.RandomEffects?

• Aneffectthatgeneralizestothepopulationstudied

• Significancerelativetobetween-studyvariation

Study 1

Study 2

Study 3

Study 4

Study 5

Study 6

0

Distribution of each study’s estimated effect

Distribution of population effect

s2FFX

s2RFX

IBMARandomEffects?

%BOLD

MNIx-axis

• CBMA– Aneffectthatgeneralizestothepopulationstudied?• 5/10signif.:OK?• 5/100signif.:OK!?

– Significancerelativetobetween-studyvariation?• Significancebasedonnullofrandomdistribution

Study 1

Study 2

Study 3

Study 4

Study 5

Study 6

Location of each study’s foci

Intensity Functione.g. ALE

WhatisaRandomEffect?

… under Ho

MNIx-axis

• BayesianHierarchicalMarkedSpatialindependentClusterProcess– Explicitlyparameterizesintra- andinter-studyvariation

Study 1

Study 2

Study 3

Study 4

Study 5

Study 6

Intensity Function

s2Study

s2Population

WhatisaRandomEffect?

Location of each study’s foci

Kang,Johnson,Nichols,&Wage(2011).MetaAnalysisofFunctionalNeuroimagingDataviaBayesianSpatialPointProcesses.JournaloftheAmericanStatisticalAssociation,106(493),124–134.

CBMASensitivityanalyses

Wageretal.(2009).Evaluatingtheconsistencyandspecificityofneuroimagingdatausingmeta-analysis.NeuroImage,45(1S1),210–221.

• Z-scoresshouldfalltozerowithsamplesize

• MetaDiagnostics– Variousplotsassesswhetherexpectedbehavioroccurs

CBMAFileDrawerBias?• Whatabout“P<0.001uncorrected”bias?

• Forrestplot–MKDAvaluesforrightamygdala

– Canexploredifferentexplanationsfortheeffect

0 20 40 60 80Percent of studies reporting a foci

within 10mm of right amygdala

Chance: whole−brain FWE threshold

Chance: small−volume FWE threshold

Chance: half of all studiesusing P<0.001 uncorrected

Chance: all studiesusing P<0.001 uncorr.

Emotion Meta Analysis from 154 studiesRight Amygdala activation

Anger (26 studies)

Disgust (28 studies)

Fear (43 studies)

Happy (24 studies)

Sad (33 studies)

All (154 studies)

T.Nichols

Foci per contrast

Den

sity

0 5 10 15 20 25

0.00

0.05

0.10

0.15

●

●●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●●

● ● ● ● ● ●

EstimatingSizeoftheFileDrawer• Estimationof“FileDrawer”prevalence

• Usefocicountstoinfernumberofmissing(0count)studies

• About1studymissingper10published– 9.02per10095%CI(7.32,10.72)

– Variesbysubarea

CountsPerContrastEmpirical&FittedDistribution

2,562StudiesfromBrainMapOnecontrastperstudyrandomlyselected

Pantelis Samartsidis,etal.OHBM2015Poster4038-W“Estimatingtheprevalenceof‘filedrawer’studies”

Conclusions

• IBMA–Wouldbegreat,richtoolsavailable

• CBMA– 2+toolsavailable– Stilllotsofworktodeliverbest(statistical)practicetoinferences