NA-MIC National Alliance for Medical Image Computing Non-Parametric Statistical Permutation Tests for Local Shape Analysis Martin Styner, UNC Dimitrios

NA-MICNational Alliance for Medical Image Computing

Non-Parametric Statistical Permutation Tests for Local

Shape Analysis

Martin Styner, UNC Dimitrios Pantazis, Richard Leahy, USC LA

Tom Nichols, University of Michigan Ann Arbor

National Alliance for Medical Image Computing http://na-mic.org

TOC

• Motivation local shape analysis• Local shape difference/distance measures• Statistical significance maps• Problem: Multiple correlated comparisons• 1st approach: It’s a hack!• 2nd approach: Let’s do it right!• Template free - Hotelling T2 measures• Example Results• Conclusions & Outlook


Motivation Shape Analysis

• Anatomical studies of brain structures– Changes between patient and healthy controls– Detection, Enhanced understanding, course of

disease, pathology– Normal neuro-development

• interest in diseases with brain changes – Schizophrenia, autism, fragile-X, Alzheimer's

• Information additional to volume• Both volumetric and shape analysis• Shape analysis: where and how?


Shape Distances

• Shape description: – SPHARM-PDM– M-rep

• Normalization:– Rigid Procrustes, brain

size normalized• Local scalar distance

– Euclidean distance– “Radius” difference– Signed vs absolute


Local Shape Analysis

• Distance to template• Distance between

subject pairs• Sets of distance-maps• Significance map

– Statistical test at each point

– Mean difference test• P-values• Significance threshold


Multiple Comparisons

• Lots of correlated statistical tests → Overly optimistic– M-rep: 2x24 tests, SPHARM: 2252 tests– Same problem with other shape descriptions and other

difference analysis schemes

• Correction needed, overly optimistic– Test locally at given level (e.g. α = 0.05)– Globally incorrect false-positive rate

• Bonferroni correction, worst case, assumption: 0% correlation – Correct False-Positive rate at α/n = 0.05/4000 = 0.0000125– Correct False-Positive rate at 1-(1- α)1/n = 0.0000128


1st Approach: SnPM

• Statistical non-Parametric Maps in SPM (SPIE 2004)

• Decomposition of distance map into separate images for processing in SnPM

• 75% overlap necessary due to distortions

• Each image is tested separately in SnPM

• ONE BIG HACK:– 6 correlated tests– Averaging in overlap


2nd Approach: Permutations

• Non-parametric permutation test using spatially summarized statistics, ISBI 2004

• Correct false positive control (Type II)• Summary:

– Random permutations of the group labels– Metric for difference between populations – Spatial normalization for uniform spatial sensitivity– Summarize statistics across whole shape – Choose threshold in summary statistic


Statistical Problem

• 2 groups: a & b, #member na, nb

• Each member: p-features (e.g. 4000)• Test: Is the mean of each feature in the 2

populations the same? – Null hypothesis: The mean of each feature is the

same– Permutations of group label leave distributions

unchanged under null hypothesis– M permutations

• Specific test– Correct false positive rate


Non-parametric Permutation Tests

• Goal: significance for a vector with 4’000 correlated variables• 50’000 to 100’000 permutations• Extrema statistic: controls false-positive

diff normSummaryStatisticMin/Max

Histogram

diff norm


Single Feature Example

• Feature fA,1-fA,n1 vs fB,1-fB,n1

• Compute difference: T0 =|A- B|

• Permute group label → A’i,B’I → Ti

• Make Histogram of Ti

• Histogram = pdf

• Sum histogram = cdf

• Cdf at 1-α = Threshold

α


Multiple features

• Testing a single feature → no problem• Testing multiple features together as a

whole, NOT individually• Summary is necessary of all features

across the surface• For correct Type II, use an extrema

measurement– Right sided distance metrics → Maxima– Left sided distance metrics → Minima


Spatial Normalization

• Extremal summary is most influenced by regions with higher variance

• Assume 2 regions with same difference, but one has larger variance– Region with larger variance contributes more

to extremal statistics and thus sensitivity in that region is higher

• Normalization of local statistical distributions is necessary for spatially uniform sensitivity


Spatial Normalization

• A) local p-values, non-parametric– Minimum, (1-α) thresh

• B) standard deviation, parametric– Maximum, α thresh

• C) q-th quantile, non-parametric– q = 68% ~ if Gaussian– Maximum, α thresh

• Assumptions: A > C > B

• Uniform sensitivity: A > C ~ B

• Numerical pdf: C > B > A

• Use A– Many permutations– High computation + space costs

Extrema statistics

Shape difference metric

α 1-α

Norm Max-stat

Norm p-valueMin-stat


Raw vs Corrected P-values• Raw significance map:

– 4000 elements, 5% → 200 will be significant at 5% by pure chance, if locations are uncorrelated.

• Corrected significance map – Correct control of false negative– Single location significant → whole

shape significant

• No assumption over local covariance – Overly pessimistic– There is room for improvement!


Raw vs Corrected P-values

• Raw p-values are comparable• But visualization of raw p-value map is misleading

even without statement about significance– Too optimistic, often viewed using linear colormap– P-value correction is non-linear !

Correction factor:F = Raw-P / Corr-P


Metric for Group difference

• Scalar Local difference:– Signed/Unsigned

Euclidean distance– Thickness difference– Pairs, Template

• Difference of mean metric → Statistical feature T = |A- B|

• Needed: Positive scalar + shape difference metric between populations

PDM: Mean difference of Euclidean distance at a selected pointGaussian, passed Lilliefors test 0.01


Template Free Stats

• No need for a scalar value at each location for each subject• Positive scalar difference value between populations• SPHARM-PDM

– So far: Signed/absolute Euclidean distance at each location to template → Scalar field analysis

– New: Difference vectors to template → Vector field analysis– Better: Location vector at each location → Template free analysis

→ Length of difference vector between mean vectors of populations

→ Hotelling T2 distance between populations = Hotelling T2 is mean difference 2 vector weighted with the pooled Covariance matrix

T2 = (μa – μ b) Σa,b (μa –μb)

Σa,b = ( (na - 1) Σa + (nb -1) Σb ) / (na +nb - 2)


Hotelling T2 histogram

Hotelling T2 distance of locations (template free)→ 2


Results

• SnPM hack vs Correct permutation tests• Sample Hippocampus study: Stanley

study, resp/non-resp SZ (56) vs Cnt (26)– Both M-rep & PDM

• Other example tests


SnPM-Hack vs Correct Stat

• SnPM too optimistic – relatively good agreement

L

0.001

0.05

R

SnPM


Hippocampus SZ Study

Left Right


M-rep Shape Analysis

Left Right


Vector Field Analysis

T2 location

T2 templatedifference

Abs templatedistance (scalar)

0.0010.05

Raw Significance Maps Corr Significance Maps


Conclusions of Methods

• Multiple comparison correction scheme for local shape analysis– Non-parametric, Permutation-based– Globally correct for false-positive across whole

object– Applicable to scalar, vectors, any Euclidean

space measures– Black box– Pessimistic estimate


NAMIC kit

• StatNonParamTestPDM– Command line tool, Win/Linux/MacOSX – E.g. StatNonParamTestPDM <listfile> -out <basename> -surfList

-numPerms 50000 -signLevel 0.05 -signSteps 1000

• Output (for meshes)– P-value of global shape difference between the

populations (mean T2 across surface)– Mean difference map (effect size)– Hotelling T2 map using robust T2 formula– Raw significance map– Corrected significance map– Mean surfaces of the 2 groups


StatNonParamTestPDM• Input: File with list of ITK mesh files• Generic features also supported using customizable text-file

input option• Currently in NAMIC-Sandbox (open)• Next: submission to Insight Journal• MeshVisu, combination of Mesh and maps

0.011 0.2324 0.123 …..

Map Txt


That’s it folks…

• Questions


Corrected Analysis – Spatial Normalization• Without normalization → incorrect, unless uniformity is assumed

– High variability → overestimation of significance– Low variability → underestimation of significance

-normalization ~ 68% normalization

No norm max stat

L

R

Documents

NA-MIC National Alliance for Medical Image Computing Non-Parametric Statistical Permutation Tests for Local Shape Analysis Martin Styner, UNC Dimitrios