View
214
Download
0
Tags:
Embed Size (px)
Citation preview
Stephen PizerMedical Image Display & Analysis Group
University of North Carolina, USA
with credit to
T. Fletcher, A. Thall, S. Joshi, P. Yushkevich, G. Gerig
Tutorial:Tutorial: Statistics of Object GeometryStatistics of Object Geometry
10 October 2002
Uses of Statistical Geometric Characterization
Uses of Statistical Geometric Characterization
Medical science: determine geometric ways in which pathological and normal classes differ
Diagnostic: determine if particular patient’s geometry is in pathological or normal class
Educational: communicate anatomic variability in atlases Priors for segmentation Monte Carlo generation of images
Object RepresentationObject RepresentationObjectivesObjectives
Relation to other instances of the shape classRepresenting the real worldDeformation while staying in shape classDiscrimination by shape classLocality
Relation to Euclidean space/projective Euclidean spaceMatching image data
Geometric aspects Geometric aspects Invariants and correspondenceInvariants and correspondence
Desire: An image space geometric representation that is at multiple levels of scale (locality) at one level of scale is based on the object and at lower levels based on object’s figures at each level recognizes invariances associated with
shape provides positional and orientational and metric
correspondence across various instances of the shape class
Object RepresentationsObject Representations
Atlas voxels with a displacement at each voxel : x(x)
Set of distinguished points {xi} with a displacement at each Landmarks Boundary points in a mesh
With normal b = (x,n)
Loci of medial atoms: m = (x,F,r,) or end atom (x,F,r,)
u
v
t
Continuous M-reps: B-splines in Continuous M-reps: B-splines in (x,y,z,r) [Yushkevich](x,y,z,r) [Yushkevich]
Building an Object Representation Building an Object Representation from Atoms from Atoms aa
Sampled aij
can have inter-atom mesh (active surface) Parametrized
a(u,v) e.g., spherical harmonics, where coefficients become
representation e.g., quadric or superquadric surfaces some atom components are derivatives of others
Object representation: Object representation: Parametrized BoundariesParametrized Boundaries
Parametrized boundaries x(u,v)n(u,v) is normalized x/u x/v
Coefficients of decompositionsx(u,v) = i ci f i(u,v)
Spherical harmonics: (u,v) = latitude, longitudeSampled point positions are linear in
coefficients: Ax=c
Object representation: Object representation: Parametrized Medial LociParametrized Medial Loci
Parametrized medial loci m(u,v) = [x,r](u,v)n(u,v) is normalized x/u x/vxr(u,v) = -cos()b
gradient per distance on x(u,v) b
x n
Sampled medial shape representation: Sampled medial shape representation: Discrete M-rep slabs (bars)Discrete M-rep slabs (bars)
Meshes of medial atoms Objects connected as host,
subfigures Multiple such objects,
interrelated
t=+1
p
x
s
br
n
t=-1
t=0
p
x
s
b
n
o
o
o o o
o
o o
o
o o
u
v
t
Interpolating Medial Atoms in a Figure
Interpolate x, r via B-splines [Yushkevich] Trimming curve via r<0 at outside control points
Avoids corner problems of quadmesh Yields continuous boundary
Via modified subdivision surface [Thall] Approximate orthogonality at sail ends Interpolated atoms via boundary and distance
At ends elongation needs also to be interpolated
Need to use synthetic medial geometry [Damon]
Medial sheet
Implied boundary
End Atoms: (x,F,r,)
Medial atom with one more parameter: elongation
Extremely rounded Extremely rounded end atom end atom
in cross-sectionin cross-section
Corner atomCorner atomin cross-sectionin cross-section
=1=1/cos()
Rounded Rounded end atom end atom
in cross-sectionin cross-section
=1.4
Sampled medial shape representation: M-rep tube figures
Same atoms as for slabs r is radius of tube sails are rotated about b Chain rather than mesh
b
x n
x+
rRb,n()bx+rRb,n(-)b
For correspondence: Object-intrinsic coordinatesGeometric coordinates from m-reps
Single figure Medial sheet: (u,v)
[(u) in 2D] t: medial side : signed r-prop’l
dist from implied boundary
3-space: (u,v,t, ) Implied boundary:
(u,v,t)
u
v
t
t=+1
p
x
s
br
n
t=-1
t=0
p
x
s
b
n
Sampled medial shape representation: Linked m-rep slabs
Linked figures Hinge atoms known in
figural coordinates (u,v,t) of parent figure
Other atoms known in the medial coordinates of their neighbors
o
o
o o o
o
o o
o
o o
x+
rRb,n()bb
x nx+rRb,n(-)b
Blend in hinge regions w=(d1/r1 - d2 /r2 )/T Blended d/r when |w| <1 and u-u0 < T Implicit boundary: (u,w, t) Or blend by subdivision surface
Figural Coordinates for Object Made From Multiple Attached Figures
Figural Coordinates for Object Made From Multiple Attached Figures
w
Inside objects or on boundary Per object In neighbor object’s
coordinates Interobject space
In neighbor object’s coordinates
Far outside boundary: (u[,v],t, ) via distance (scale) related figural convexification ??
??
Figural Coordinates for Multiple Objects
Figural Coordinates for Multiple Objects
Heuristic Medial Correspondence
Radius
Original (Spline Parameter) Arclength
0.2 0.4 0.6 0.8 1
0.2
0.4
0.6
0.8
1
Coordinate Mapping
Continuous Analytical Features
Can be sampled arbitrarily.
Allow functional shape analysis
Possible at many scales: medial, bdry, other object
Medial Curvature Boundary texture scale
Feature-Based Correspondence on Medial Locus by Statistical Registration of Features
curvature
dr/ds dr/ds
Also works in 3D
What is Statistical Geometric Characterization
What is Statistical Geometric Characterization
Given a population of instances of an object class e.g., subcortical regions of normal males of age 30
Given a geometric representation z of a given instance e.g., a set of positions on the boundary of the object
and thus the description zi of the ith instance
A statistical characterization of the class is the probability density p(z)
which is estimated from the instances zi
Benefits of Probabilistically Describing Anatomic Region Geometry
Benefits of Probabilistically Describing Anatomic Region Geometry
Discrimination among geometric classes, Ck Compare probabilities p(z | Ck)
Comprehension of asymmetries or distinctions of classes Differences between means Difference between variabilities
Segmentation by deformable models Probability of geometry p(z) provides prior
Provides object-intrinsic coord’s in which multiscale image probabilities p(I|z) can be described
Educational atlas with variability Monte Carlo generation of shapes, of diffeo- morphisms,
to produce pseudo-patient test images
Necessary Analysis Provisions To Achieve Locality & Training Feasibility
Necessary Analysis Provisions To Achieve Locality & Training Feasibility
Multiple scales Allows few random
variables per scale
At each scale, a level of locality (spatial extent) associated with its random variable
Positional correspondence Across instances Between scales
Large scale Smaller scale
Discussion of ScaleDiscussion of Scale
Spatial aspects of a geometric feature Its position Its spatial extent
Region summarized Level of detail captured
Residues from larger scales Distances to neighbors with
which it has a statistical relationship
Markov random field Cf PDM, spherical harmonics,
dense Euclidean positions, landmarks, m-reps
Large scale Smaller scale
Location
Leve
l of
Det
ail
Coarse
Fine
Scale Situations in Various Statistical Geometric Analysis Approaches
Scale Situations in Various Statistical Geometric Analysis Approaches
Location Location
Global coef for Multidetail feature, Detail residues
each level of detail,
E.g., spher. harm. E.g., boundary pt. E.g., object hierarchy
Coordinates at one scale must relate to parent coordinates at next larger scale
Coordinates at one scale must be writable in neighbor’s coordinate system
Statistically stable features at all scales must be relatable at various scale levels
Principles of Object-Intrinsic Coordinates at a Scale LevelPrinciples of Object-Intrinsic Coordinates at a Scale Level
Figurally Relevant Spatial Scale Levels: Primitives and Neighbors
Multi-object complex Individual object
= multiple figures in geom. rel’n to neighbors in relation to complex
Individual figure = mesh of medial atoms subfigs in relation to neighbors in relation to object
Figural section = multiple figural sections
each centered at medial atom medial atoms in relation to neigbhors in relation to figure
Figural section residue, more finely spaced, .. => multiple boundary sections (vertices)
Boundary section vertices in relation to vertex neighbors in relation to figural section
Boundary section more finely spaced, ...
If the total geometric representation z is at all scales or smallest scale, it is not stably trainable with attainable numbers of training cases, so multiscale Let zk be the geometric representation at scale level k Let zk
i be the ith geometric primitive at scale level k
Let N(zki) be the neighbors of zk
i (at level k) Let P(zk
i) be the parent of zki (at level k-1)
Probability via Markov random fields p(zk
i | P(zki), N(zk
i) ) Many trainable probabilities
If p(zki rel. to P(zk
i), zki rel. to N(zk
i) ) Requires parametrized probabilities
Multiscale Probability Leads to Trainable Probabilities
Multiscale Probability Leads to Trainable Probabilities
Multi-Scale-Level Image AnalysisGeometry + Probability
Multiscale critical for effectiveness with efficiency O(number of smallest scale primitives) Markov random field probabilistic basis Vs. methods working at small scale only or at
global scale + small scale only
Multi-Scale-Level Image Analysisvia M-reps
Thesis: multi-scale-level image analysis is particularly well supported by representation built around m-reps Intuitive, medically relevant scale levels Object-based positional and orientational correspondence Geometrically well suited to deformation
Statistics/Probability Aspects : Principal component analysis Any shape, x, can be written as
x = xmean + Pb + r
log p(x) = f(b1, … bt,|r|2)
x1
x2
p1
xi
xmean
b1
Visualizing & Measuring Global Deformation
c = cmean + z11p1 c = cmean + z22p2
Shape MeasurementModes of shape variation across patients Measurement = z amount of each mode
Statistics/Probability Aspects : Markov random fields
Suppose zT= (z1 … zn) p(zi | {zj, ji}) =
p(zi | {zk : k a neighbour of i})(i. e., assume sparse covariance matrix)
Need only evaluate O(n) terms to optimize p(z) or p(z | image)
Can only evaluate p(zi), i.e., locally Interscale; within scale by locality
If z is at all scales or smallest scale, it is not stably trainable, so multiscale Let zk be the geometric rep’n at scale k Let zk
i be the ith geometric primitive at scale k
Let N(zki) be the neighbors of zk
i
Let P(zki) be the parent of zk
i
Let C(zki) be the children of zk
i
Probability via Markov random fields p(zk
i | P(zki), N(zk
i), C(zki) )
Many trainable probabilities Requires parametrized probabilities for training
Multiscale Geometry and ProbabilityMultiscale Geometry and Probability
z1 (necessarily global): similarity transform for body section z2
i: similarity transform for the ith object Neighbors are adjacent (perhaps abutting) objects
z3i: “similarity” transform for the ith figure of its object in its parent’s figural coordinates
Neighbors are adjacent (perhaps abutting) figures z4
i: medial atom transform for the ith medial atom Neighbors are adjacent medial atoms
z5i: medial atom transform for the ith medial atom residue at finer scale (see next slide)
z6i: boundary offset along medially implied normal for the ith boundary vertex
Neighbors are adjacent vertices
Probability via Markov random fields p(zk
i | P(zki), N(zk
i), C(zki) )
Many trainable probabilities Requires parametrized probabilities for training
Examples with m-reps components p(zk
i | P(zki), N(zk
i), C(zki) )
Geometrically smaller scale Interpolate (1st order) finer spacing of
atoms Residual atom change, i.e., local
Probability At any scale, relates figurally
homologous points Markov random field relating medial
atom with its immediate neighbors at that scale its parent atom at the next larger scale and
the corresponding position its children atoms
Multiscale Geometry and Probability for a Figure
Multiscale Geometry and Probability for a Figure
coarse, global
coarse resampled
fine, local
Published Methods of Global Statistical Geometric Characterization in MedicinePublished Methods of Global Statistical Geometric Characterization in Medicine
Global variability via principal component analysis on
features globally, e.g., boundary points or landmarks, or
global features, e.g., spherical harmonic coefficients for boundary
Global difference via linear (or other) discriminant on features
globally or on global features Globally based diagnosis
via linear (or other) discriminant on features globally or on global features
Example authors: [Bookstein][Golland] [Gerig] [Joshi] [Thompson & Toga][Taylor]
Published Methods of Local Statistical Geometric Characterization
Published Methods of Local Statistical Geometric Characterization
Local variability via principal component analysis on features
globally or on global features, plus display of local properties of principal component
Local difference via linear (or other) discriminant on global
geometric primitives, plus display of local properties of discriminant direction
On displacement vectors: signed, unsigned re inside/outside
Example authors: [Gerig] [Golland] [Joshi] [Taylor] [Thompson & Toga]
A
Outward, p < 0.05
Inward, p < 0.05
p > 0.05
R L
Displacement significance: Schizophrenic vs. control hippocampus
Shortcomings of Published Methods of Statistical Geometric CharacterizationShortcomings of Published Methods of Statistical Geometric Characterization
Unintuitive Would like terms like bent, twisted, pimpled, constricted,
elongated, extra figure Frequently nonlocal or local wrt global template
Depends on getting correspondence to template correct Need where the differences are in object coordinates
Which object, which figure, where in figure, where on boundary surface
Requires infeasible number of training cases Due to too many random variables (features)
Overcoming Shortcomings of Methods of Statistical Geometric Characterization
Overcoming Shortcomings of Methods of Statistical Geometric Characterization
Intuitive Figural (medial) representation provides terms like
bent, twisted, pimpled, constricted, elongated, extra figure
Local Hierarchy by scale level provides appropriate level of
locality Object & figure based hierarchy yields intuitive locality
and good positional correspondences Which object, which figure, where in figure, where on
boundary surface Positional correspondences across training cases & scale levels
Trainable by feasible number of cases Few features in residue between scale levels
Relative to description at next larger scale level Relative to neighbors at same scale level
Conclusions re Object Based Image Analysis
Work at multiple levels of scaleWork at multiple levels of scale At each scale use representation appropriate for that scaleAt each scale use representation appropriate for that scale
At intermediate scalesAt intermediate scalesRepresent mediallyRepresent mediallySense at (implied) boundarySense at (implied) boundary
Papers at midag.cs.unc.edu/pubs/papers
Extensions
Variable topologyjump diffusion (local shape)level set?
Active Appearance Modelsshape and intensity‘explaining’ the imageiterative matching algorithm
Recommended Readings
For deformable sampled boundary models: T Cootes, A Hill, CJ Taylor (1994). Use of active shape models for locating structures in medical images. Image & Vision Computing 12: 355-366.
For deformable parametrized boundary models: Kelemen, Gerig, et al
For m-rep based shape: Pizer, Fritsch, et al, IEEE TMI, Oct. 1999
For 3D deformable m-reps: Joshi, Pizer, et al, IPMI 2001 (Springer LNCS 2082); Pizer, Joshi, et al, MICCAI 2001 (Springer LNCS 2208)