Upload
cece
View
21
Download
1
Embed Size (px)
DESCRIPTION
Template Learning from Atomic Representations:. A Wavelet-based Approach to Pattern Analysis. Clay Scott and Rob Nowak. Electrical and Computer Engineering Rice University www.dsp.rice.edu. Supported by ARO, DARPA, NSF, and ONR. The Discrete Wavelet Transform. - PowerPoint PPT Presentation
Citation preview
Template Learning from Atomic Representations:
Supported by ARO, DARPA, NSF, and ONR
Clay Scott and Rob Nowak
A Wavelet-based Approach to Pattern Analysis
Electrical and Computer EngineeringRice University
www.dsp.rice.edu
• prediction errors wavelet coefficients
• most wavelet coefficients are zero sparse representation
The Discrete Wavelet Transform
Wavelets as Atomic Representations
• Atomic representations: attempt to decompose images
into fundamental units or “atoms”
Examples: wavelets, curvelets, wedgelets, DCT
• Successes: denoising and compression
• Drawback: not transformation invariant
poor features for pattern recognition
Pattern Recognition
Class 1
Class 2
Class 3
Hierarchical Framework
Realization from wavelet-domain statistical model
Pattern template in spatial domain
Random transformation of pattern
Noisy observation of transformed pattern
Realization from wavelet-domain statistical model
Pattern template in spatial domain
Random transformation of pattern
Noisy observation of transformed pattern
Wavelet-domain statistical model
• Model wavelet coefficients as independent Gaussian mixtures
where
)()()( 200
211 1 i,i,ii,i,ii ,σμNs,σμNs~w
20
200 0 σσ,μ i,i,
• Constraints:
• Sparsity can divide wavelet coefficients into significant and insignificant coefficients
ii ws 1 is significant
•Template parameters:
TNss ),...,(
},,{
1
2011
s
Σμθ
where
),...,(diag
),...,(2
1,21,11
1,1,11
N
TN
Σ
μ
Model Parameters
• Finite set of pre-selected transformations
model variability in location and orientationL ,...,1
Pattern Synthesis
1. Generate a random template
2. Transform to spatial domain
3. Apply random transformation
4. Add observation noise
),(~ 2obsIyx N
}{DWT 1 wz
),|(~ sθww p
zy
Template Learning
Given: Independent observations of the same pattern
arising from the (unknown) transformations
Goal: Find , s, that “best describe” the observations
Approach: Penalized maximum likelihood estimation (PMLE)
),...,( 1 TxxΧ
),...,( 1 T
PMLE of , s, and
• Complexity penalty function
where is the number of significant
coefficients
Nkc log2)( s
isk
• PMLE maximize
)(),,|(log),,( ssθXsθ cpF
• Complexity regularization Find low-dimensional template that captures essential structure of pattern
Minimum description length (MDL) criterion
TEMPLAR: Template Learning from Atomic Representations
),,(maxarg
),,(maxarg
),,(maxarg
1
11
jjj
jjj
jjj
F
F
F
sθ
sθs
sθθ
s
θ
• Simultaneously maximizing F over , s, is intractable
• Maximize F with alternating-maximization algorithm
Non-decreasing sequence of penalized likelihood values
Each step is simple, with O(NLT) complexity
Converges to a fixed point (no cycling)
Airplane Experiment
Picture of me gathering data
Airplane Experiment
• 853 significant coefficients out of 16,384• 7 iterations
Face Experiment
Training data for one subject, plus sequence of template convergence
Why Does TEMPLAR Work?
• Wavelet-domain model for template is low-
dimensional (from MDL penalty and inherent
sparseness of wavelets)
• Low-dimensional template allows for improved
pattern matching by giving more weight to
distinguishing features
Classification
Given:
Templates for several patterns and an unlabeled observation x
Cccc 1},{ sθ
Classify:
)],,|(max[ maxarg* cc
cpc sθx
• Invariant to unknown transformations
• O(NT) complexity
• sparsity low-dimensional subspace classifier
robust to background clutter
Face Recognition
Results of Yale face test
Image Registration
If I get results
Conclusion
• Wavelet-based framework for representing pattern observations with unknown rotation and translation
• TEMPLAR: Linear-time algorithm for automatically learning low-dimensional templates based using MDL
• Low-dimensional subspace classifiers that are invariant to spatial transformations and background clutter