Upload
tricia
View
90
Download
0
Embed Size (px)
DESCRIPTION
Structured Face Hallucination. Chih-Yuan Yang Sifei Liu Ming-Hsuan Yang Electrical Engineering and Computer Science. Outline. Motivation Related work Proposed method Experimental results Conclusions. Motivation. Algorithm. Generate high-quality face images. Challenges. - PowerPoint PPT Presentation
Citation preview
Structured Face Hallucination
Chih-Yuan Yang Sifei Liu Ming-Hsuan Yang
Electrical Engineering and Computer Science
1
Outline• Motivation• Related work• Proposed method • Experimental results• Conclusions
2
Motivation• Generate high-quality face images
3
Algorithm
Challenges
4
• How to effectively model a face?– Landmark points
• How to preserve the consistency of details?– Transfer details of a whole component– Maintain consistency of edges in
upsampling– Exploit statistics of edge sharpness
Face Hallucination [Liu07]• PCA on intensities
– Global constraint• MRF on residues
– High-frequency details• Bilateral filtering as post-processing
– Suppress ghost effects
5
Sparse Representation [Yang08]• NMF on intensity
– Global constraint• Patch mapping through a pair of
sparse dictionaries– Restore the high-frequency details
6
Position Patch [Ma10]• No global constraint• Only local constraint by patch
position– Only use exemplar patches at the same
position– Weighted averaging exemplar patches
7
Proposed Approach
• Three classes– facial components
• Transfer the HR details from the whole region of a component– edges
• Preserve edge structures and restore sharpness by statistical prior– smooth regions
• Transfer the HR details from small patches
Aligning Component Exemplars• Exemplar images are labeled • Each component is aligned individually
9
Align high-resolution exemplar images: coordinates of landmark points
Generate low-resolution exemplar images
Search for the most similar exemplar
Insights• Consistency
– Consistent details because the whole component is transferred
– The pair of eyes is considered as one component, as well as the eyebrows
• Effectiveness– Landmark points enable the comparison for
a whole component– Effective for various shapes, sizes, and
positions
10
Preserve Edge Structures• Direction-Preserving Upsampling
11
Directional similarity in LR patches
Bilinear interpolation preserves the directional similarity in HR
Regularize theHR image
Restore Edge Sharpness
12
upsampled edge center mag. of grad. enlarged restored restored
Statistical priors
𝑑=0 𝑑=1 𝑑=√2
Smooth Regions• Approach
– Find the most similar LR patch and transfer the HR gradients
• Advantage– Highly adaptive
• Achieved by– PatchMatch
algorithm– Low computational
load
• Restriction– Consistency
– Accuracy
13
Patch only Component Exemplar
Patch only Edge Model and Priors
Generate Output Images
14
𝑤𝑐 𝑤𝑒 𝑈𝑒 𝑈𝑏𝑈𝑐𝑈
Merge gradient maps
Generate output images
Experimental Results
15
16
17
Conclusions• Structured face hallucination
– Effective whole component exemplars– Preserved edge structures and robust
statistical sharpness priors• Preliminary results
– Effective and consistent high-frequency details
– Robustness
18