View
48
Download
1
Category
Tags:
Preview:
DESCRIPTION
Nonparametric Modeling of Textures. Outline Parametric vs. nonparametric Image patches and similarity distance Efros-Leung’s texture synthesis by non-parametric sampling Next week Application into image inpainting Application into image quilting Demos and discussions. - PowerPoint PPT Presentation
Citation preview
Nonparametric Modeling of Textures
Outline Parametric vs. nonparametric Image patches and similarity distance Efros-Leung’s texture synthesis by
non-parametric sampling Next week
Application into image inpainting Application into image quilting Demos and discussions
A Simple Example of Nonparametric Model
Class A: blue square, Class B: red triangle
What if we use parametric models?
N(m1,C1) N(m2,C2)
Why Nonparametric? Nonparametric = “Distribution Free”
E.g., we might assume that X1,X2,…,Xn are independent identically distributed (iid) but we do not know its specific distribution – this is particularly useful for handling data in high-dimensional space
Advantage: the resulting inferential statements are relatively more robust than those from parametric models
Disadvantage: limited application because it is difficult, and often impossible to build into the model more sophisticated structures based on our scientific knowledge (i.e., purely data-driven)
Examples Regression analysis: predict the stock market
value based on the history Parametric regression: use AR model to fit the
observation data Nonparametric regression: use heuristics – e.g., if the
value of stock A increases, then the value of stock B is likely to increase (or decrease)
Texture synthesis: Parametric: two images will look similar if they have
similar first-order/second-order statistics Nonparametric: two images will look similar if they
form similar “clouds” in high-dimensional patch space
Nonparametric Sampling in Natural Language
I took a walk in town one dayAnd met a cat along the way.What do you think that cat did say?Meow, Meow, Meow
I took a walk in town one dayAnd met a pig along the way.What do you think that pig did say?Oink, Oink, Oink
I took a walk in town one dayAnd met a cow along the way.What do you think that cow did say?Moo, Moo, Moo
- cited from “Wee Sing for Baby”
Efros-Leung’ Scheme (1999) Image patches
Look at a group of pixels instead of individual one
Similarity distance Are two patches visually similar?
Scanning order Which pixel to synthesize first?
Nonparametric sampling
Image Patches
For the convenience of implementation, patches areoften taken as square blocks (overlapping is allowed)
Similarity Distance MSE metric
Weighted MSE
N
i
N
j
jiyjixN
YXd1 1
22
)),(),((1
),(
N
i
N
j
jiyjixjiWN
YXd1 1
22
)),(),()(,(1
),(
2D Gaussian kernel
Scanning Order
Colored regions denote where synthesis is needed
Onion-peel scanning
Putting Things Together
? 1. Form an inquiry patch
2. Find best matched patches
3. Obtain the histogram of center pixels in all matched patches
4. The ? intensity value isgiven by sampling the empirical distribution
Pseudo-Code Implementation
http://graphics.cs.cmu.edu/people/efros/research/NPS/alg.html
Image Examples
Image Examples (Con’d)
http://graphics.cs.cmu.edu/people/efros/research/EfrosLeung.html
More examples can be found at
Extensions Similarity metric
Cosine distance = normalized Euclidean distance
A
B
Extensions (Con’t)
AB
Sim(A,B) is large but Sim(A,fliplr(B)) is small
Scientific Puzzle Behind
Photoreceptors
cones
rods
Receptive Fields
Direction Selectivity
Recommended