View
65
Download
5
Category
Tags:
Preview:
DESCRIPTION
2D Texture Synthesis. Instructor: Yizhou Yu. Texture synthesis. Goal: increase texture resolution yet keep local texture variation. Synthesis by global statistics. Idea: Obtain statistics of input image, match with output image Histograms: normalized graph of intensity frequencies. - PowerPoint PPT Presentation
Citation preview
2D Texture Synthesis
Instructor: Yizhou Yu
Texture synthesis
Goal: increase texture resolution yet keep local texture variation
Synthesis by global statistics Idea: Obtain statistics of input image,
match with output image
Histograms: normalized graph of intensity frequencies
Synthesis by global statistics
Consider extreme case:
Histogram matching can generate two images:
Using higher-order global stats Histogram matching does not take into
account spatial information. Need additional info:
For every pixel, examine local neighborhood Gives local but not global features (e.g. – veins)
Use higher order stats: Correlation
average of the product of the intensities =
tells how interdependent pixel values are
1[ ( ) ( )] ( ) ( )
x
E I x I x I x I xn
General procedure 1.) Define error metric:
What do you want to match?
2.) Match statistics Histogram matching:
Get the density value of a pixel in the output image. Map this density to a pixel intensity in the input image Overwrite the pixel in the output image with the mapped pixel intensity in the input image
Pyramid-based Texture Analysis/Synthesis
Paper by David J. Heeger and James R. Bergen from SIGGRAPH 1995
http://www.cns.nyu.edu/~david/ftp/reprints/heeger-siggraph95.pdf
Pyramid-based Synthesis Downsample image several times and
keep track of only differences in a feature image
To reconstruct, upsample small images and add differences
Pixels in feature images are close to zero Provides multiple scales with each feature
image representing different feature sizes
Laplacian Pyramid
1-D filter: (1/16) (1 4 6 4 1) To generalize to 2D, use tensor
product
Normalize by dividing by 256
Upsampling To upsample, copy pixels to every
other pixel in larger image
Fill the rest with zeroes, run low pass filter to fill in values
Oriented Filters
Gaussian filters are symmetric, thus edges and contours are not detected
Use multiple oriented filters to catch non-symmetric features
Left top series: Oriented filters Right image: Texture
Left bottom series: Filtered textures
Pyramids with Oriented Filters
Each oriented filter creates its own feature image
Thus, for each downsampled image, keep one image for each oriented filter
Each oriented filter captures one orientation of lines
Matching Image Features
Input parameters: noise: initial noisy texture texture: texture to be matched
Output texture is stored in noise
Matching Image Features Helper functions:
MatchHistogram(noise, texture) Matches histogram, using method described last
time MakePyramid(texture)
Create pyramid images (base and feature images) CollapsePyramid(pyramid)
Constructs high resolution image from base and feature images
Matching Image FeaturesMatchTexture(noise, texture){
MatchHistogram(noise, texture) // first-order matchinganalysis_pyr = MakePyramid(texture) // create pyramid from input texturefor several iterations
synthesis_pyr = MakePyramid(noise) // create pyramid from noise
for each feature image, fi of analysis_pyrfor each feature image, fj of synthesis_pyr of same
orientationMatchHistogram(fi, fj)
end forend fornoise = CollapsePyramid(synthesis_pyr)MatchHistogram(noise, texture)
end for}
Matching Image Features MatchTexture matches histograms of
feature images, not pixel values, thus providing a much better matching
Feature images already consider local neighborhoods, resulting in better approximations
Good for randomized textures Textures with large scale features or
thin and long features not matched well e.g. stripes in wood, lines in coral
Texture Synthesis Using Local Neighborhoods Main goal is to keep local spatial
coherence, but not global stats. Randomized method
Pick pixel and copy it along with its neighborhood to random parts in synthesized image
If two neighborhoods overlap, just blend This can result in features getting cut
off if larger than local neighborhood
Neighborhood-Based Texture Synthesis
Patch-Based Patch-based sampling achieves real-time speed.
[Liang et. al. 2001] Image quilting: high-quality results and simple
implementation [Efros & Freeman 2001] Graph cut provides a powerful and refinable
scheme. [Kwantra et. al. 2003] Pixel-Based
[Efros & Leung 99], [Wei & Levoy 2000], [Ashikhmin 2001], [Hertzmann et. al. 2001], [Zhang et. al. 2003]
Pixel-wise Synthesis Grow pixel by pixel Start from an existing patch of the
texture (as opposed to noise texture like Pyramid-based Synthesis)
Look for regions in input texture most similar to current region in new texture
Copy pixels next to best-match region to expand new texture
Following Raster Order Blue region: already set by algorithm Green region: compare this region to input image Yellow region: closest match Red region: replace these pixels with magenta
region to maintain local integrity
Hierarchical Synthesis
Build a multi-resolution pyramid for the example texture
Generate a synthesized pyramid for the output texture At each level, follow the raster order
Randomizing Synthesis Instead of picking pixels from closest
matching region, use threshold to pick a few candidate matches
Randomly pick one of the candidates Refer to “Texture Synthesis by Non-
parametric Sampling” http://www.cs.berkeley.edu/~efros/
research/synthesis.html
Results
Patch-Based Synthesis
Search in a sample texture for neighborhoods most similar to a context region
Merge a patch with the partially synthesized output texture
The Seam Problem
Feature discontinuities may appear in patch-based synthesis.
The Second Reason Didn’t find the smoothest transition
between the incoming patch and context region.
Solutions: use dynamic programming [Efros and Freeman 2001] or graph cut [Kwatra et al. 2003] to find an optimal cut.
Original Warped
The First Reason Rigid template
matching (SSD) often employed in the first step does NOT consider geometric similarity
Original Warped
Feature Map Guided Texture Synthesis
Basic Steps [Wu and Yu 2004]
Maintain an input and output feature map Match and align curvilinear features Integrate feature maps into template-
based texture synthesis
Comparisons
Sample Feature Map Graphcut Quilting Texton Mask
Acceleration Schemes Fourier Transform Acceleration techniques for nearest-neighbor
search Tree-structured vector quantization Kd-trees
Minimizing the candidate set Coherent synthesis, [Ashikhmin 2001]
Precomputing candidate set K-coherent search, [Tong et al. 2002] Jump Map, [Zelinka and Garland 2002]
GPU with parallel synthesis [Lefebvre and Hoppe 2005]
Original Warped
Recommended