Texture Synthesis on Surfaces

Paper by Greg TurkPaper by Greg Turk

Presentation by Jon SuperPresentation by Jon Super

Texture Synthesis on Surfaces

+

Goals of texture placement

Take a 2D texture and wrap it onto a surface

Avoid noticeable seams between patchesAvoid noticeable seams between patches Minimize stretching and distortionMinimize stretching and distortion Produce a natural-looking resultProduce a natural-looking result

The next best thing

Lapped Textures – Praun Lapped Textures – Praun et al. – SIGGRAPH00et al. – SIGGRAPH00

Uses many overlapped Uses many overlapped texture patchestexture patches

Aligns patches with Aligns patches with underlying vector fieldunderlying vector field

Generally minimizes Generally minimizes distortiondistortion

The next best thing

Lapped texture’s Lapped texture’s shortcomingsshortcomings

Low-frequency Low-frequency componentscomponents

Boundary mismatchBoundary mismatch Direction field Direction field

singularitiessingularities

Texture synthesis to the rescue!

Process Create a mesh hierarchy over the modelCreate a mesh hierarchy over the model Set up a vector field over the meshSet up a vector field over the mesh Calculate sweep distance for each vertexCalculate sweep distance for each vertex Assign colors based on neighborsAssign colors based on neighbors

Process Create a mesh hierarchy over the modelCreate a mesh hierarchy over the model Set up a vector field over the meshSet up a vector field over the mesh Calculate sweep distance for each vertexCalculate sweep distance for each vertex Assign colors based on neighborsAssign colors based on neighbors

Mesh Hierarchy

Equivalent of a Gaussian pyramid to a Equivalent of a Gaussian pyramid to a surfacesurface

Multiple layers of different Multiple layers of different resolutionsresolutions

Each layer has 4 times the number of Each layer has 4 times the number of vertices of the layer above itvertices of the layer above it

Each layer contains every vertex of Each layer contains every vertex of the layer above itthe layer above it

Point placement

n points randomly placed on the surfacen points randomly placed on the surface Repulsion between the points spaces them Repulsion between the points spaces them

evenlyevenly These points are now fixed in place, These points are now fixed in place,

representing level m of an m-level meshrepresenting level m of an m-level mesh Repeated for each layer, adding 3 times Repeated for each layer, adding 3 times

more points each time (3n, 12n, etc)more points each time (3n, 12n, etc)

Connectivity

Connectivity between vertices of each layerConnectivity between vertices of each layer Delaunay triangulation appliedDelaunay triangulation applied Mesh is connected in a web of trianglesMesh is connected in a web of triangles

Mesh hierarchy operations

InterpolationInterpolation

Low-pass filteringLow-pass filtering

DownsamplingDownsampling UpsamplingUpsampling

Process Create a mesh hierarchy over the modelCreate a mesh hierarchy over the model Set up a vector field over the meshSet up a vector field over the mesh Calculate sweep distance for each vertexCalculate sweep distance for each vertex Assign colors based on neighborsAssign colors based on neighbors

Vector field

Provides order to the placement of the Provides order to the placement of the texture on the modeltexture on the model

Each mesh vertex is given a vectorEach mesh vertex is given a vector User initiates the process by defining a few User initiates the process by defining a few

key vectors (~12)key vectors (~12)

Vector field creation All vertices not user-defined set to 0All vertices not user-defined set to 0 User-defined vectors are downsampled to User-defined vectors are downsampled to

the coarsest meshthe coarsest mesh Low-pass filtering to fill out top meshLow-pass filtering to fill out top mesh Upsample for each layer to fill to bottomUpsample for each layer to fill to bottom

Process Create a mesh hierarchy over the modelCreate a mesh hierarchy over the model Set up a vector field over the meshSet up a vector field over the mesh Calculate sweep distance for each vertexCalculate sweep distance for each vertex Assign colors based on neighborsAssign colors based on neighbors

Surface sweeping

Uses the directional flow of the vector field Uses the directional flow of the vector field to assign ordering to the verticesto assign ordering to the vertices

Ordering will allow a sort of raster scan Ordering will allow a sort of raster scan over the 3D modelover the 3D model

The color in the imageThe color in the image

cycles as the sweepcycles as the sweep

distance increasesdistance increases

Sweep distance calculation

Select an anchor point to base the distanceSelect an anchor point to base the distance Each vertex takes the value that is the weighted sum Each vertex takes the value that is the weighted sum

of what its neighbors thinkof what its neighbors think Distance along the flow direction added to the Distance along the flow direction added to the

neighbor’s sweep distanceneighbor’s sweep distance

Sweep distances will propagate out from the anchor Sweep distances will propagate out from the anchor until every vertex has oneuntil every vertex has one

Process Create a mesh hierarchy over the modelCreate a mesh hierarchy over the model Set up a vector field over the meshSet up a vector field over the mesh Calculate sweep distance for each vertexCalculate sweep distance for each vertex Assign colors based on neighborsAssign colors based on neighbors

Pixel Neighborhoods

Similar technique to 2D texture synthesisSimilar technique to 2D texture synthesis Examine nearby colors and look for a Examine nearby colors and look for a

similar pattern in the texture imagesimilar pattern in the texture image Different kinds of pixel neighborhoodsDifferent kinds of pixel neighborhoods

Mesh coordinate space

Pixel neighborhoods don’t translate directly Pixel neighborhoods don’t translate directly to the unorganized vertices in a meshto the unorganized vertices in a mesh

Vertex vector is downVertex vector is down Rotate 90 degrees CCW for rightRotate 90 degrees CCW for right

Multi-level synthesis

Start at top, work downStart at top, work down Initialize the vertices of mesh layer k with Initialize the vertices of mesh layer k with

color values from Gaussian pyramid layer kcolor values from Gaussian pyramid layer k Make a pass with the half squareMake a pass with the half square Top layer should have a crude representation Top layer should have a crude representation

of the textureof the texture

Multi-level synthesis

Extrapolate and refine each lower levelExtrapolate and refine each lower level Extrapolate initializes the colors using Extrapolate initializes the colors using

a large full square neighborhood of the a large full square neighborhood of the level abovelevel above

Refine improves on that by combining Refine improves on that by combining a large full square on the current level a large full square on the current level with a small full square on the level with a small full square on the level aboveabove

Algorithm

Results

Results

Results

Results

Results

Results