Non-Rigid Registration of 3D Surface by Deformable 2D Triangular Meshes Speaker:James Chang

Non-Rigid Registration of 3D Surface by Deformable 2D Triangular Meshes

Speaker:James Chang

Introduction

• Non-rigid surface registration, particularly registration of human face, finds a wide variety of application in computer vision and graphics.

• This study present a new automatic surface registration method utilized both attraction forces originating from geometrical and textural similarities, and stresses due to non-linear elasticity of the surface.

Project flow chart

3D Reference

Surface

3D Target Surface

2D Target Plane

2D Reference

Plane Registration via Triangular

Mesh Deformation

Image Matching with Triangular Meshes

Nonlinear Elastic Deformation

Multiresolution Registration Algorithm

Deformed Surface

Mapping Surfaces onto Image Planes

result



• Blanz and Vetter: A morphable model for the synthesis of 3-D faces. Proc. SIGGRAPH’99, Los Angeles, USA, 187–194, 1999.

- Cylindrical projection: Project their vertices onto planes.

- Large distortions in the regions where the angle between the surface normal and the projection direction is large.

- Worse still, a bijective mapping is not guaranteed.

Bijection


• Briefly the algorithms are categorized according to mapping distortion criterion that they minimize.

• Length-preserving, angle-preserving, are-preserving parameterization approaches.

• Least squares conformal mapping technique.(LSCM),angle-preserving.

B. Levy, S. Petitjean, N. Ray and J. Maillot. Least squares conformal maps for automatic texture atlas generation. ACM Transactions on Graphics, 21(3):362–371, 2002. Proceedings of SIGGRAPH 2002.


Registration via Triangular Mesh Deformation in 2D Domain


• Lets denote the reference surface as A which is to be mapped to a target surface B.

• Their planar parameterizations yield 2D domains

• If we can find a mapping from to ,this leads to the mapping between the surfaces in 3D space.


• Image Matching with Triangular Meshes.• Nonlinear Elastic Deformation.• Multiresolution Registration Algorithm.


A mapping from to can be expressed via a vector field.

Where denotes the 2D image coordinates and is a constant displacement

AB


• For each triangle of the reference mesh, we have a mapping function that maps a point in a triangle of the reference to the point in the target.

• This function interpolates the mapped coordinates of the triangle vertices by


Where barycentric coordinates are obtained byP1

P2 P3

P

is the area of triangle


• An image matching energy , ,which accounts for the total square matching error over domain becomes

Where is the set of triangles, , is the domain of triangle , and represent multi-modal image values, and is a weighting matrix.


• To exploit different surface attributes during matching , weighted sums of image attributes are used by means of diagonal weighting.

• The weights are determined according to both variance and importance of the channel.


• The driving forces that deform a reference mesh to a target come from minimization of the image matching energy.

• This energy can be minimized by using the gradients at each mesh node with respect to its mapped coordinate s


• The gradient at vertex is obtained through the chain rule as

is the set of triangles connected to the node , is the .vertex of triangle that corresponds to node , and is thus the .barycenteric coordinate for the point .


• The gradients are evaluated at each node of the mesh to update the mapping vector, , in a gradient decent scheme.

• The image gradients are evaluated using 3 X 3 Scharr mask.

J. Weickert and H. Scharr. A scheme for coherenceenhancing diffusion filtering with optimized rotation invariance.Jornal of Vis. Comm. Im. Rep., 13:103–118, 2002.

• We approximate the integrals in (4) and (5) by sampling at the recursively subdivided triangle centers.



• St. Venant Kirchoff material model.M. Bro-Nielsen. Medical image registration and surgery simulation.IMM-DTU

Ph.D. Dissertation, 1996 .

• The potential energy for St. Venant Kirchoof material is given by

Where and are the Lame material constants.


• Green-Lagrange strain tensor


• We use minimization of the Green-Lagrange strain tensor to attain elasticity. The deformation energy is defined as the Froebenius norm of this strain,


• Analytic expressions for gradients of with respect to vertex coordinated mappings must be obtained for the energy minimization task.

• With triangular mesh elements nonlinearities are not too complex and can be solved directly with a gradient descent scheme.

• During this minimization, stresses at each node of the mesh deform its neighboring triangles to regularize the displacements due to forces generated by image mating errors.

Multiresolution Registration Algorithm.


• The total energy minimized over the reference triangulation T is

• We minimize the total energy via gradient descent methods in a coarse-to-fine approach.

In this equation controls the amount of elasticity and thus the regularization during the registration


• To solve the problem in different scales:• First, Gaussian image pyramids are

constructed one at the reference, the other at the target.

• Second, at each resolution, meshes are generated adapted to the scale of the pyramid level.


• Registration starts from the coarsest level, and the estimated deformations are subsequently transferred to a finer level.

• The deformation transfer from one mesh to another is realized by barycentric mapping.

• For each node of a finer level, its barycentric coordinates at the coarser mesh are calculated, and thus mapping of that node is obtained by a weighted sum of the previous mesh node value.


• To produce meshes:• First, constrained Delaunay triangulation is

performed . The boundary vertices, are used as constraints in this triangulation. Feature points can also be used as additional constraints.

J. D. Boissonnat, O. Devillers, S. Pion, M. Teillaud, and M. Yvinec. Triangulations in CGAL. Comput. Geom. Theory Appl., 22:5–19, 2002.


• Second, a mesh generation algorithm which limits the maximum allowed triangle edge length is applied to the triangulation.

J. R. Shewchuk. Mesh generation for domains with small angles. Proc. 16th Annu., AACM Sympos. Comput. Geom.,1-10, 2000.


• Energy minimization restarts in one higher resolution level initialized with the values of the previous step.

• This procedure continues until the finest and final deformation estimates are found.


Reference target

Coarsest level

Finest level

Mesh deformation

Barycentric mapping


Result

Identities

Expressions

Result

• This algorithm is quite efficient.• The number of iterations till convergence

were in between 50 and 200, and total registration duration changed from 3 to 7 seconds using 3.0GHz CoreDuo processor.

Conclusion

• A new automatic algorithm to solve the non-rigid registration problem of two surfaces is presented.

• Registration is realized by deforming one surface onto the other one after mapping the onto a plane.

Conclusion

• This is achieved by first using a bijective mapping, and then utilizing finite element method to obtain the matching deformation.

• The differences of geometric and texture related surface attributes generates deformation forces, while elastic stresses regularize the resulting deformations.

Conclusion

• Nunlinear elasticity is used to cope with the large deformations by minimizing Green-Lagrange strain tensor.

• Also, following a multiresolution approach, good results can be obtained in a short time.

Future work

• Currently the algorithm is not designed to handle expression with open mouth.

• This team also plan to experiment using different surface attributes and investigate better strategies to combine these modalities.

Documents

Non-Rigid Registration of 3D Surface by Deformable 2D Triangular Meshes Speaker:James Chang