Artiﬁcial Mosaic Generation: a Surveybattiato/download/IB3.pdf · mosaics. Islamic motifs are mainly geometric and mathematical [2]. In central Europe, mosaic went into general

Artificial Mosaic Generation: a Survey

Giovanni Puglisi and Sebastiano Battiato

Abstract The focus of the chapter is related to review techniques for the automaticgeneration of good quality digital mosaics from raster images. Mosaics, in the digi-tal realm, are illustrations composed by a set of small images called ”tiles”. The tilestessellate an input image with the aim of reproducing the original visual informa-tion rendered into a mosaic-like style. This chapter will review the major differentapproaches for digital mosaic generation reporting a short description and a discus-sion about the most relevant and recent issues. Particular emphasis will be devotedto techniques able to generate artificial mosaics that emulates in some way ancientmosaics both in terms of tile positioning and tile cutting procedures. Visual compar-isons among different approaches together with suggestions for future work will bealso provided.

1 Introduction

The first examples of mosaic goes back some 4,000 years or more, with the useof terracotta cones pushed pointfirst into a background to give decoration [1] asdemonstrated by unusual cone shaped-tesserae of various length found in Sumerianancient manufacts. Other relevant examples, realized in almost the same period, canbe found in some Egyptian and Phoenicia mosaics. After that we remind differentpebble pavements, using different coloured stones to create patterns, although thesetended to be unstructured decoration (about eighth century BC). In the four centuriesBC the Greek artists re-discovered the pebble technique to an art form, with precisegeometric patterns and detailed natural scenes. Next specially manufactured pieces(tesserae) were being used to give extra detail and range of colour to the work. Using

Giovanni PuglisiUniversity of Catania, Italy, e-mail: [email protected]

Sebastiano BattiatoUniversity of Catania, Italy, e-mail: [email protected]

1

2 Giovanni Puglisi and Sebastiano Battiato

small tesserae with different shapes and colors, the mosaics could imitate paintings.Many of the mosaics preserved at, for example, Pompeii were the work of Greekartists. Although not so famous it should be mentioned the important mosaic art inMexico due to the ancient Maya civilisation.

The next evolution in the field was due to the expansion of the Roman Empirealthough the level of skill and artistry was diluted. Typically Roman subjects werescenes celebrating their gods, domestic themes and geometric designs. Differentcharacteristics were exploited with the rise of the Byzantine Empire from the 5thcentury onwards, centred on Byzantium (now Istanbul, Turkey). In particular themain novelties were involved in style (i.e., eastern influences) and the use of specialmaterial (e.g., glass tesserae called smalti). Differently than before the Byzantinesspecialised in covering walls and ceilings. The smalti were ungrouted, allowing lightto reflect and refract within the glass. Also, they were set at slight angles to the wall,so that they caught the light in different ways. The gold tesserae sparkle as theviewer moves around within the building. Roman images were absorbed into thetypical Christian themes of the Byzantine mosaics, although some work is decora-tive and some incorporates portraits of Emperors and Empresses. Another impor-tant aspects of the mosaici history is based on the west of Europe where the Moorsbrought Islamic mosaic and tile art into the Iberian peninsula in the 8th century,while elsewhere in the Muslim world, stone, glass and ceramic were all used inmosaics. Islamic motifs are mainly geometric and mathematical [2].

In central Europe, mosaic went into general decline throughout the Middle Ageseven if the tile industry led to mosaic tiling patterns in abbeys and other majorreligious buildings. During Gothic Revival there was some influence of the medievalthemes. Some famous artist names like Antonio Salviati and Antoni Gaudı̀ were ableto give new emphasis to the mosaic world in the modern area.

Mosaics, in essence, are images obtained cementing together small colored frag-ments. Likely, they are the most ancient examples of discrete primitive based im-ages. In the digital realm, mosaics are illustrations composed by a collection of smallimages called ”tiles”. The tiles tessellate a source image with the purpose of repro-ducing the original visual information rendered into a new mosaic-like style. Thesame source image may be translated into many strikingly different mosaics. Fac-tors like tile dataset, constraints on positioning, deformations and rotations of thetiles are indeed very influent upon the final results. As an example, the creation of adigital mosaic resembling the visual style of an ancient looking man-made mosaic isa challenging problem because it has to take into account the polygonal shape of thetiles, the small size of the tiles, the need to pack the tiles as densely as possible and,not last, the strong visual influence that tile orientation has on the overall perceptionof the mosaic. In particular orientation cannot be arbitrary but it is constrained tofollow the gestalt choices made by the author of the source picture. Tiles, hence,must follow and emphasize the main orientations chosen by the artist.

Artificial Mosaic Generation: a Survey 3

2 Digital Mosaic Generation

The generation of a digital mosaic from a raster image can be formulated as a math-ematical optimization problem:

Given a rectangular region I2 in the plane R2, a tile dataset and a set of constraints, find Nsites Pi (xi,yi) in I2 and place N tiles, one at each Pi, such that all tiles are disjoint, the areathey cover is maximized and the constraints are verified as much as possible.

This definition is general and applicable in fields different than Computer Graph-ics. The first result related to this kind of problem for modeling human perceptionand automatic pattern recognition has been published by Harmon in [3]. Takinginto account the mosaic generation process four different definitions can be given tosolve specific problems:

Crystallization Mosaic - Given an image I2 in the plane R2 and a set of constraints (i.e.,on edge features), find N sites Pi (xi,yi) in I2 and place N tiles, one at each Pi, such thatall tiles are disjoint, the area they cover is maximized, each tile is colored by a color whichreproduces the image portion covered by the tile. In this case in order to allow a solutionthe requirements have to be relaxed asking only that the constraints are verified as much aspossible.

Ancient Mosaic - Given an image I2 in the plane R2 and a vector field φ (x,y) definedon that region by the influence of the edges of I2, find N sites Pi (xi,yi) in I2 and place Nrectangles, one at each Pi, oriented with sides parallel to φ (xi,yi), such that all rectanglesare disjoint, the area they cover is maximized and each tile is colored by a color whichreproduces the image portion covered by the tile ([4]).

Photo-mosaic - Given an image I2 in the plane R2, a dataset of small rectangular imagesand a regular rectangular grid of N cells, find N tile images in the dataset and place them inthe grid such that each cell is covered by a tile that “resembles” the image portion coveredby the tile.

Puzzle Image Mosaic - Given an image I2 in the plane R2, a dataset of small irregularimages and an irregular grid of N cells, find N tile images in the dataset and place them inthe grid such that the tiles are disjoint and each cell is covered by a tile that “resembles”the image portion covered by the tile.

The first two types of mosaics decompose a source image into tiles (with dif-ferent color, size and rotation), reconstructing the image by properly painting thetiles (Figg. 1, 2). The last two kind of mosaics are obtained by fitting images froma database to cover an assigned source image. They may hence be grouped togetherunder the denomination of multi-picture mosaics (Figg. 3, 4). This chapter will re-view several solutions focusing on Ancient Mosaic generation algorithms.

3 Ancient Mosaic

Ancient mosaics are artworks constituted by cementing together small colored tiles.A smart and judicious use of orientation, shape and size may allow to convey much


Fig. 1 Example of Crystallization Mosaic generated by using [5].

Fig. 2 Example of Ancient Mosaic generated by using [6].

more information than the uniform or random distribution of N graphic primitives(like pixels, dots, etc.). For example, ancient mosaicists avoided lining up theirtiles in rectangular grids, because such grids emphasize only horizontal and verticallines. Such evidence may distract the observer from seeing the overall picture. To


Fig. 3 Example of Image Mosaic generated by using [7].

Fig. 4 Example of Puzzle Image Mosaic generated by using [8].

overcome such potential drawback, old masters placed tiles emphasizing the strongedges of the main subject to be represented.

One of the earliest algorithms of artificial mosaic generation was developed byHausner [4] who also proposed the mathematical formulation of the mosaic problem


Fig. 5 Example of Ancient Mosaic generated by using [4].

(see Section 2). Hausner’s technique is an iterative approach in which the user se-lects important feature edges to be used for mosaic generation. An orientation fieldis then computed considering the gradient of the Euclidean distance transform fromthe edges. This field φ(x,y) follows edge orientation if (x,y) is close to the edge.Tile placing is then performed by using a modified version of Centroidal VoronoiDiagrams (CVD). CVD, that usually arrange points in regular hexagonal grids, hasbeen adapted to place tiles in curving square grids. This adaption is performed by us-ing Manhattan distance instead of the Euclidean one. Computing the CVD is madepossible by leveraging the z-buffer algorithm available in many graphics cards. Al-though this approach is able to produce good results (see Fig. 5) the convergence ofthe iterative algorithm could be a drawback if there is no direct access to the graphicacceleration engine.

Another strategy of artificial mosaic generation has been proposed in [10] and in[11]. The proposed approach tries to emulate mosaicist work making use of direc-tional guidelines and distance transform. First they segment the image by using Sta-tistical Region Merging algorithm ([12]) and divide the image into background andforeground regions. Although this step is optional, it permits to generate mosaicsthat mimic ”opus vermiculatum” style. Later, for each pixel the distance transformfrom the segmented region bounds is evaluated. The gradient matrix and the levelline matrix is then computed. Finally based on the previous steps tiles are place. The


authors, in order to obtain a high degree of similarity in terms of style with respect toancient mosaics, also consider tile cutting. Several examples of mosaics generatedby using [10] with and without considering tile cutting are shown in Figg. 6 and 7.

In [13] an interesting framework for stroke-based rendering based on a multi-agent system is proposed. These agents (RenderBots) represent one stroke and dur-ing the simulation disseminate themselves in the environment (a source image andpossibly additional G-buffer support images such as edge image, luminance image,etc.). The output of the algorithm is created at the end of the simulation when eachRenderBot executed its painting function. Different style can be created by using thesame framework: stippling, hatching, painterly rendering and mosaics. This high de-gree of flexibility is achieved by using a specific class of Renderbot for each style.Fig. 8 has been obtained by using RenderBots algorithm without user interaction.Better results can be achieved by using a manual segmentation.

In [14] a technique that tries to simulate the classic mosaic art form is presented.First, feature curves are extracted from the input image. Later, offset curves that gettrimmed-off the self intersecting segments with the guidance of Voronoi diagramsare computed. Finally, tiles are placed along the computed curves without overlap-ping. Although results are interesting (see Fig. 9(a)), this technique requires userinteraction in the selection of the edges to be enphatized.

In [15] an interactive technique able to create visually pleasing ancient mosaicsis proposed. The authors, starting from the study of ancient mosaics, derive a setof charactesistic features of a generic mosaic. Specifically, they consider the splicesamong tiles (should be costant), the variations of colors (tone and luminance), sizeand shape of the tiles. Moreover, tiles should be placed along feature lines of theinput image and their orientation should vary in a smooth way. To satisfy the above-mentioned properties, the authors designe a method based on the Lloyds algorithmfor CVT (Centroidal Voronoi Tessellation) computation and can be viewed as asmart extension and/or optimization of the technique proposed by Hausner [4].Instead of consider heuristics to automatically generate the artificial mosaic, theymodel their solution as an interactive tool. The user can arrange tiles of variousshapes and sizes, control the distribution process by adding additional data such ascontour lines and directional information. Moreover, tiles can be sized or shaped tobetter approximate the master image features. Although this algorithm obtains im-pressive results (see Fig. 9(b)) it requires user interaction depending hence on theuser’s aesthetic skill and experience.

A novel technique for ancient mosaics generation has been presented in [16]and further refined in [17]. Tile orientation field is generated considering the strongedges of the underlying image. Moreover orientations are forced to vary smoothlyin order to produce pleasing mosaics and reduce the gap between tiles. This fieldis obtained by using a global optimization approach (α-expansion algorithm) [18].The packing of the tiles is then performed in two steps. First a set of mosaic layers(M1,M2, ...,Mn) is generated. Starting from a random pixel p each mosaic layer iscreated with a region growing strategy based on a greedy assumption (the nearbypixel s that does not overlap and with the minimum gap space with respect to p ischosen). Later the mosaic layers are stitched together taking into account gap space


Fig. 6 Examples of Ancient Mosaic generated by using [10] without performing tile cutting.


Fig. 7 Examples of Ancient Mosaic generated by using [10] with tile cutting.


Fig. 8 Examples of Ancient Mosaic generated by using [13] without performing manual segmen-tation.

minimization, the absence of broken tiles and the crossing of strong edge inten-sity. This task is performed through the graph cuts algorithm [18]. Several mosaicsgenerated by the aforementioned approach can be seen in Fig. 10.

In [6, 19] the authors propose a mosaic generation approach based on GradientVector Flow (GVF) [20] computation together with some smart heuristics to drivetiles positioning. This vector field can be used to effectively drive tile positioning.Edge information is preserved, propagated in the close regions and merged togetherin a smoothly way. Once the orientation field has been computed, several heuristicsare emploied to follow principal edges and cover, as much as possible, the overallmosaic area. Several examples of mosaics generated by [6] are show in Fig. 11. Toobtain mosaics with irregular tiles, the authors extended their approach considering


(a)

(b)

Fig. 9 Other ancient mosaics: (a) Elber and Wolberg [14], (b) Fritzsche et al. [15].

two different strategies of tile cutting: subtractive and shared cut (see Fig. 12. Theformer cuts only the novel tiles, i.e., tiles that are not already present in the mosaic;the latter cuts both novel and already placed tiles. In order to improve the mosaicvisualization experience they also developed an application for the 3D mosaic ren-dering [21]. Each 2D tile becomes a 3D truncated pyramid with bottom base slightlybigger than the top one; oblique sides increase the effect of depth. By properly mod-ifying pixels’ coordinates adapting them to the 3D virtual environment a planar 3Dmosaic is simple obtained. A cement like background is also used to improve theoverall aesthetic effect. Several 3D surfaces can be obtained by simply tranformingpixel coordinates. The following surfaces have been considered in their approach:Plane, Dome, Cylinder, Pyramid. Several examples are reporte in Fig. 13.


Fig. 10 Examples of Ancient Mosaic generated by using [17].


Fig. 11 Examples of Ancient Mosaic generated by using [6] without performing tile cutting.


Fig. 12 Example of Ancient Mosaic generated by using [6] with tile cutting.


Fig. 13 Examples of Ancient Mosaic generated by using [21].


To summarize this Section the key of any technique aimed at the production ofdigital ancient mosaics is clearly the tile positioning and orientation. The methodspresented in this Section use different approaches to solve this problem, obtainingdifferent visual results. Some techniques are based on a CVD approach ([4], [14]and [15]) whereas other methods ([16], [6], [19]) compute a vector field by mak-ing use of different strategies (i.e., graph cuts minimization, gradient vector flow).Tile positioning is then performed with iterative strategies ([4], [14], [15], [16]) orreproducing the ancient artisans style by using a “one-after-one” tile positioning([10], [11], [6] and [19]). A different non-deterministic approach is used in [13].

4 Conclusion

References

1. I. Fiorentini Roncuzzi and E. Fiorentini, Mosaic: materials, techniques and history. MWeV,2002.

2. M. Du Sautoy, Symmetry: A Journey Into the Patterns of Nature. HarperCollins, 2008.3. L. Harmon, “The recognition of faces.,” Scientific American, vol. 229, no. 5, pp. 71–82, 1973.4. A. Hausner, “Simulating decorative mosaics,” in Proceedings of the 28th annual conference

on Computer graphics and interactive techniques (SIGGRAPH ’01), pp. 573–580, 2001.5. D. Mould, “A stained glass image filter,” in Fourteenth Eurographics Workshop on Rendering

(2003), pp. 20–25, 2003.6. S. Battiato, G. Di Blasi, G. Gallo, G. C. Guarnera, and G. Puglisi, “Artificial mosaics by

gradient vector flow,” in Short Proceedings of EUROGRAPHICS, 2008.7. A. FINKELSTEIN and M. Range, “Image mosaics,” in Proceedings of Raster Imaging &

Digital Typography (RIDT), pp. 11–22, 1998.8. J. Kim and F. Pellacini, “Jigsaw image mosaics,” ACM Transactions on Graphics, vol. 21,

no. 3, pp. 657–664, 2002.9. “Mosaic art in vitreous glass, millefiori, tesserae mosaics by shelby glass studio;

http://www.mosaic-tile-art.com/mosaic.html,” 2006.10. G. Di Blasi and G. Gallo, “Artificial mosaics,” The Visual Computer, vol. 21, no. 6, pp. 373–

383, 2005.11. S. Battiato, G. Di Blasi, G. Farinella, and G. Gallo, “A novel technique for opus vermiculatum

mosaic rendering,” in Proceedings of the 14th International Conference in Central Europe onComputer Graphics, Visualization and Computer Vision (WSCG’06), pp. 133–140, 2006.

12. R. Nock and F. Nielsen, “Statistical region merging,” IEEE Transactions on Pattern Analysisand Machine Intelligence, vol. 26, no. 11, pp. 1452–1458, 2004.

13. S. Schlechtweg, T. Germer, and T. Strothotte, “Renderbots-multi-agent systems for direct im-age generation,” Computer Graphics Forum, vol. 24, no. 2, pp. 137–148, 2005.

14. E. Elber and G. Wolberg, “Rendering traditional mosaics,” The Visual Computer, vol. 19,no. 1, pp. 67–78, 2003.

15. L. Fritzsche, H. Hellwig, S. Hiller, and O. Deussen, “Interactive design of authentic lookingmosaics using voronoi structures,” in In Proceedings of the 2nd International Symposium onVoronoi Diagrams in Science and Engineering VD 2005 Conference, pp. 1–11, 2005.

16. Y. Liu, O. Veksler, and O. Juan, “Simulating classic mosaics with graph cuts,” in Proceedingsof Energy Minimization Methods in Computer Vision and Pattern Recognition, pp. 55–70,2007.

17. Y. Liu, O. Veksler, and O. Juan, “Generating classic mosaics with graph cuts,” ComputerGraphics Forum, vol. 29, no. 8, pp. 2387–2399, 2010.


18. Y. Boykov, O. Veksler, and R. Zabih, “Fast approximate energy minimization via graph cuts,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, pp. 1222–1239,Nov 2001.

19. S. Battiato, G. Di Blasi, G. Gallo, G. C. Guarnera, and G. Puglisi, “A novel artificial mosaicgeneration technique driven by local gradient analysis,” in Proceedings of International Con-ference on Computational Science (ICCS’08) - Seventh International Workshop on ComputerGraphics and Geometric Modeling (CGGM’08), vol. 5102, pp. 76–85, 2008.

20. C. Xu and L. Prince, “Snakes, shapes, and gradient vector flow,” IEEE Transactions on ImageProcessing, vol. 7, no. 3, pp. 359–369, 1998.

21. S. Battiato and G. Puglisi, “3D ancient mosaics,” in Proceedings of ACM Multimedia Techni-cal Demos, 2010.

Documents

Artiﬁcial Mosaic Generation: a Surveybattiato/download/IB3.pdf · mosaics. Islamic motifs are mainly geometric and mathematical [2]. In central Europe, mosaic went into general