Natural-looking strokes for drawing applications

  • Published on

  • View

  • Download

Embed Size (px)


  • Visual Comput (2006) 22: 415423DOI 10.1007/s00371-006-0021-7 O R I G I N A L A R T I C L E

    Kyoko MurakamiReiji TsurunoEtsuo Genda

    Natural-looking strokes for drawingapplications

    Published online: 9 May 2006 Springer-Verlag 2006

    K. Murakami ()Tsuruno lab., Faculty of Design, KyushuUniversity, Fukuoka, Tsuruno E. GendaFaculty of Design, Kyushu University,Fukuoka, Japan{genda,tsuruno}

    Abstract This paper presents analgorithm for generating realisticdrawing strokes in real-time that cantake on the appearance of pastels,charcoals, or crayons. The similaritybetween the pigment deposit patternson paper surfaces produced by pastelstrokes and the shadows/shades onilluminated paper surfaces have beeninvestigated. Multiple paper textureshave been prepared and the papersurfaces have been ascertained by

    illumination from various directionsto represent strokes in arbitrarydirections. These textures havebeen processed as if they could beused as a height field, and pigmentsdeposited on the paper have beencalculated using the height field andtablet inputs.

    Keywords NPR Stroke Drawing Multiple paper texture

    1 Introduction

    The modeling of traditional drawing/painting materialsand techniques is an ongoing and open-ended topic inthe field of non-photorealistic rendering (NPR). A varietyof characteristics can be expressed in the final renderedimages through elaborate simulations. Because most realpictures are constructed through a series of strokes, manyNPR studies have focused on modeling the strokes them-selves. For each individual work, the stroke that is inherentin each painting/drawing tool is generated specifically foreach system and is affected by brush types, support media,and the interaction of these materials.

    When using drawing tools that have soft tips, like pen-cils, pigments are scraped from the tip and then physicallydeposited onto the surface of the support media. In thiscase, paper is often used as the support medium. Whenthe tip of the drawing tool is softer than that of pen-cils, including pastels, charcoals, chalks, or crayons, thepaper grains are not flattened by the stroke and the pa-per texture becomes visible. Realistic art tools cannot berepresented accurately without simulating the interactionbetween such materials. However, elaborate simulation re-

    quires considerable computation time for a drawing toolthat is to work in real-time. In this work, realistic draw-ing strokes are generated in real-time using twelve illu-minated paper surface samples. Instead of calculating theinteraction between each pigment and paper, the grada-tion of the illuminated paper surfaces is used as the heightfield, which requires less time. The method of Northrupand Markosian [12] is basically employed to generate thedrawing tools (a stroke and an eraser).

    2 Related works

    In several NPR studies on drawings, the results are highlyaffected by the support medium because of soft-firmedpigments, including pencils [16], colored pencils [18],charcoals [2], crayons [15] and pastels [11]. These stud-ies can be divided into two categories: texturing methodsand media-simulation methods. Several authors have pro-posed a texturing method for soft tips and pigments [2, 8]and [5]. Bleser et al. [2] generated a stroke from sampleimages of real charcoal strokes, and regenerated virtualcharcoal strokes by combining these sample images using

  • 416 K. Murakami et al.

    information from inputs generated by a five degree-of-freedom stylus. Scanning and regeneration methods havealso been used in some studies. Kalnins et al. [8] ren-dered various crayon and pencil strokes with a texturecreated from scanned images of several real strokes. Du-rand et al.[5] generated a threshold matrix from a scannedimage and represented various pen pressures and overlap.These texture-based methods are beneficial for both real-time rendering and generating realistic strokes. However,in real drawings, the textures that appear on strokes areaffected by various elements, such as pen pressure varia-tion, pen pressure distribution on the cross-section, cross-section form variation due to rubbing down or collapsingtips, and stroke direction. Several stroke samples cannotadequately represent the entire span of variation. Media-simulation methods of drawing tools can sufficiently rep-resent complicated pigment behavior [15, 16, 18] and [11].In some studies, all of the simulated media suitable for thesystem are generated. Takagi et al. [18] generated paperusing a volumetric approach that considered its construc-tion. They approximated colored pencil tips as spheresand used offset distance accessibility (ODA) [10] tomodel pigment deposition. Sousa and Buchanans papermodel [16] uses the height field from digitized real paperor photographed paper samples converted from 0.0 to 1.0.They also generated the 3D height field procedurally, asdetailed by Curtis et al. [4]. The height field is used in bothfaithful and simple simulations in research on various arttools. Tips of all pencil drawing tools are defined as poly-gons that have the pencil pressure at the center and the ver-tices of the polygon. Leads are scraped when they comeinto contact with the higher parts of the paper surface.The NPR by Rudolf et al. [15] defined the wax depositedby crayons on paper as the overlapping cross-sections ofthe crayon and the paper. Murakami and Tsuruno [11] de-posited particles of pigment on a paper surface to generatepastel strokes. All of these methods simulate drawing toolsthat allow for arbitrary stroke styles. Unfortunately, thesemethods also include some drawbacks. Suitable interac-tion is needed between the paper surface and the drawingtools in order to effectively regenerate the strokes, and nu-merous parameters are carefully set based on actual data toproduce realistic results. Furthermore, elaborate simula-tion requires considerable processing time in some cases,making it difficult to draw in real-time without a high-endwork station or GPU. For example, the crayon renderingof Rudolf et al. requires up to four seconds per strokewhen calculated on a 2.4-GHz Pentium 4. Computer gen-erated strokes are commonly used in existing paintingapplications at a real-time rate. Painter [3] determined theappearance of strokes according to the paper texture, andtheir shapes are a train of the stylus cross-section. Pho-toshop also uses the cross-section stylus for stroke shapedecisions, and adds slight rotation and translation to itsplacement. The strokes in Illustrator [7] are assigned ac-cording to a vector pattern placed along the base path.

    However, these strokes do not adequately resemble thoseproduced by real art tools. The proposed method simu-lates pigment deposition of drawing strokes by texturingthat combines these two approaches. Based on the strokegeneration for the real-time 3D NPR system, we have de-veloped a method for a drawing system that requires bothreality and interactivity.

    3 Paper model

    This paper focuses on the similarity between the appear-ance of drawn pastel strokes and paper texture that is illu-minated by a light. Multiple illuminated paper textures areused instead of simulating the interactions between toolsand paper surfaces. This method obtains realistic strokesthat can be used in real-time systems such as drawing ap-plications.

    3.1 Illuminated paper texture

    This section takes a case where a pastel stroke is drawnon paper as an example. The pastel is a typical drawing

    Fig. 1. Pastel strokes drawn in different directions

  • Natural-looking strokes for drawing applications 417

    Fig. 2. Microstructure of paper and deposited pigments

    Fig. 3. Height field from the paper height/illumination

    tool and is comprised primarily of pigments. Pastel strokesdrawn in different directions on a paper surface generatesdifferent pattern appearances, as shown in Fig. 1.

    Fig. 2 shows the zoom up of the paper and the pig-ment deposition by a pastel stroke as observed by animage scanner. When a stroke is drawn on paper, pigmentsscraped by irregularities in the paper are deposited notonly on top of the convex area, but also on the area nearthe top to a greater extent. Therefore, to generate a realis-tic stroke representation using the height field as a paperheight, a careful simulation that sometimes incorporates

    Fig. 4. The interface of thedrawing application

    the interaction between the drawing tool and the papersurface is necessary. In contrast, when an irregular papersurface is illuminated from one direction, causing shadeand a shadow to appear, the brightest part is not the highestpart, but an area slightly lower than the highest part in thedirection of the light. Fig. 3 represents this phenomenonusing a pattern diagram that depicts a cross-section of thepaper surface and its height field. Based on this similarity,the illuminated paper texture is used as the height field forsimulating pigment deposition by a stroke drawn in one di-rection. When a stroke is drawn, in fact, the stroke usesa height field consisting of the converted paper texture il-luminated by a light coming from the same direction as thestroke. In the presented application, twelve paper texturesare designated as key textures and interpolations betweenthem are performed for an arbitrary drawing direction.

    3.2 Preparing paper texture

    Paper textures are captured by the equipment shown inFig. 4. A camera is positioned above a piece of paper thatis attached to the flat ground. In order to illuminate thepaper in such a way as to distinguish paper irregularities,a light source is mounted at a suitable position with an el-evation angle of 45 degrees. The light intensity is arbitraryunless it obscures paper features. Next, the twelve surfaceimages are prepared. For each image, the paper surface isilluminated by a single light source with the azimuth set inincrements of 30 degrees from 0 to 360 degrees.

    Five types of paper textures were captured: drawingpaper, canson mi-tientes paper, mermaid paper, OK muse-cotton paper, and Watson paper. The sizes of the papersamples depend on the resolution and accuracy of the cap-turing process. The selected areas contain no warp causedby camera lens on the photographed paper image. As a re-sult, they range from 100 100 to 300 300 pixels. Thedrawing paper samples are shown in Fig. 4. In this fig-ure, the images are converted to 0-1 gray values, so thatthe minimum value in the 12 original images becomeszero and the maximum value becomes 1. This value isthen used as the deposit efficiency of the pigments. Theseconverted paper surfaces are referred to as paper tex-ture Ti, and each texture Ti has a direction vector to light

  • 418 K. Murakami et al.

    source Vt . The paper texture is tiled on the pigment bufferwith a wrap-over at the edge. The pigment buffer is thesame width and height as the resulting image, and the pig-ments are stored within it. The final image depends on theamount of pigment deposited in a pigment buffer.

    4 Implementation

    A stroke is defined as a stroke mask. Inside the strokemask, the number of pigments deposited in the pigmentbuffer is calculated by considering the interaction betweenthe pen pressure and the paper texture. All of the tools areproduced using basically the same method.

    4.1 Stroke mask

    There are various methods for generating strokes [6, 14,17]. In this study, a two-dimensional stroke mask based onthat of Northrup and Markosian [12] was used, which isideally defined as a train of cross-sectional surfaces. Thebase path consists of control points that are denotedby a mouse or tablet. WinTab API (LCS/TelegraphicsConfigdential) [9] is used for obtaining information froma tablet as packets. In the computer environment, the num-ber of packets is large enough to attain a smooth shapeusing only the linear interpolation between them. How-ever, a Catmull-Rom spline also can be used, such as thatof Kainins et al. [8], in the case when sparse control pointsneed to be defined. A stroke width is then added to bothsides of the base path. It is placed perpendicularly to thestroke vector that is calculated by two contiguous con-trol points. The locations for the stroke width are alsocalculated using a pen azimuth, as they are perpendicu-lar to the direction of pen azimuth. The pen azimuth is 0when the tip of a stylus indicates the top of the tablet, andreaches a maximum when the tip indicates the bottom. Thestroke contour is then defined as the stroke mask consist-ing of a triangle strip from the base path and the strokewidth. It is rare that a constant pressure is applied whena stroke is drawn. The convex part at the cross-sectionalsurface of the drawing tool receives more pressure thanthe other points, causing more pigment to be depositedon the paper surface. Furthermore, most drawing tools arefragile: convex parts are immediately flattened, causingnew convex points to appear. This results in variation ofgradation inside a stroke. In order to accurately representthis effect, a maximum pressure line segment consistingof maximum pressure points at each cross-section alongthe base path is implemented, as shown in Fig. 5. At eachtransverse of the stroke mask, a local 2D coordinate sys-tem is assigned in which x indicates the pressure positionc (0 for the rightmost side and 1 for the leftmost side ofthe stroke mask in the stroke direction) and y indicatesthe pen pressure F set vertically to the stroke mask. Then,

    Fig. 5. Stroke definition

    the maximum pressure points are set on the cross-sectionsthat contain the control points. The value c that decides thepositions of maximum pressure points are calculated bythe Perlin noise function [13], which is frequently used inrepresenting hand drawn-like appearance. The maximumpressure line segment is calculated by linear interpolationbetween the maximum pressure points. At the transversesof a stroke, a linear or a quadratic curve is used to inter-polate the c values at the maximum pressure line segmentand both sides where a minimum value exists. The min-imum value is set to 2/3 of the maximum value in ourexperiment. Controlling the coefficient and the intercept ofthe functions that set c values containing a maximum anda minimum value allows for variation in the stroke.

    4.2 Selecting paper texture

    After the stroke mask is set, the paper texture is applied.As described above, the paper texture that appears ona stroke varies with the stroke direction. Therefore, the ap-propriate drawing direction vector must be selected to ob-tain the appropriate paper texture. Furthermore, the pres-sure distribution displaces pigments inside the stroke. Thisis approximated by considering the movement of the max-imum pressure points on the stroke segments along thestroke when selecting the paper texture. When a stroke isdrawn in one direction, its direction vector Vs is used to se-lect two paper textures T1 and T2, which have Vt1and Vt2similar to Vs:

    Gs = Gt1(1 ratio)+ Gt2 ratio , (1)with Gs: the gray value for th...