Upload
others
View
8
Download
0
Embed Size (px)
Citation preview
__ University of Magdeburg
Pixel Manipulation of Images 1
Non-PhotorealisticComputer Graphics
Pixel Manipulation of Images
__ University of Magdeburg
Pixel Manipulation of Images 2
Contents
1. Basic Terms and Definitions
2. Halftoning
- Traditional Halftoning
- Digital Halftoning
- Applications to NPR
- Screening
- Screening with Texts
- Embedding Morphed Contours
3. Image Mosaics
__ University of Magdeburg
Pixel Manipulation of Images 3
1. Basic Terms and Definitions
• first approach: work on the level of the image’s pixels
• input: gray level image in our algorithms usually S(x,y)
• output: black/white or gray level imagein our algorithms usually O(x,y)
• also applicable to color imagesThe application of the discussed algorithms to color images can be derived relatively easy and should not be discussed in detail here.
• application of the algorithms from this chapter: as post-processing to a photorealistic renderer
apply algorithms to scanned images
• introduce artifacts
• goal in photorealistic CG: reduce or remove such artifacts
__ University of Magdeburg
Pixel Manipulation of Images 4
• pixel intensity
- monotonously between 0.0 and 1.0
- 1.0 black; 0.0 white
This notion is not uniformly used throughout the literature, some authors use it the other way around!
- intensity represents “blackness”
intensity i=1.0 intensity i=0.0intensity ramp
__ University of Magdeburg
Pixel Manipulation of Images 5
2. Halftoning
• Halftoning is the process of rendering an image on a display device with fewer colors than are in the image.
Halftoning is often also referred to as “Dithering” or “Color Reduction”
• well known image reproduction technique(s)
- traditional halftoning
- digital halftoning
__ University of Magdeburg
Pixel Manipulation of Images 6
2.1. Traditional Halftoning
• photographic process
• often used in former times for newspaper production
• requires the original photograph, a “halftoning screen” and a camera
• time consuming and cost intensive
• halftone screens:
- two thin glas plates with parallel lines cemented together such that the lines cross at right angles
- contact sheets made of film
• output image composed of a set of variably sized circles (dots)
• newspapers usually 60 to 80 variably sized and variably shaped areas per inch
• magazines and books: 100 to 200
__ University of Magdeburg
Pixel Manipulation of Images 7
Project a strong light onto the image. The lighter areas, or highlights, of theimage reflect more light, while the darkerareas, or shadows, absorb the light (andreflect less)
When the light is reflected through the camera, it is filtered through a halftone screen, which breaks up the image into a pattern of dots making a halftoned negative on the film. Areas of the original that reflect more light, create larger dots; areas that reflect less light create smaller dots.
__ University of Magdeburg
Pixel Manipulation of Images 8
Different number of lines per inch on the screen
Different angle between the lines on the two glass plates
33 lpi 53 lpi 75 lpi
0° 20° 45°
__ University of Magdeburg
Pixel Manipulation of Images 9
__ University of Magdeburg
Pixel Manipulation of Images 10
2.2. Digital Halftoning
• replace film and halftoning screen by the computer
• algorithmically reduce color depth
• many methods:
Threshold Quantization
Ordered Dithering
Error Diffusion
Screening
__ University of Magdeburg
Pixel Manipulation of Images 11
2.2.1. Threshold Quantization
• reduce gray levels to 2 by comparing with a given threshold value
input signal
outp
ut sig
nal
0
1
1
threshold
__ University of Magdeburg
Pixel Manipulation of Images 12
• simple algorithm: iterate over all pixels and do the comparison
• single parameter: threshold
• no control over induced artifacts
1 for x := 0 to width(S)-1 do
2 for y := 0 to height(S)-1 do
3 if S[x,y] > threshold
4 then O[x,y] := 1.0
5 else O[x,y] := 0.0
6 od
7 od
__ University of Magdeburg
Pixel Manipulation of Images 13
Threshold quantization
__ University of Magdeburg
Pixel Manipulation of Images 14
2.2.2. Ordered Dithering
• Replace areas of pixels with patterns of the same size as the area which correspond to the medium intensity of the area.
• Introduces artifacts from the patterns that are used.
• applications
- display devices
- printing
__ University of Magdeburg
Pixel Manipulation of Images 15
• dither patterns
__ University of Magdeburg
Pixel Manipulation of Images 16
• dither matrices
- different representation for dither patterns
- To achieve intensity i, turn on all pixels whose values are less than i.
01
2
3
4
5
6
7
8
__ University of Magdeburg
Pixel Manipulation of Images 17
• more dither matrices
__ University of Magdeburg
Pixel Manipulation of Images 18
1 for x := 0 to width//n step n do
2 for y := 0 to height//n step n do
3
4
5
6
7 od
8 od
R := get n×n region of the image
medium intensity = 63%
i := mediumIntensity(R)
P := pattern[i]
replace R with P
__ University of Magdeburg
Pixel Manipulation of Images 19
Ordered Dithering
__ University of Magdeburg
Pixel Manipulation of Images 20
2.2.3. Error Diffusion
• ordered dithering introduces artifacts
• When replacing a pixel of intensity i (0.0 i 1.0) by a pixel of intensity i being either 0.0 or 1.0, an error is introduced.
• basic idea: distribute this error to the neighboring pixels
• “smoothes” the image reduces artifacts
__ University of Magdeburg
Pixel Manipulation of Images 21
1 for y := height(S) to 1 step -1 do
2 for x := 1 to width(S)-1 do
3 K := approximate(S[x,y])
4 O[x,y] := K
5 error := S[x,y] - K
6 S[x+1,y] := S[x+1,y] + 7/16 * error
7 S[x-1,y-1] := S[x-1,y-1] + 3/16 * error
8 S[x,y-1] := S[x,y-1] + 5/16 * error
9 S[x+1,y-1] := S[x+1,y-1] + 1/16 * error
10 od
11 od
Floyd/Steinberg error diffusion algorithm;portions of the error term ar distributed
among neighboring pixels.
__ University of Magdeburg
Pixel Manipulation of Images 22
Floyd/Steinbergerror diffusion
__ University of Magdeburg
Pixel Manipulation of Images 23
2.2.4. Concluding Examples
__ University of Magdeburg
Pixel Manipulation of Images 24
__ University of Magdeburg
Pixel Manipulation of Images 25
2.3. Applications to NPR
• algorithms so far: reducing the number of intensity levels in an image while keeping the introduced artifacts at a minimum
This goal has been set in “photorealistic” image synthesis as well as in applications for printing and display devices
• our goal now: use the algorithms to introduce artifactsuse properties of some algorithms to achieve the opposite goal as stated above
__ University of Magdeburg
Pixel Manipulation of Images 26
2.3.1. Non-photorealistic Dither Matrices
• primary goals for the design of dither matrices:
- distribute pixels as evenly as possible
- prevent patterns
• interesting effects if pixels are grouped so that they form visible patterns
__ University of Magdeburg
Pixel Manipulation of Images 27
• build matrix starting from the bottom left element and moving diagonally up to the top right element
triangular patterns
• large matrices needed to achieve a visible effect
6101315
371114
14812
0259
4D
__ University of Magdeburg
Pixel Manipulation of Images 28
715110
38134
51492
110126
4D
• build matrix starting in the middle and moving outwards
linear artifacts
• large matrices yield better results
• in general: artifacts increase with matrix size
__ University of Magdeburg
Pixel Manipulation of Images 29
2.3.2. Halftoning Using Lines
• Error diffusion algorithms result in a proper distribution of “dots” (pixels).
• instead of setting single pixels draw lines
• if a line is drawn, images get much darker compensate with new error term
• variations in line slope, width, length …
__ University of Magdeburg
Pixel Manipulation of Images 30
1 for y := height(S) to 1 step -1 do
2 for x := 1 to width(S)-1 do
3 K := approximate(S[x,y])
8 O[x,y] := K
9 S[x+1,y] := S[x+1,y] + 7/16*error
10 S[x-1,y-1] := S[x-1,y-1] + 3/16*error
11 S[x,y-1] := S[x,y-1] + 5/16*error
12 S[x+1,y-1] := S[x+1,y-1] + 1/16*error
13 od
14 od The rest of the algorithm stays the same.
4 if (k=1) then
5 drawLine(O,x,y,m)
If we would set a pixel in the original, we draw a line which is m pixels long.
6 error := S[x,y]-K-(m-1)Instead of placing just 1 pixel, there are m pixels set now. These are (m-1) pixels to many the error term has to be reduced.
7 else error := S[x,y]-K
Nothing changes if there is no pixel set (line drawn).
• Example: Floyd/ Steinberg
__ University of Magdeburg
Pixel Manipulation of Images 31
__ University of Magdeburg
Pixel Manipulation of Images 32
2.3.3. Summary
• Artifacts from the dithering algorithms remain visible.
• parameters for lines (or dither patterns):
- random variations
Not a function of the image nor the portrayed geometry
- use attributes from the underlying geometry
So called “G-Buffers” have already been discussed
- ask user for additional input
__ University of Magdeburg
Pixel Manipulation of Images 33
2.4. Screening
• Ordered Dithering/Error Diffusion: almost no control over the introduced patterns
• unimaginative and uninformative patterns
• Screening:
combination of two images
viewed from a distance: texture in the original image
Viewed from close up: information of the second image
__ University of Magdeburg
Pixel Manipulation of Images 34
2.4.1. Basic Method
• given:
input gray-level image S
nm gray-level threshold matrix M
• Algorithm:
compare each pixel in S with a pixel in M
Use values in M as threshold, i.e. if value in S is larger than in M set the output pixel
• also possible:
repeat M over the area of S to get a “dither screen” D
do the comparison with pixels in D
This avoids the modulo operations!
__ University of Magdeburg
Pixel Manipulation of Images 35
1 for x := 0 to width(S)-1 do
2 for y := 0 to height(S)-1 do
3 if S[x,y] < M[x mod n,y mod m]
4 then O[x,y]:=1
5 else O[x,y]:=0
6 fi
7 od
8 od
S
M
O
__ University of Magdeburg
Pixel Manipulation of Images 36
1 construct D by repeating M over the image
2 for x := 0 to width(S)-1 do
3 for y := 0 to height(S)-1 do
4 if S[x,y] < D[x,y]
5 then O[x,y]:=1
6 else O[x,y]:=0
7 fi
8 od
9 od
__ University of Magdeburg
Pixel Manipulation of Images 37
Basic Screening
__ University of Magdeburg
Pixel Manipulation of Images 38
• texture determined by the threshold matrix
• arbitrary images can be used
• properties of the threshold matrix determine quality of the result
uniform distribution of threshold values
homogeneous spatial distribution of threshold values
• design matrices to meet these requirements or tune them afterwards
__ University of Magdeburg
Pixel Manipulation of Images 39
1. uniform distribution of threshold values
- dither screen should contain an equal number p of pixels of each possible intensity value (0 i imax)
- enables the uniform reproduction of the maximum range of gray tones
2. Homogeneous spatial distribution of threshold values
- pixels of like threshold value should be spread uniformly throughout the dither screen
- approximation of the same gray tone in different regions of the image in the same fashion
- automatically met when matrices with a uniform threshold distribution are replicated in a screen
__ University of Magdeburg
Pixel Manipulation of Images 40
2.4.2. Histograms and Their Use
• histogram:
- map of an image’s grayscale luminance values
- comes from statistics to describe (frequency) distributions
- aka: Probability Mass Function – PMF
describes the probability with which a pixel in the image has a certain intensity value
• histogram computation:
- count all pixels that have a certain intensity value i
- list the numbers in a “table”
__ University of Magdeburg
Pixel Manipulation of Images 41
1 for i:=0 to 255 do H[i]:=0
2 for x := 0 to width(S)-1 do
3 for y := 0 to height(S)-1 do
4 H[S[x,y]] := H[S[x,y]] + 1
5 od
6 od
7 for i:=0 to 255 do
8 H[i]:=H[i]/(height(S)*width(S))
9 od
2.4.2.1. Histogram Computation
• count all pixels that have a certain intensity value i
• list the numbers in a “table”
__ University of Magdeburg
Pixel Manipulation of Images 42
Example image and histogram
__ University of Magdeburg
Pixel Manipulation of Images 43
2.4.2.2. Histogram Equalization
• need to achieve a uniform distribution of gray values
• apply pixel level transformation techniques that change the intensity distribution
• most promising: Histogram Equalization
• compute cumulative sums of the histogram values:
• Cumulative Mass Function (CMF)The CMF is a statistical measure of the probability that a pixel is of a given intensity or less.
• alter the PMF using the CMF as a lookup table
• compute for each pixel a new gray value g´ by g´=255·C[g]
2550][][0
ikHiCi
k
__ University of Magdeburg
Pixel Manipulation of Images 44
1 H:=histogram(S)
2 C[0]:=H[0]
3 for i:=1 to 255 do C[i]:=C[i-1]+H[i] od
4 for x:=0 to width(S)-1 do
5 for y:=0 to height(S)-1 do
6 O[x,y]:=255*C[S[x]]
7 od
8 od
• Algorithm for histogram equalization
__ University of Magdeburg
Pixel Manipulation of Images 45
__ University of Magdeburg
Pixel Manipulation of Images 46
• Results of Histogram Equalization:
- histogram is spread out more uniformly
- gray value distribution is more uniform
• many HE algorithms possible
• Here: non-adaptive uniform histogram equalization (NAHE)
- works uniformly on the whole image
- transformation of one pixel independent from transformation of neighboring pixels
- drawback: enhances low-contrast detail, thus also increases contrast of noise in the image
__ University of Magdeburg
Pixel Manipulation of Images 47
2.4.2.3. Other options…
• … for achieving the two qualities required for dither screens
spatial homogeneity still not achieved
• subdivide the screen into small blocks and apply NAHE to each of these blocks
pixels of each block are transformed using PMF and CMF of this block only
good approximation of input gray levels
strong blocking artifacts
dither screen
__ University of Magdeburg
Pixel Manipulation of Images 48
• avoid these problems
- … by taking neighboring blocks into account
- Adaptive Histogram Equalization (AHE)
1 split input image into small blocks
2 foreach block b do
3 hb := CMF(b)
4 hb* := linear interpolation of the hb
of all neighboring blocks
5 Transform intensity values of b
based on hb*
6 od
__ University of Magdeburg
Pixel Manipulation of Images 49
__ University of Magdeburg
Pixel Manipulation of Images 50
2.4.3. Procedural Screening
• introducing texture in an image in photorealistic CG procedural textures
• Here: procedural dither screens
- created directly without the need for a second input image or image processing techniques
- can be produced at any scale
- can produce non-repeated textures of any size
• procedural dither screens
- have to satisfy the conditions for dither screens
- Spatial homogeneity can be achieved by periodically repeating the screen over the whole image.
concentrate on the uniform distribution of gray values
__ University of Magdeburg
Pixel Manipulation of Images 51
• Let (s,t) be a function over [0,1][0,1] with a uniform distribution of values.
• Let all values of be between 0 and 1:: [0,1][0,1][0,1]
• Define a,b to be a set of points such that a (s,t) b where a,b [0,1] and a < b
• |a,b| counts the number of all points in such a set
• A uniform distribution of values is achieved, when for any value of n (n>1) the value of is a constant for any value of i with 0i<n.
• Such functions are called dither kernels.
|| 1,
n
i
n
i
__ University of Magdeburg
Pixel Manipulation of Images 52
2.4.3.1. Dither Kernels
• compute an intensity level for each pair of input values s
and t
• for screening: map pixel positions to input values for such a dither kernel
• mapping function M to transform (x,y) into (s,t)
• Example:
describes a double sided ramp
most simple mapping: apply kernel to a small image block and repeat over the image
otherwise22
5.0if2),(
s
ssts
m
my
n
nxyxM
mod,
mod),(
__ University of Magdeburg
Pixel Manipulation of Images 53
1 define a dither kernel function
2 define a mapping function M
3 for x:=0 to width(S)-1 do
4 for y:=0 to height(S)-1 do
5 (s,t):= M(x,y)
6 i := (s,t)
7 if S[x,y]<i then O[x,y]:=1
else O[x,y]:=0
8 od
9 od
(s,t)
Input image Output imageMapping
Texture
__ University of Magdeburg
Pixel Manipulation of Images 54
• choose between different kernels and mappings
• dither kernel controls texture shape
• mapping function controls size and orientation of the texture
• example for a 2D dither kernel:
Any areas with an intensity below I are hatched using a linear ramp, in all other areas crosshatching is achieved
otherwise)1(
if),(
IsI
IsItts
__ University of Magdeburg
Pixel Manipulation of Images 55
modulo mappingrotation by 20°
modulo mappingrotation by 20° (I=0.7)
modulo mappingrotation by 20° (I=0.3)
modulo mappingrotation by 50°
modulo mapping
modulo mappingrotation by 50°
__ University of Magdeburg
Pixel Manipulation of Images 56
2.4.3.2. Texture variation
• adding a displacement function D T
Displacement functions take both variables, thus dependencies between s and t can be modelled. Also keep in mind that the requirements for dither screens have to be fulfilled
• image based control of texture
C(s,t) is an auxiliary image that contains a map of a special feature that has to be encoded in the resultant image
)),(),,((),( tsDttsDstsT ts
)),(),(),,(),((),( tsDtsCttsDtsCstsT ts
__ University of Magdeburg
Pixel Manipulation of Images 57
Veryovka and Buchanan: „Comprehensive Halftoning of 3D Scenes“. In: Proceedings of EUROGRAPHICS’99. Computer Graphics forum, vol. 18, no. 3, pp. C13-C21
__ University of Magdeburg
Pixel Manipulation of Images 58
__ University of Magdeburg
Pixel Manipulation of Images 59
__ University of Magdeburg
Pixel Manipulation of Images 60
2.4.3.3. Screening … and then …
• shape of introduced artifacts is controlled by a second image (or function)
• Designing dither screens manually …
select a pattern for the artifacts
observe the requirements for gray value distribution
• Is there any way to do it better?
screening with text and contours
__ University of Magdeburg
Pixel Manipulation of Images 61
2.5. Screening with Texts
• history:
- using an old printer (typewheel) and produce dark regions by overprinting several characters
- chose small characters (period, colon) for bright regions
- microletters on money bills
__ University of Magdeburg
Pixel Manipulation of Images 62
Pnueli, Y. and Bruckstein, A. M.“Gridless Halftoning: A
Reincarnation of the Old Method”Graphical Models and Image
Processing: GMIP, 58(1):38–64.
__ University of Magdeburg
Pixel Manipulation of Images 63
V. Ostromoukhov, R.D. Hersch: “Artistic Screening”
In: Proceedings of SIGGRAPH’95, pp. 219-228.
__ University of Magdeburg
Pixel Manipulation of Images 64
microletters on a german 10 DM bill
__ University of Magdeburg
Pixel Manipulation of Images 65
2.5.1. Preprocessing: Dither Rows
• render characters for different intensities in a block of given size
• increasing character size/width makes the block darker
• start with tiny and huge characters and interpolate
dither rows
__ University of Magdeburg
Pixel Manipulation of Images 66
V. Ostromoukhov, R.D. Hersch: “Artistic Screening” In: Proceedings of SIGGRAPH’95, pp. 219-228.
__ University of Magdeburg
Pixel Manipulation of Images 67
V. Ostromoukhov, R.D. Hersch: “Artistic Screening”
In: Proceedings of SIGGRAPH’95, pp. 219-228.
__ University of Magdeburg
Pixel Manipulation of Images 68
2.5.2. The Algorithm
1 foreach row of blocks r in S do
2 foreach block b in row r of S do
3 i := average intensity if block b
4 place letter L[i,b mod t] at the
next available position in O
5 od
6 od
__ University of Magdeburg
Pixel Manipulation of Images 69
2.5.3. An Example
__ University of Magdeburg
Pixel Manipulation of Images 70
2.6. Embedding Morphed Contours
• Text yields nice result but is not very “general”
• want to produce images like this and similar artwork by M.C. Escher
more scientific approach
- distinguish between
Screen Element Definition Space (SEDS) and
Screen Element Rendition Space (SERS)
- i.e. distinguish between
definition of a screen element
rendering of a screen element
- What is a Screen Element?
M.C. Escher Sky and Water
__ University of Magdeburg
Pixel Manipulation of Images 71
2.6.1. Screen Element
• any shape defined in its own coordinate system by its contour
• for example, one of our characters (letters)
• fixed contours are associated with fixed intensity levels
• shapes may be complex or simple – preferred defined via Bézier curves
__ University of Magdeburg
Pixel Manipulation of Images 72
2.6.2. The Algorithm
• four main steps:
1. define the shapes of the screen elements for key intensity levels
2. scale these shapes to achieve the desired intensity
3. interpolate to get shapes for all intensity levels
4. perform the actual screening
• screening process may be accompanied by an additional non-linear mapping to achieve distortions
__ University of Magdeburg
Pixel Manipulation of Images 73
2.6.2.1. Defining Screen Elements
• use a drawing program (e.g., Adobe Illustrator)
• Make sure that each contour is defined using the same number of control points.
• Screen Element Definition Space is the coordinate space in which the Screen Elemnts are drawn. Use a convenient coordinate system (e.g., 0 to 1).
• assign shapes to key intensity levels
__ University of Magdeburg
Pixel Manipulation of Images 74
2.6.2.2. Achieve Desired Intensity
• so far only shapes are defined
• scale these shapes such that they cover exactly a fraction of size i of the “block”
• i=0 is an infinitely small dot
__ University of Magdeburg
Pixel Manipulation of Images 75
2.6.2.3. Shapes for All Intensity Levels
• screen elements only defined for a fixed number of intensities interpolation to get shapes for all intensity levels
• usually linear interpolation
• interpolation speed can be controlled by parameters
__ University of Magdeburg
Pixel Manipulation of Images 76
• control interpolation speed by parameters i(z) varying between 0 and 1
• coordinates of a contour control point P at intensity z interpolated between two fixed contour points Pi and Pi+1
given by:
• Pi and Pi+1 belong to intensities zi and zi+1
• i(z) are mapped to the intensity range [zi,zi+1]
• i(z) can be defined to be any function
))(()))((1()( 0100 PPzPPzPzP iiii
__ University of Magdeburg
Pixel Manipulation of Images 77
V. Ostromoukhov, R.D. Hersch: “Artistic Screening”. In: Proceedings of SIGGRAPH’95, pp. 219-228.
__ University of Magdeburg
Pixel Manipulation of Images 78
2.6.2.4. The Screening Process
• so far everything in SEDS
• transfer screen elements into the output bitmap SERS
• enables screen elements to be defined independent from size or resolution of the output bitmap
• coordinate transformation from SEDS to SERS
• generate a discrete screen element for each of the 256 intensity levels
divide intensity interval between z=0 and z=1 by 255
Generate screen dot contours at levelsz=0, z=1/255, z=2/255, … z=1
rasterize screen dots by applying well known shape rasterization techniques
fill the contours appropriately
• select screen element based on the medium intensity of the target region (block)
__ University of Magdeburg
Pixel Manipulation of Images 79
V. Ostromoukhov, R.D. Hersch: “Artistic Screening”
In: Proceedings of SIGGRAPH’95, pp. 219-228.
__ University of Magdeburg
Pixel Manipulation of Images 80
2.6.2.5. Non-linear Mapping
• normally: A rectangular area in SEDS is mapped to a rectangular area in SERS (block of pixels).
• mapping can be defined as a non-linear function
• interesting effects but rasterization more complicatedconformal mapping: rectangular gridgrid following electro-magnetic field lines
fisheye lens mapping
V. Ostromoukhov, R.D. Hersch: “Artistic Screening” In: Proceedings of SIGGRAPH’95, pp. 219-228.
__ University of Magdeburg
Pixel Manipulation of Images 81
conformal mapping: rectangular gridgrid following electro-magnetic field lines
V. Ostromoukhov, R.D. Hersch: “Artistic Screening”
In: Proceedings of SIGGRAPH’95, pp. 219-228.
__ University of Magdeburg
Pixel Manipulation of Images 82
View of the Ibn Tulun Mosque V. Ostromoukhov, R.D. Hersch: “Artistic Screening”
In: Proceedings of SIGGRAPH’95, pp. 219-228.
__ University of Magdeburg
Pixel Manipulation of Images 83
View of the Ibn Tulun Mosque
V. Ostromoukhov, R.D. Hersch: “Artistic Screening”
In: Proceedings of SIGGRAPH’95, pp. 219-228.
__ University of Magdeburg
Pixel Manipulation of Images 84
2.6.3. Summary
• also known as “Artistic Screening”
• preprocessing time required for computing in SEDS
• non-linear mapping yields nice images
• microlettering
• Non-repetitive screens can not be photocopied and scanned without producing Moiré patterns.
__ University of Magdeburg
Pixel Manipulation of Images 85
V. Ostromoukhov, R.D. Hersch: “Artistic Screening”. In: Proceedings of SIGGRAPH’95, pp. 219-228.
__ University of Magdeburg
Pixel Manipulation of Images 86
3. Image Mosaics
• mosaic in arts: surface decoration of small colored components (stone, mineral, glass, tile, shell) closely set to an adhesive ground
• such mosaics can also be simulated
• image mosaics: small components are images themselves, together they portray a larger subject
__ University of Magdeburg
Pixel Manipulation of Images 87
© Adam Finkelstein, Princeton University -- http://www.cs.princeton.edu/gfx/proj/mosaic/blair.html
This is a picture of Blair Arch at Princeton University composed of 250 smaller pictures from around the Princeton campus. The picture was created in celebration of Princeton's 250th anniversary.
__ University of Magdeburg
Pixel Manipulation of Images 88
This image was created by dividing up a picture of Marilyn into small tiles, and then arranging those tiles in a grid. The arrangement was based on an algorithm which attempted to optimally suggest the big picture of JFK with the grid of tiles. Then the grid of tiles was used as a printer's screen to „halftone“' the JFK image. We ran the algorithm twice (using two different size images of Marilyn) and then cut-and-pasted between the two resulting images to select the best parts from each.
© Adam Finkelstein, Princeton Universityhttp://www.cs.princeton.edu/~af/cool/jfk-mm.html
__ University of Magdeburg
Pixel Manipulation of Images 89
• Four steps for producing an image mosaic
1. choose images which are to be used as tile images
2. choose a tiling grid
3. find an arrangement of the mosaic tiles within the grid
4. possibly perform a color correction on the tiles to match the tone of the target image
__ University of Magdeburg
Pixel Manipulation of Images 90
3.1. Choose Tile Images
• rather artistic choice
• tiles should match the subject of the target image (e.g., JFK and Marylin)
• small version of the target image itself, then color correction is necessary
• selection from an image database
__ University of Magdeburg
Pixel Manipulation of Images 91
3.2. Choosing a Tiling Pattern
• Question: How to arrange the tiles spatially?
• Also: How to overlay the target image with a grid where in each cell fits exactly one tile?
• several possibilities:
scattered layout
regular tiling grid using different tile shapes
multiresolution tiling
__ University of Magdeburg
Pixel Manipulation of Images 92
3.2.1. Scattered Layout
• no tiling grid at all
• random placement of the tiles
• tiles may or may not be rotated
• tiles may overlap
• looks like a huge pile of photographs
• for later: always compare the tile with the original image and not with the possible tile being already at a target position
__ University of Magdeburg
Pixel Manipulation of Images 93
Tom McKenna and Gonzalo R. Arce: „New Image Mosaic Structures“. Technical Report, University of Delaware, 1998.
Mosaic with Scattered Layout
__ University of Magdeburg
Pixel Manipulation of Images 94
T. McKenna and G. R. Arce:„New Image Mosaic Structures“. Technical Report, University of Delaware, 1998.
Mosaic with Scattered andRotated Layout
__ University of Magdeburg
Pixel Manipulation of Images 95
3.2.2. Regular Tiling Grid
• tiles placed regularily next to each other
• tiles do not overlap
• several tile shapes possible
__ University of Magdeburg
Pixel Manipulation of Images 96
Tom McKenna and Gonzalo R. Arce: „New Image Mosaic Structures“. Technical Report, University of Delaware, 1998.
Mosaic with RectangularGrid Tile Layout
__ University of Magdeburg
Pixel Manipulation of Images 97
Courtesy of Ken Chidlaw.
Mosaic with HexagonalGrid Tile Layout
__ University of Magdeburg
Pixel Manipulation of Images 98
3.2.3. Multiresolution Mosaics
• basic principle: tiles are mosaics themselves
• more and smaller tiles placed in regions with high detail
• fewer and larger tiles in regions with low detail
• need a method to break up the target image into the desired regions
• principle idea: use a quadtree, subdivisions based on the level of detail in the image
• measuring level of detail: identification of the frequency content of the image region
using a Fourier or wavelet transform. Regions with high frequent content can be considered very detailed.
Computing the average value of an edge detection algorithm. Regions with many edges can be considered very detailed.
• subdivision criterion for the quadtree is level of detail
__ University of Magdeburg
Pixel Manipulation of Images 99
__ University of Magdeburg
Pixel Manipulation of Images 100
T. McKenna and G. R. Arce:„New Image Mosaic Structures“.
Technical Report, University of Delaware, 1998.
Multi-ResolutionImage Mosaic
__ University of Magdeburg
Pixel Manipulation of Images 101
Tom McKenna and Gonzalo R. Arce: „New Image Mosaic Structures“. Technical Report, University of Delaware, 1998.
Combinationof Quad Tree Placementand Rotated Tiles
__ University of Magdeburg
Pixel Manipulation of Images 102
• more simple way to produce multiresolution mosaics:
- create a mosaic using just a few tiles
- use these mosaics as new tiles for the second step
- … and so on
• works best when using just one image (i.e., smaller versions of the target image as initial tiles)
• color correction necessary
__ University of Magdeburg
Pixel Manipulation of Images 103
The image of Mona is made up of a bunch of little images of Mona, which in turn are made up of even littler images of Mona, which you probably cannot see in this image.
© Adam Finkelstein, Princeton Universityhttp://www.cs.princeton.edu/~af/cool/mona.html
__ University of Magdeburg
Pixel Manipulation of Images 104
3.3. Arrange Tiles Within the Grid
• Given a target region, which tile should go there?
- use the same tile image everywhere – color correction necessary
- don’t care, just take any random tile – color correction necessary
- arrange tiles manually by hand – You don’t want to do this …
- select tiles by matching features of the tile and the target region
__ University of Magdeburg
Pixel Manipulation of Images 105
• match the average color of the target region and the average volor of the tile image
compute the mean squared error between the RGB channels of both regions (images)
select the tile with the smalles J(I,T)
• match different properties of the tartget region and the tile
match edges, texture, …
high computational effort
__ University of Magdeburg
Pixel Manipulation of Images 106
3.4. Color Correction
• if tiles not chosen in a way that the tile color matches the target region color
• following description for gray levels, can be extended to color by working with different color channels in a color model
RGB
YIQ
HSV
LAB
…
__ University of Magdeburg
Pixel Manipulation of Images 107
3.4.1. The Problem
• given an image tile T
average color of the tile: at
• given a target region R
average color of the region: ar
• Change the color of the tile’s pixels so that after the adjustment the tile’s average color is ar.
• Color Correction Rules:
- input: image tile, desired average color
- generate a color correction function F: R1 R1
__ University of Magdeburg
Pixel Manipulation of Images 108
3.4.2. Color Correction Functions
• constant function F(x)=a each pixel set to a
original colors completely ignored
mosaic with uniformly colored tiles
• color scaling function assume at larger than desired average color a
scaling factor for each pixel then a/at F(x) = (a/at)x
• color shifting function shift color of the tile’s pixel by difference between a and at
F(x) = x + (a - at)
may shift out of the allowed range
• more sophisticated functions other means of color correction
gamma correction etc.
__ University of Magdeburg
Pixel Manipulation of Images 109
3.4.2.1. Scaling and Shifting
__ University of Magdeburg
Pixel Manipulation of Images 110
3.4.2.2. Combination Scaling and Shifting
• sufficient results for image mosaics
• relatively easy to compute
• basic idea:
- shift colors when not leaving the allowed color range
- otherwise shift as far as possible ans scale the result until the desired average color a is achieved
Shifting and scaling colors to darken an image tile. The distributionis shown as a histogram. Dashed line indicates average.
__ University of Magdeburg
Pixel Manipulation of Images 111
3.4.2.3. An Example
• assume desired average color is less than the average color in the tile a < at
• If minimum color in tile mt is greater than the difference at-a then shift using F(x)=x+(at-a) without leaving allowed color range
• otherwise (if mt(at-a)) combine shift and scaling using F(x)=a(x-mt)/(at-mt)
• similar formula for symmetric cases
__ University of Magdeburg
Pixel Manipulation of Images 112
© Adam Finkelstein,Princeton University
Original gray ramp,dithered usingbasic screeningand different mosaicingtechniques
__ University of Magdeburg
Pixel Manipulation of Images 113
__ University of Magdeburg
Pixel Manipulation of Images 114
__ University of Magdeburg
Pixel Manipulation of Images 115
__ University of Magdeburg
Pixel Manipulation of Images 116