Efficient Editing of Aged Object Textures

By:

Olivier Clément

Jocelyn Benoit

Eric Paquette

Multimedia Lab

2

Multimedia Lab

Introduction

Realistic image synthesis Virtual reality, video games, special effects,

etc. Aging (or weathering)

Many effects Many objects Time consuming

3

Multimedia Lab

IntroductionRedesign iterations

Produces theappropriate texture

Visualizes theappearance of an object

Reviewsthe result

4

Multimedia Lab

Outline

Objectives Previous Work Aging Editing Process

Segmentation Phase Elimination Phase Reproduction Phase

Results and Limitations Conclusion

5

Multimedia Lab

Objectives

To build a system To edit aging effects on textures To increase realism To reduce the amount of work Adapted for artists

adequate control interactive no complex parameters

6

Multimedia Lab

Outline

Objectives Previous Work Aging Editing Process

Segmentation Phase Elimination Phase Reproduction Phase

Results and Limitations Conclusion

7

Multimedia Lab

Previous Work

Physically based methods[Dorsey and Hanharan 2000; Merillou et al. 2001; O’Brien et al. 2002; etc.]

Highly realistic results but lengthy calculations Non-intuitive physical parameters

Empirical methods[Chain et al. 2005; Gobron and Chiba 2001; Paquette et al. 2002; etc.]

More intuitive parameters Both approaches

Do not provide the control required by artists Target a single aging effect

Aging methods

8

Multimedia Lab

Previous Work

Image based[Gu et al. 2006; Wang et al. 2006; etc.]

Capture the time-varying aspects of the material Similar to our approach

Focus of our approach Simple capture process Adequate control

Aging methods

9

Multimedia Lab

Outline

Objectives Previous Work Aging Editing Process

Segmentation Phase Elimination Phase Reproduction Phase

Results and Limitations Conclusion

10

Multimedia Lab

Aging Editing Process

Source image Image, photograph Containing aging

effects Target aging mask

Binary image Desired pattern

Reproduction image New aging effects

Process overview

11

Multimedia Lab

Aging Editing Process

Segmentation phase Semi-automatic Aged regions

Elimination phase Automatic Aging removed

Reproduction phase Automatic New aging effects

Phase description

Red

esig

n it

erat

ion

s

12

Multimedia Lab

Aging Editing ProcessImages summary

13

Multimedia Lab

Outline

Objectives Previous Work Aging Editing Process

Segmentation Phase Elimination Phase Reproduction Phase

Results and Limitations Conclusion

14

14

Multimedia Lab

Segmentation Phase

Identifies aged regions

Could be done with Segmentation tools Image editing software

Stroke-based technique Lischinski et al. [2006]

Worked efficiently for semi-automatic identification

15

Multimedia Lab

Segmentation PhaseStroke-base technique - Video

16

Multimedia Lab

Outline

Objectives Previous Work Aging Editing Process

Segmentation Phase Elimination Phase Reproduction Phase

Results and Limitations Conclusion

17

17

Multimedia Lab

Elimination Phase

Constrained texture synthesis Match the non-aged neighbourhood

Search using ANN library Arya et al. [1998]

The algorithm

best match

…

newbest match

Elimination image Source image

copy thepixel color

18

Multimedia Lab

Elimination Phase

The boundary pixels Non-aged pixels in

their neighbourhood Must be filled first

The aged region is filled iteratively

Hole-filling

19

Multimedia Lab

Outline

20

Objectives Previous Work Aging Editing Process

Segmentation Phase Elimination Phase Reproduction Phase

Results and Limitations Conclusion

20

Multimedia Lab

Reproduction Phase

Extension of the elimination algorithm

Consider the aged / non-aged context

The new term

21

Multimedia Lab

Reproduction Phase

Does not synthesize the entire image

Only specified regions

Iterative construction from multiple source images

Aging effects transfer and combination

22

Multimedia Lab

Outline

Objectives Previous Work Aging Editing Process

Segmentation Phase Elimination Phase Reproduction Phase

Results and Limitations Conclusion

23

Multimedia Lab

Results

Source image Elimination image Reproduction imageSource aging mask Target aging mask

24

Multimedia Lab

ResultsSource image Elimination image Reproduction image

25

Multimedia Lab

ResultsSource image Elimination image Reproduction image

26

Multimedia Lab

ResultsSource image Aging masks Reproduction image

More results in thepaper and the video…

27

Multimedia Lab

Results

User interaction is minimal Interactive computation time Efficient for redesign iterations

Efficiency

2.5 minutes - once25 seconds - once2 minutes

every iteration3 seconds

every iteration

Obtained on a PC with 3.2 GHz CPU and 3GB of RAM

28

Multimedia Lab

Limitations

Apply only on surfaces No fractures or deformations

Camera-based texture acquisition Specular lighting Surface distortion

Current implementation Interactive on textures up to 512 x 512

29

Multimedia Lab

Outline

Objectives Previous Work Aging Editing Process

Segmentation Phase Elimination Phase Reproduction Phase

Results and Limitations Conclusion

30

Multimedia Lab

Conclusion

A framework To edit aging effects on textures To reduce the amount of work needed during

the redesign iterations Benefits

Appropriate for artists adequate control and interactivity no complex parameters

Works well for several types of aging effects

31

Multimedia Lab

Conclusion

Synthesize the target aging mask For numerous regions Ex: scratches

Handle layers in effects combination Multiple effects over the same regions Ex: dirt on top of rust

Faster synthesis To handle higher resolution textures

Future work

32

Multimedia Lab

? We would like to thank :

And all our reviewers…

Questions

33

Multimedia Lab

Previous Work

Texture synthesis[Efros 1999; Hertzmann 2001; Kwatra 2003; Lefebvre 2006; Liang 2001; etc.]

Synthesis based on neighbourhood matching

Our system Extends from these algorithms Specializes for the aging context

Texture synthesis

34

Multimedia Lab

Previous Work

Image analogies, Hertzmaan et al. [2001] The output image is completely synthesized Our approach uses a similar algorithm that

synthesize only regions of the output Our approach should be considered as an

extension

Texture synthesis

35

Multimedia Lab

Elimination Phase

The replacement pixel is : Selected from the non-aged pixels of the source

image One of the best neighbourhood matches

The system seeks a replacement pixel that minimizes the following L2 norm :

The replacement pixel

36

Multimedia Lab

Elimination Phase

An exhaustive search would require processing time far from interactive

Thus, an approximation of the best match is found with the ANN library (Arya et al. [1998]) Approximate nearest neighbour searching algorithm

based on a kd-tree structure Our feature vector is composed of the RGB

components of the non-aged pixels around the pixel to replace

Interactivity