Upload
bryony
View
103
Download
5
Embed Size (px)
DESCRIPTION
Seam Carving for Content-aware Image Resizing. 資訊碩一 10077034 蔡勇儀 Date : 2012/01/03 @LAB 603. Outline. Introduction Basic Theory Application & Implementation Aspect Ratio Change Retargeting with Optimal Seams-Order Enlarging Content Amplification - PowerPoint PPT Presentation
Citation preview
Seam Carving for Content-aware Image Resizing
資訊碩一 10077034 蔡勇儀Date : 2012/01/03
@LAB 603
Introduction Basic Theory Application & Implementation
◦ Aspect Ratio Change◦ Retargeting with Optimal Seams-Order◦ Enlarging◦ Content Amplification◦ Seam Carving in the gradient domain◦ Object Removal
Multi-size Images Limitation Conclusions and Future Work
Outline
Muti-Media and Embedding System(e.g. Cell Phone) grow fast , Resize or Multi-Size scaling are more important than past.
Standard image scaling is not sufficient since it is oblivious to the image content and typically can be applied only uniformly.
For improve the problem, many researcher prove some good idea.
Introduction(1/2)
The following is main methods for scaling :◦ Corp (figure2(b))◦ Column or Row removal (figure2(c))◦ Pixel energy removal (figure2(e))◦ Optimal Pixel energy removal (figure2(f))◦ Object detection ◦ Seam Craving (figure2(d))
We can found the seam method have better result!
Introduction(2/2)
Introduction Basic Theory Application & Implementation
◦ Aspect Ratio Change◦ Retargeting with Optimal Seams-Order◦ Enlarging◦ Content Amplification◦ Seam Carving in the gradient domain◦ Object Removal
Multi-size Images Limitation Conclusions and Future Work
Outline
Step1 – Find seam◦ Find a path which have the minimum energy sum
from image top to bottom.
Step2 – Remove the Min. seam◦ When found all seam, select the Min. seam
remove.
Step3 – Repeat above step until get the demand size
Basic Theory
Give an energy function
Define Seam
Define the Seam Cost
Basic Theory – Step1
Find the minimum seam
Remove S* form image and lnstead of neighbors
Basic Theory – Step2
Repeat above step until get the demand size
Basic Theory – Step3
What’s energy function is the best?◦ e1
◦ Entropy 9*9 Windows add to e1
◦ Segmentation ( add to e1)◦ Histogram of Gradients
11*11cell around a pixel, 8-bins
Basic Theory – Energy Function
Basic Theory – Energy Function They all accommodate a similar range for
resizing.
We found either e1 or eHoG to work quite well.
Introduction Basic Theory Application & Implementation
◦ Aspect Ratio Change◦ Retargeting with Optimal Seams-Order◦ Enlarging◦ Content Amplification◦ Seam Carving in the gradient domain◦ Object Removal
Multi-size Images Limitation Conclusions and Future Work
Outline
Only one axis adjust
A picture size n*m n*m’ or n’*m n >= n’ m >= m’
Remove n-n’ or m-m’ seams
Enlarge at other page
Aspect Ratio Change
What’s the optimal order for remove seams? Column or Row or Other? How could decide?
Using dynamic programming
◦ where k = r+c, c = (m−m’), r = (n−n’)◦ αi is used as a parameter
that determine if at step i we remove a horizontal or vertical seam:
Retarget with Optimal Seam-Order(1/3)
Define transport map T◦ T(r,c)=min(T(r-1,c)+E(sy(In-r+1*m-c)), T(r,c-1)+E(sx(In-r*m-c+1)) )◦ where In-r*m-c
denotes an image of size (n−r)×(m−c), ◦ E(sx(I)) and E(sy(I))
are the cost of the respective seam removal operation.
Build the 1 bit map for record the direction
Retarget with Optimal Seam-Order(2/3)
Retarget with Optimal Seam-Order(3/3)
When m’ > m or n’ > n, we should insert seams to the picture.
Find the smallest energy seam for copy and insert, repeat until equal the demand scale.
But…
Enlarge(1/3)
Every time found the same seam, so we should decide all seams which need copy at first.
If m’ > m then we need insert (m’-m) seams. Find them and copy it for insertation.
Enlarge(2/3)
Enlarge(3/3)
The origin picture
Scalar Seam
Using same scalar enlarge then use seams-carving for recover to the origin size.
Content Amplification
If energy funciton use the gradient, then color show at remove place will be more nature after seam carving.
Seam Carving in the gradient domain
User mark the part which want to remove.
Decrease the energy on the part is removed.
Insert seams for keeping origin size.
Object remove
Introduction Basic Theory Application & Implementation
◦ Aspect Ratio Change◦ Retargeting with Optimal Seams-Order◦ Enlarging◦ Content Amplification◦ Seam Carving in the gradient domain◦ Object Removal
Multi-size Images Limitation Conclusions and Future Work
Outline
User want find the optimal picture scalar for their demand, so we need the real time opreation.
But the picture’s size 400*500 to 100*100 in
about 2.2 seconds, it is too long to real time.
How could do for real time?
Multi-size Images(1/3)
Make the index map for seams before user operation.
Build the horizontal & vertical index map (H&V)
But there will a big problem for operation that is H & V will be collided.
The sample solution is decide one just do one direction and then other direction need degenerate the index and redo the select seams operation
Multi-size Images(2/3)
Multi-size Images(3/3)
Introduction Basic Theory Application & Implementation
◦ Aspect Ratio Change◦ Retargeting with Optimal Seams-Order◦ Enlarging◦ Content Amplification◦ Seam Carving in the gradient domain◦ Object Removal
Multi-size Images Limitation Conclusions and Future Work
Outline
this method ◦ does not work automatically on all images.◦ can be corrected by adding higher level cues, either manual
or automatic. Figure 14, Figure 15 Other times,
◦ not even high level information can solve the problem. two major factors that limit this seam carving approach.
◦ The first is the amount of content in an image. If the image is too condensed, it does not contain ‘less important’ areas, then any type of content-aware resizing strategy will not succeed.
◦ The second type of limitation is the layout of the image content. In certain types of images, albeit not being condensed,the content is laid
out in a manner that prevents the seams to bypass important
Limitation(1/2)
Limitation(2/2)
Introduction Basic Theory Application & Implementation
◦ Aspect Ratio Change◦ Retargeting with Optimal Seams-Order◦ Enlarging◦ Content Amplification◦ Seam Carving in the gradient domain◦ Object Removal
Multi-size Images Limitation Conclusions and Future Work
Outline
to extend this approach to other domains, ◦ the first of which would be resizing of video. ◦ Since there are cases
when scaling can achieve better results for resizing, would like to investigate the possibility
to combine the two approaches, Specifically to define more robust multi-size images.
◦ would also like to find a better way to combine horizontal and vertical seams in multi-size
images.
Conclusions and Future Work
Q&A
Thank for your listening