Download pptx - Background Blurring

Background [email protected]

mailto:[email protected]

2

Outline Introduction

Documentation◦Object Segmentation◦Depth Map Generation◦ Image Defocus

3

Method

Lazy Snapping

Perspective Box

Pop-up Card

Camera Settings

Defocus BlurDefocus Blur

Privacy PreservingPrivacy Preserving

Photo BrowserPhoto Browser

Object Segmentation

Depth Map Generation

Applications

Image Defocus

Lazy Snapping

Perspective Box

Pop-up Card

Camera Settings

MethodSegment◦Lazy snapping◦Mean shift

5

Resize to 20%

Do hard graph cut by lazy snapping

User Stroke

Mean shift

Lazy Snapping. ACM Trans. On Graphics 2004.Mean Shift: A Robust Approach Toward Feature Space Analysis. PAMI 2002.A Bayesian Approach to Digital Matting. CVPR 2001.

MethodDepth map

◦ Perspective Box Vanish point Rear wall

Depth Map

Perspective Box

Normal vector

Pop-up card

6

Depth Map

Normal vector

Pop-up card

Tour Into the Picture: Using a spidery mesh user interface to make animation from a single image. SIGGRAPH 1997

MethodDepth map

◦ Pop-up card

Depth Map

Perspective Box

Normal vector

Pop-up card

7

Depth Map

Normal vector

Pop-up card

MethodDepth map

◦ Normal vector

Depth Map

Perspective Box

Normal vector

Pop-up card

8

Depth Map

Normal vector

MethodDefocus blur◦bokeh

𝐼𝑑𝑒𝑓𝑜𝑐𝑢𝑠 ሺ𝑖,𝑗ሻ= σ𝐸𝑛ሺ𝑖,𝑗ሻσ𝑤𝑒𝑖𝑔ℎ𝑡𝑛(𝑖,𝑗) + 𝛼𝑛(𝑖,𝑗)

𝑤𝑒𝑖𝑔ℎ𝑡𝑛ሺ𝑖,𝑗ሻ= 1 𝑎𝑟𝑒𝑎 𝑜𝑓 𝑅𝑒𝑔𝑖𝑜𝑛(𝑥𝑛,𝑦𝑛,𝑠,𝑏)Τ

𝛼𝑛(𝑖,𝑗) =൜𝑤𝑒𝑖𝑔ℎ𝑡𝑛(𝑖,𝑗), 𝐼ሺ𝑥𝑛,𝑦𝑛ሻ< 𝑇ℎ𝑏𝑜𝑘𝑒ℎ0, 𝐼ሺ𝑥𝑛,𝑦𝑛ሻ≥ 𝑇ℎ𝑏𝑜𝑘𝑒ℎ

9

𝑤𝑒𝑖𝑔ℎ𝑡𝑛ሺ𝑖,𝑗ሻ= 1𝑎𝑟𝑒𝑎 𝑜𝑓 𝑅𝑒𝑔𝑖𝑜𝑛ሺ𝑥𝑛,𝑦𝑛,𝑠,𝑏ሻ× 𝑝𝑜𝑠𝑖𝑡𝑖𝑜𝑛(𝑥𝑛,𝑦𝑛)𝑙𝑢𝑚𝑖𝑛𝑜𝑢𝑠(𝑥𝑛,𝑦𝑛)

……

……E1(i,j)

I(x1,y1) I(x2,y2) I(x3,y3)

I(xn,yn) I(xn+1,yn+1)

E2(i,j) E3(i,j)

En(i,j) En+1(i,j)

En+2(i,j)

I(xn+2,yn+2)

𝐸𝑛ሺ𝑖,𝑗ሻ= 𝐼ሺ𝑥𝑛,𝑦𝑛ሻ 𝑎𝑟𝑒𝑎 𝑜𝑓 𝑅𝑒𝑔𝑖𝑜𝑛(𝑥𝑛,𝑦𝑛,𝑠,𝑏)Τ 𝑤𝑒𝑖𝑔ℎ𝑡𝑛ሺ𝑖,𝑗ሻ= 1 𝑎𝑟𝑒𝑎 𝑜𝑓 𝑅𝑒𝑔𝑖𝑜𝑛(𝑥𝑛,𝑦𝑛,𝑠,𝑏)Τ x

Method

10

Defocus blur……

……E1(i,j)

I(x1,y1) I(x2,y2) I(x3,y3)

I(xn,yn) I(xn+1,yn+1)

E2(i,j) E3(i,j)

En(i,j) En+1(i,j)

En+2(i,j)

I(xn+2,yn+2)

Method

◦Near-by object case

11

Focus on the flower

Focus on the flower


Method Blur circle diameter

Objects in front of the focus

Objects behind

the focus

Defocus Blur

Texture synthesis inpainting

Blur shape as alpha map

Defocus Blur

Interpolation by alpha map

Shallow focus image 12

Experimental Result 1 Defocus blur method proposed in the system

comparing with other blur filter results

13Deep focus Shallow focus

Experimental Result 1 Defocus blur method proposed in the system comparing

with other blur filter results

Gaussian blur Defocus blur Defocus blur + Bokeh

14

Experimental Result 1

15

Deep focus

Shallow focus

Defocus blur


16

Gaussian blur Defocus blur + Bokeh

Original photograph

• Defocus blur method proposed in the system comparing with other blur filter results



17

Focus on the flower

Focus on the flower


Without inpainting

Interpolation result


18

Applications - Privacy Preserving

19

Applications - Image Transition at Photo Browser

20

21


Movie with defocus blur

effect 22


Documentation

23

./ 系統主要分成三部份，前兩塊產生出 depth 和segment 的資訊之後，可以在第三部份根據不同參數設定產生出不同的效果◦system

1_segmentation Tool1: lazy-snapping with simple UI Tool2: 光影魔術手

2_depthmap depth of background scene (3d-box) depth of each segment

3_refocusblur camera parameter refocusing (inpainting, defocus blur)

◦sample // sample input, parameter and result

24

Segmentation◦Lazy snapping

Tool1: preprocessing(resize for mean shift ) + lazy snapping

Lzsegment.m Input : parameter file name先做 preprocessing ，這裡可以用 meanshift 或是直接

downsample 成 64 色階，並且在 meanshift 的地方 resize為五分之一加快速度，

讀取 stroke_1.bmp, stroke_2.bmp… 等前後景筆劃圖，前景是紅色，後景是藍色。然後執行 lazy snapping 處理

◎ 每個 segment 都存一個灰階的 bmp 檔，檔名分別是seg_1.bmp, seg_2.bmp… 沒有先後順序。將每個 segment扣除剩下的 background 的部份，即是 seg_segN.bmp

◎ 需要 libgil2 的 library

Tool2: 光影魔術手每個 segment 都存一個 png 檔，檔名分別是 obj_1.png,

obj_2.png… 將這些 png 檔的 alpha 值轉成 bmp, 並且扣除只剩下 background 的部份 , 即是 seg_segN.bmp 25

Depth map◦depth.m (function depth(parafilename))

Input: parameter file name dep_1.dat, dep_2.dat, …dep_(segN-1).dat★ 每個 dep_n.dat 的內容 :

nx ny

p1x p1y

p2x p2y

… ★ n 表示 normal vector, 預設是 (0,0), vector 方向代表物體在

平面上的法向量方向，長度表示深度變化量 ★ p1,p2… 是 user 劃下的筆劃座標，即此 segment 的深度參考

點，預設是 segment 最下方的點。如果沒有 p1,p2… 系統會自動取 segment 最下方一列的點作為參考點。

Output: dep_1.bmp, dep_2.bmp … dep_segN.bmp

26

Refocus Blur◦ refocus.m ( function [finalresult]=defocus(parafilename) )

Get parameter from file, and load the segment/depth bmp files

Get the focus position from focusposition.dat Compute diffusion blur circle diameter for each

pixel Devide image into two parts, the part in front of the

focus, and run defocus_blur.m Use alpha.m to get alpha map, and find the pixels to

run inpaint.m Use the result by inpaint.m to call defocus_blur.m Combine the front part and back part by alpha map return the result image

27

others◦ Inpainting function needs the

estexemplarhelper.c run mex estexemplarhelper.c before the

function call at first timeThe inpainting code (inpaint.m) is from

http://www.cc.gatech.edu/~sooraj/inpainting/

◦Meanshift code is downloaded from◦http://www.caip.rutgers.edu/riul/

◦

28

http://www.cc.gatech.edu/~sooraj/inpainting/

http://www.caip.rutgers.edu/riul/



29

中間檔案 /參數 Generate from for

focuspoint.dat UI畫定 refocus.m 的聚焦點資訊

dep_1.dat, dep_2.dat …

UI畫定 depth.m每個segment的深度參考點和 normal值，如果沒有給予參考點則視為最下方一排

stroke_1.bmp, stroke_2.bmp…

UI畫定 Lazy snapping 的前後景分割依據

seg_1.bmp, seg_2.bmp…

lzsegment.m 產生，或是用 tool2取得obj.png之後，用

seg_png2bmp.m產生

depth.m和 refocus.m都需要用到的 segment map

dep_1.bmp, dep_2.bmp…

depth.m產生 refocus.m需要用到的深度資訊

v1,v2,a1,a2,b1,b2.. Vanish box和消失點座標 ,可用 UI介面畫

定

depth.m產生背景深度資訊所需要

其他 camera 參數使用者自行調整

Input parameter file 1.dirname diratory of the files, include '/' on the tail, ex: 'sample1/' 2.imagename file name of input image, ex: test.jpg 3.segN number of segment image, include the background.

ex: 2 (foreground*1+background) 4.vx position of vanish point, must be in the rear wall, 5.vy but can be outside of the image 6.ax 7.ay left_top position of the rear wall 8.bx 9.by right_bottom position of the rear wall 10.dc_f lens focal length,(mm) ex: 50 mm, 100 mm ...etc. 11.dpmax the depth of pixels with color level 255 in depth map, (mm) 12.dnear the depth of pixels with color level 0 in depth map, (mm) 13.dc_N the aperture number 14.thr=250 threshold of bokeh color level 15.light_e=0.5 little constant number for light_weight function 16.lens shape default setting 0 is flat disk

30

Camera parameter dc_f 相機的焦距，單位是 mm ，數值越大景深越淺，

建議在 50~100 之間 Dpmax rear wall 距離成像平面的距離，建議為 5~50 公尺

(5000~50000)

Dnear 場景裡距離成像平面最近的物體距離，建議為 1 公尺，至少要在兩倍焦距以上

dc_N 光圈值，數值越大，光圈孔隙越小，景深越深，為達到淺景深效果建議值在 1.4~2.8 之間，越小效果越

強 thr=250 將 pixel 產生散景效果的臨界值，建議不要低於 240 light_e=0.5 weight function 的明度微調參數，為一個大於 0 的值，

數值越大明度影響越低，建議為 0.5~1 之間 lens shape 光圈孔隙的形狀，預設為 0

對應到 refocus/lens/ 資料夾底下的 0.bmp, 1.bmp

31

Sample◦ ./

parameter01-.dat

◦ ./sample01 original image seg_1.bmp seg_2.bmp … dep_1.dat dep_2.dat …

可由 UI 產生 focusposition.dat

可由 UI 產生，給定要聚焦的點，可以為單一或是多個點，將會取其對應到的深度平均值，格式為

x1 y1x2 y2…

dep_1.bmp dep_2.bmp …

32

執行：◦用 system 的前兩步驟產生出前頁所述的參數和

bmp files ，並放在所屬資料夾下◦ result = refocus(parameter file name);

33

34

END

THANK YOU