Level Set Based Selection Editing

  • Published on
    13-Feb-2017

  • View
    122

  • Download
    0

Embed Size (px)

Transcript

Adobe Title

Jaegwang Lim, Byungmoon Kim, Sunil HadapLevel Set Based Selection Editing

2012 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

1

Photoshop Selection ToolsQuick SelectionMagic WandMagnetic Lasso

Hard to modify the selection

2

Quick selection Magic wand Magnetic Lasso

2012 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

There are many selection tools in photo-shop: quick selection, magnetic lass and magic wand.

These tools are very easy to use but often inaccurate at selection boundaries.

So selection boundary needs local adjustment.

I tried to select one birds head using selection tools in photo-shop

As shown in the picture, we need to interactively edit selection boundary,

but it is not easy.2

Selection Using Brush-ToolUsing brush tools for selectionCan modify selection boundary by painting/erasingNot in incredibly awesome way, becauseShould create a separate layerSimply, brush is not designed for selectionNo automatic snapping to edgesNo interactive grow/shrinkNo direct control on smoothness of selection curves

3

BrushingSelectionRemove Brush layer

2012 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

To interactively edit selection, photo-shop users can use the brush tool for selection.

Using the brush tool, users can modify selection boundary.

But brush tool is not designed for selection so it is not a good way because

Users should create a separate layer for brushing and remove the layer after brushing selection region

Also brush tool can not smoothen or sharpen selection curves, and can not make the curve snapped to image edges automatically.

3

Our IdeaA New Selection ToolUse Special Selection BrushPaint regionSnapping To EdgeGrow/ShrinkCurvature Control

4

2012 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

So, we implemented a new selection tool.

It use special selection brush for painting region, snapping, grow/shrink and curvature control.

Selection curve snaps to image edge automatically and

We can make selection curve more smooth or sharp by controlling curvature of selection curve.

4

Our Idea : Grow/Shrink5

SinkingSource, Grow Sink, ShrinkSourcing5

2012 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

During sourcing/grow, level set values are set negative in red circle and velocities are set to grow in blue circle.

Growing velocity is vector computed from level set gradient.

The behavior looks similar to ink spreading out.

In contrast, when user erase selection region by using sink/shrink (ALT-left, or right mouse button).

Positive value are assigned in red circle and velocities are set to shrink in blue circle.

5

Our Idea : Brush Definition

6

2012 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

Red circle is hard region that always adds or removes selection.

Blue circle is soft region where selection grows or shrinks while snapping or smoothening curves.

6

Our Idea : Brush DefinitionDEMO7

2012 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

We can select similar to brush tool.

And we can adjust red and blue circle size.

Red circle is adding selection region, removing selection region.

Blue circle is growing selection, and shrinking selection.

7

Our Idea : SnapDetect edge and automatically snap to the edge Only inside the soft region

8

2012 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

Also in blue circle, snapping to image edge becomes possible by adding additional components to the velocity.8

Our Idea : SnapDEMO9

2012 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

I have increased snapping velocity.

And we can see snapping selection curve to image edge.

9

Level Set10

2012 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

We use level-set for that is a signed distance field stored on a grid.

Selected region is where level-set is negative and selection curve is where level-set is zero.

Outside selection region is filled out with positive distance.

Selection curves moves when level-set is updated by velocity field.

But level-set can be slow in high resolution images.

So we use adaptive quad tree grid and update the level-set only inside the cursor circle.

10

Previous Works on Level Set Based Image Segmentation11Cannot be used sinceEdge-based

Leak problemRegion-based

Useful only when foreground and background have different color (statistics)

We focus on local editing

2012 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

Edge based image segmentation is to partition an image into two or more regions.

The idea is to grow or shrink the boundary until it stops at feature boundary.

selection starts from inside and grow until it fills a feature.

Or, staring from outside of a feature, selection may shrink wrap the feature.

However if the feature boundary is weak, selection will spill.

Region based image segmentation does not have leak problem.

However, it work only when regions have different colors.

So we focus local editing.

11

Locally Solving Level Set on Adaptive Grid Use Grizzly quad tree template12

2012 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

Since our editing selection would be processed in real time. High performance is required.

So, we implemented adaptive grid. Using adaptive grid, we reduce unnecessary processing grid points effectively.

Also refining grid cell around selection curve, we can control selection curve more effectively in detail.

And we activate level set only locally inside the brush circle,

So the amount of computation is small even for very large image.

12

Snapping13

2012 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

For snapping. We need directions to image edge.

First of all, we convert RGB data of image to gray-scale data to make computing gradient of image easy.

Using LUMA equation, we get gray-scale data I

And then we compute gradient of gradient of I that mean a direction to image edge

So we can get image edge normal field.

13

Snapping14

Edge of image

Interface curveVelocity to edge

2012 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

Edge snapping velocity is the vector towards edge multiplied by gradient of image and snapping coefficient.

As we just update snapping velocity in blue circle, selection curve will be snapped to image edge automatically.

Gradient of image is large where image edge is strong.

So, where gradient of image is strong, snapping velocity is higher.

And where gradient of image is weak, snapping velocity is lower.

14

Modifying SelectionSelection could spill at weak image edgeBut we can easily modify selection.15

SpillWeak Edge

2012 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

In snapping selection curve to image edge.

As shown picture, Selection curve may spill at weak edge of feature.

But our level-set is very interactive so we can just modify selection curve.15

Modifying SelectionWe modify selection to overcome at strong image edgeThe closer to center of circle the stronger the velocity.Selection curve close to center of circle is easy to overcomed16

Strong edgeStrong edge can be overcomed with red circleif you want to select beyond

2012 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

Sometimes, we may select beyond a strong edge.

This may be done using that fact that the closer to center of circle the stronger velocity.

If it is not enough, the use can simply use the red circle.

16

Nudging (Dragging Curves)17Interface curve

DraggingVelocityIn progress

2012 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

Nudging is not implemented yet.

It is in progress.

We plan to implement way to drag and rotate selection curve with a control handle.

Using this control handle, the user can drag and rotate curve similar to convert anchor point tool in illustrator.

But unlikely to illustrator, in our selection editing, we could set control handle anywhere on the selection curve.

17

Demo18

2012 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

I will show you demo selecting some object.

If selection spills, we can just modify selection simply

At string edge region, to overcome string edge. We can close circle to selection curve.

18

Future Works19NudgingLocal region-based or graph-cut segmentation

2012 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

And we plan to use region based level-set to improve image segmentation.

19

2012 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

Done, thanks.20

Recommended

View more >