Video Segmentation

Prepared ByPrepared By

NID@L M. AlburbarSupervised By:Supervised By:

Mr. Nael Abu Ras

University of PalestineInteractive Multimedia Application

Development I

Contents:

oIntroduction.

oVideo.

oSegmentation in Video.

oSegmentation using Motion.

oMotion detection

oImage differencing

oBackground subtraction

oAdvanced background subtraction

oVery advanced background subtraction

2

Introduction

Video segmentation is different from segmentation of a single image. While several correct solutions may exist for segmenting a single image, there needs to be a consistency among segmentations of each frame for video segmentation. 3

VideosoVideos are Image Sequences over Time

y

x

t

• 25 Images/s.• An image is a

function:

• At each time step t we have an image

• Frame rate = the number of images per second.

),(),,( yxftyxf t

),( yxf

4

5

Segmentation in VideooFinding the object(s)

oPreprocessing, segmentation

Knowledge baseProblemdomain

Imageacquisition

Preprocessing

SegmentationRepresentationand description

Recognitionand InterpretationResult

6

Segmentation

oSeparation of Foreground (object) and Background (everything else = noise)

oResult could be a

oBinary image

oContaining foreground only

oUseful for further processing, such as using silhouettesالظل , etc.

oApproach

oMotion

7

Segmentation using Motion

oAssume that only the object is moving => motion can be used to find the object

oMotion detection

oImage differencing

oBackground subtraction

oAdvanced background subtraction

oVery advanced background subtraction

8

Image Differencing

9

Image Differencing

oThe motion in an image can be found by subtracting the current image from the previous image

oAlgorithm1.Save image in last frame.2.Capture current camera image.3.Subtract image (= difference = motion).4.Threshold.5.Delete noise.

10

Subtract ImageoCompute pixel-wise

oSubtract previous image from input image:

oUsually the absolute distance is applied

),(),(),( yxByxIyxF

),( yxF1. Save image in last frame2. Capture camera image3. Subtract image4. Threshold5. Delete noise

11

Threshold

oDecide, when a pixel is supposed to be considered as a background pixel, or when it is to be considered as a foreground pixel:

oPixel is foreground pixel, if

oPixel is background pixel, if

oProblem: What TH?!?

),( yxTHyxF ),(

),( yxTHyxF ),(

1. Save image in last frame2. Capture camera image3. Subtract image4. Threshold5. Delete noise

12

Deleting NoiseoSingle pixels are likely to appear:

oPixel-noise!!

oApply Median filter:

oDepending on filter size, bigger spots can be erased

oAlternative: Morphology

1. Save image in last frame2. Capture camera image3. Subtract image4. Threshold5. Delete noise

(show: patch: diff)

13

Background Subtraction

14

Background Subtraction

o Foreground is moving, background is stable

o Algorithm1. Capture image containing background2. Capture camera image3. Subtract image (difference = motion)4. Threshold5. Delete noise

Advanced Background Subtraction

15

16

Advanced Background Subtraction

oWhat if we have small motion in the background?

oBushed, leaves, etc. and noise in the camera/lighting

o(show histo patch)

oLearn(!) the background

oCapture N images and calculate the average background image (no object present)1. Calculate average background image

2. Capture camera image3. Subtract image (= motion)4. Threshold5. Delete noise

17

Very Advanced Background Subtraction

18

Very Advanced Background Subtraction

oLearn the background and its variations!!oGaussian models (mean,var) for each pixel!!!

oThe more images you train on the better!!

oIdea: oSome pixels may vary more than other pixels

oAlgorithm:oConsider each pixel (x,y) in the input image and

check, how much it varies with respect to the mean and variance of the learned Gaussian models?

1. Calculate mean and variance for each pixel 2. Capture camera image3. Subtract image (= motion)4. Threshold according to variance5. Delete noise

19

Threshold According to Variance

oThreshold can be chosen depending on the varianceoA local threshold

oStandard Deviation

oFor example: oIf Thmin < diff. < Thmax => object pixel

oThmin = mean –

oThmax = mean +

1. Calculate mean and variance for each pixel 2. Capture camera image3. Subtract image (= motion)4. Threshold according to variance5. Delete noise

20

ConclusionoMotion segmentation

oImage differencing (two images)

oBackground subtraction (one bg. image)

oAdvanced background subtraction (many bg. images)

oVery advanced background subtraction (learn each pixel)

References:

ohttp://www.wisdom.weizmann.ac.il/~bagon/slides/Shai_Tal.ppt

ohttp://www.cvmt.dk/education/teaching/e07/MED3/IP/IP8-video.ppt

ohttp://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.37.7579&rep=rep1&type=pdf

ohttp://www.download-it.org/free_files/Pages%20from%20Chapter%206-1f21e0d0b871a64f220ccdd97ed6070c.pdf

21

THANKSFOR

LESTENING

22