Motion Noise Motion Noise Separation In Digital VideoSeparation In Digital Video

E. E. A. Yfantis, Y. Terradas, P. Olson

F. Image Processing, Computer Graphics and Intelligent Systems Lab.

G. School of Computer Science, UNLV

1

Video Capture MechanismsVideo Capture Mechanisms

Mos-CCDs (Charge Couple Devices) Composite, S-Video, 3-ChannellCMOS (Complementary Metal Oxide Semi)FoveonAnalog to Digital Converter Video

DecodersCost-Noise

2

Noise SourcesNoise Sources

Electronic HardwarePower CablesCable Television-Cable InternetNearby AntennasAlternating CurrentLight Fluctuations Due to Atmospheric

fluctuations

3

Probability Distribution of Probability Distribution of TheNoiseTheNoise

The probability distribution of the noise depends on the channel, light intensity, quality of the camera, shape of the object, Atmospheric conditions, ground temperature, position of the camera with respect to the sun, refresh frequency and speed of the object.

4

16x16 Macroblocks for Motion16x16 Macroblocks for Motion

Motion Detection Is very important to Compression, Pattern Recognition, and many other Application Areas.

The 16x16 Macroblock motion Detection, Motion Estimation and motion compensation results into many missclassifications.

5

6

VOP’s EXTRACTION - VOP’s EXTRACTION - STEPSSTEPSMOTION DETECTIONMOTION DETECTION

For the motion detection process we For the motion detection process we made some assumptions:made some assumptions:

•The background is stationary.The background is stationary.

•We have access to the We have access to the background at the beginning of background at the beginning of the sequence.the sequence.

7

MOTION DETECTION (cont’d)MOTION DETECTION (cont’d)

•Getting information of the background: Getting information of the background: The first step of the motion detection process involves getting the edge-map of the background. To do so we used the Canny Edge Detector.

we thought that it would be helpful to, at first, to give a brief description on the implementation of the Canny’s Operator and later come back to the VOP extraction itself.

CANNY EDGE DETECTORCANNY EDGE DETECTOR

Summarizing all the steps within this edge Summarizing all the steps within this edge detector algorithm we have: detector algorithm we have: smoothingsmoothing, , gradient gradient calculationcalculation, , non-maximal suppression, non-maximal suppression, and and hysteresis. hysteresis.

8

CANNY EDGE DETECTOR (cont’d)CANNY EDGE DETECTOR (cont’d)

SMOOTHING:SMOOTHING: This process is done by applying a This process is done by applying a Gaussian filter to the image. Gaussian filter to the image. The Gaussian smoothing The Gaussian smoothing operator is a 2-Doperator is a 2-D convolution operator convolution operator that is used to `blur' that is used to `blur' images and remove detail and noiseimages and remove detail and noise. The implementation . The implementation of the Gaussian may be very expensive (many floating-of the Gaussian may be very expensive (many floating-point computations required). Yet, there is a way to create point computations required). Yet, there is a way to create an integer-valued mask that approximates very well this an integer-valued mask that approximates very well this floating-point mask. But we know that the Gaussian floating-point mask. But we know that the Gaussian distribution is near zero within three standard deviations distribution is near zero within three standard deviations from the mean. Hence, we can truncate the kernel at this from the mean. Hence, we can truncate the kernel at this point. The following mask is a suitable point. The following mask is a suitable integer-valuedinteger-valued convolution kernel that approximates a Gaussian with a convolution kernel that approximates a Gaussian with a of of 1.0.1.0.

1 4 7 4 1

4 16 26 16 41

7 26 41 26 7273

4 16 26 16 4

1 4 7 4 1

9

CANNY EDGE DETECTOR (cont’d)CANNY EDGE DETECTOR (cont’d)CANNY EDGE DETECTOR (cont’d)CANNY EDGE DETECTOR (cont’d)

Using a discrete approximation mask allows the MMX Using a discrete approximation mask allows the MMX technology to perform up to four operations at the same technology to perform up to four operations at the same time considerably reducing the execution time of the time considerably reducing the execution time of the process. Throughout this report I will use the CIF sequence process. Throughout this report I will use the CIF sequence named BOWING to show the results at each step. Right named BOWING to show the results at each step. Right now, I will show the resulting smoothed image of the frame now, I will show the resulting smoothed image of the frame number 73 from the video sequence mentioned above.number 73 from the video sequence mentioned above.

Gray Scale ImageGray Scale Image Smoothed ImageSmoothed Image

10

CANNY EDGE DETECTOR (cont’d)CANNY EDGE DETECTOR (cont’d)

By applying the Gaussian filter we are eliminating most of By applying the Gaussian filter we are eliminating most of the noise while we keep the most important edges of the the noise while we keep the most important edges of the image. The greater the image. The greater the , the stronger the blurring effect is. , the stronger the blurring effect is. The selection of the value for the standard deviation (The selection of the value for the standard deviation () ) depends, mainly, on the contents of the image (textures, depends, mainly, on the contents of the image (textures, shapes, signal/noise ratio, etc) as well as on the results we shapes, signal/noise ratio, etc) as well as on the results we are seeking. In this case I am working with a are seeking. In this case I am working with a =1.0.=1.0.

GRADIENT CALCULATION:GRADIENT CALCULATION: After smoothing the image After smoothing the image and eliminating the noise, the next step is to find the edge and eliminating the noise, the next step is to find the edge strength by taking the gradient of the image. strength by taking the gradient of the image. The main The main purpose of doing this ispurpose of doing this is to highlight regions with high to highlight regions with high spatialspatial derivatives.derivatives. There are two well known algorithms to There are two well known algorithms to find edges: one of them finds the second derivative of the find edges: one of them finds the second derivative of the smoothed image and looks for the zero crossing pixels, smoothed image and looks for the zero crossing pixels, while the other finds the first derivative of the smoothed while the other finds the first derivative of the smoothed image and the pixels that are maximum (locally).image and the pixels that are maximum (locally).

11

CANNY EDGE DETECTOR (cont’d)CANNY EDGE DETECTOR (cont’d)

Let us put the last statement graphically:Let us put the last statement graphically:

When finding edges we are looking for the steepest When finding edges we are looking for the steepest descend as well as the steepest ascend because both descend as well as the steepest ascend because both represent a high change in the intensity of the image. represent a high change in the intensity of the image. There are many ways and masks to perform the gradient There are many ways and masks to perform the gradient calculation. In our case we are going to apply an easy calculation. In our case we are going to apply an easy approximation of the derivative operation. We just find approximation of the derivative operation. We just find the difference between the intensity value of the two the difference between the intensity value of the two consecutive pixels in both directions (x and y). By doing consecutive pixels in both directions (x and y). By doing that we end up with two values per pixel named that we end up with two values per pixel named dxdxx,yx,y and and dydyx,yx,y..

12

CANNY EDGE DETECTOR (cont’d)CANNY EDGE DETECTOR (cont’d)For instance if we take the pixel at the For instance if we take the pixel at the x,yx,y position, then: position, then:

,x yI1,x yI 1,x yI

, 1x yI

, 1x yI

1, 1,,

, 1 , 1,

( )

2

( )

2

x y x yx y

x y x yx y

I Idx

I Idy

, , ,,x y x y x yI dx dy

Gradient at the pixel x,yGradient at the pixel x,y

Once we have the gradient value for each pixel we could Once we have the gradient value for each pixel we could get the magnitude and the orientation of the gradient by get the magnitude and the orientation of the gradient by doing:doing:

2 2

, , ,

,

,

arctan

x y x y x y

x y

x y

I dx dy

dy

dx

13

CANNY EDGE DETECTOR (cont’d)CANNY EDGE DETECTOR (cont’d)The output of the gradient calculation step consists of two The output of the gradient calculation step consists of two images each containing the values of the derivatives in images each containing the values of the derivatives in the horizontal and vertical direction, respectively. Both the horizontal and vertical direction, respectively. Both images are shown below.images are shown below.

Horizontal DerivativeHorizontal Derivative Vertical DerivativeVertical Derivative

The next step deals with the gradient and how to use it to The next step deals with the gradient and how to use it to find the local maximum pixel in its direction.find the local maximum pixel in its direction.

14

CANNY EDGE DETECTOR (cont’d)CANNY EDGE DETECTOR (cont’d)NON-MAXIMAL SUPRESSION:NON-MAXIMAL SUPRESSION: This step works with the This step works with the magnitude and orientation of the gradient at the pixel magnitude and orientation of the gradient at the pixel under consideration and creates 1 pixel-width edges. From under consideration and creates 1 pixel-width edges. From the previous stage we get the values of each component of the previous stage we get the values of each component of the gradient so we can get its magnitude and direction. the gradient so we can get its magnitude and direction. To calculate the direction of To calculate the direction of the gradient we don’t need to the gradient we don’t need to use the arctangent. Instead, use the arctangent. Instead, we just analyze the value and we just analyze the value and sign of the components of the sign of the components of the gradient. If we are at the gradient. If we are at the pixel pixel ppx,yx,y and the values of the and the values of the derivatives at that pixel are derivatives at that pixel are dx dx and and dydy, we can , we can approximate the direction of approximate the direction of the gradient at the gradient at p to one of the p to one of the sectors showed in the figure. sectors showed in the figure. For instance, if dx=-10 and For instance, if dx=-10 and dy=12 we can say that the dy=12 we can say that the gradient at this pixel goes gradient at this pixel goes approximately between 90 approximately between 90 and 135 degrees.and 135 degrees.

0

0

dy dx

dx

dy

0

0

dx

dy

dy dx

0

0

dx

dy

dy dx

0

0

dy dx

dx

dy

0 0

dx dy

dx dy

0 0

dx dy

dx dy

0 0dx dy

dx dy

0 0dx dy

dx dy

15

Once we have figured out the direction of the gradient we need Once we have figured out the direction of the gradient we need to interpolate the values of the pixels found in the to interpolate the values of the pixels found in the neighborhood of the point under analysis. The pixel that has no neighborhood of the point under analysis. The pixel that has no locallocal maximummaximum gradient magnitude is eliminated. The gradient magnitude is eliminated. The comparison is made among the actual pixel and its neighbors comparison is made among the actual pixel and its neighbors along the along the directiondirection of the gradient. Let us take again the last of the gradient. Let us take again the last example where we approximated the gradient orientation as to example where we approximated the gradient orientation as to be between 90 and 135 degrees, then we can build the next be between 90 and 135 degrees, then we can build the next figure.figure.

CANNY EDGE DETECTOR (cont’d)CANNY EDGE DETECTOR (cont’d)

gra

die

nt Px,y

Pa

Pb

To compare the values of the gradient we don’t need to apply To compare the values of the gradient we don’t need to apply the square root nor the power functions. We can just compare the square root nor the power functions. We can just compare the summation of the absolute values of the summation of the absolute values of dx dx and and dydy at each of at each of the pixels under consideration.the pixels under consideration.

1, 1 , 11, 1 1, 1 1, 1 , 1 , 1 , 12

, where and x y x ya x y x y x y x y x y x y

p pp p dx dy p dx dy

, 1 1, 1, 1 , 1 , 1 1, 1 1, 1 1, 12

, where and x y x yb x y x y x y x y x y x y

p pp p dx dy p dx dy

16

If the value at If the value at pp is is greatergreater than than bothboth ppaa and p and pbb then the pixel p then the pixel p is is a candidate to be an edge pixel, if not a candidate to be an edge pixel, if not pp is eliminated.After non- is eliminated.After non-maximal suppression we end up with an image that contains all maximal suppression we end up with an image that contains all the pixels that are a the pixels that are a local maximumlocal maximum. The corresponding image . The corresponding image would be like the one below. would be like the one below.

CANNY EDGE DETECTOR (cont’d)CANNY EDGE DETECTOR (cont’d)

The problem here is that we may The problem here is that we may have some pixels that despite have some pixels that despite being a local maximum, they being a local maximum, they represent noise. Therefore, we represent noise. Therefore, we can’t just take this as our final edge can’t just take this as our final edge map image. At this stage most of map image. At this stage most of the edge detectors apply a the edge detectors apply a threshold process. They define the threshold process. They define the threshold so that each pixel with a threshold so that each pixel with a value below the threshold is value below the threshold is eliminated. The trade off is that if eliminated. The trade off is that if we pick a high level value we we pick a high level value we assure that assure that

most of the noise is eliminated. However, we could be deleting most of the noise is eliminated. However, we could be deleting weak yet meaningful edges. On the other hand, if we utilize a weak yet meaningful edges. On the other hand, if we utilize a low value for the threshold we get all the important edges as low value for the threshold we get all the important edges as well as many of the noisy ones. Our next goal is to handle this well as many of the noisy ones. Our next goal is to handle this problem.problem.

17

CANNY EDGE DETECTOR (cont’d)CANNY EDGE DETECTOR (cont’d)Hysteresis:Hysteresis: Unlike single level thresholding we will select Unlike single level thresholding we will select twotwo levels : high threshold (T levels : high threshold (THH) and low threshold (T) and low threshold (TLL). ). RoughlyRoughly speaking speaking, edges start from pixels with gradients of , edges start from pixels with gradients of TTHH or more; they then extend over any connected pixels with or more; they then extend over any connected pixels with gradients of gradients of TTLL or more. If you think of following an edge, or more. If you think of following an edge, you need a gradient of you need a gradient of TTHH to start but you don't stop till you to start but you don't stop till you hit a gradient below hit a gradient below TTLL . In other words, for a given pixel, if . In other words, for a given pixel, if the gradient magnitude is below the gradient magnitude is below TTLL it is unconditionally set it is unconditionally set to zero. If the gradient is at least to zero. If the gradient is at least TTHH the pixel is left alone. If the pixel is left alone. If the gradient is between these two, then it is set to zero the gradient is between these two, then it is set to zero unless there is a path from this pixel to a pixel with a unless there is a path from this pixel to a pixel with a gradient above gradient above TTHH ; the path must entirely ; the path must entirely be be through pixels through pixels with gradients of at least with gradients of at least TTLL . The path can include diagonal . The path can include diagonal moves - i.e. we use 8-connectivity. This algorithm does not moves - i.e. we use 8-connectivity. This algorithm does not use the gradient direction estimates to follow edges - it use the gradient direction estimates to follow edges - it simply treats an edge as a connected set of pixelssimply treats an edge as a connected set of pixels.. The The likelihood of streaking is reduced drastically since the line likelihood of streaking is reduced drastically since the line segment points must fluctuate above the upper limit and segment points must fluctuate above the upper limit and below the lower limit for streaking to occur.below the lower limit for streaking to occur. One more time One more time the selection of Tthe selection of THH and Tand TLL depends on the type, content and depends on the type, content and predicted signal-to-noise ratiospredicted signal-to-noise ratios of the image. of the image.

18

CANNY EDGE DETECTOR (cont’d)CANNY EDGE DETECTOR (cont’d)If we now apply hysteresis to the output image from the non-If we now apply hysteresis to the output image from the non-maximal supression step we get the following images for two maximal supression step we get the following images for two different threshold levels.different threshold levels.

TL=50 TH=100TL=50 TH=100 TL=50 TH=200TL=50 TH=200

Once we get the edges map we can just set all the pixels that Once we get the edges map we can just set all the pixels that are edges (greater than zero) to 255 so they are clearly visible. are edges (greater than zero) to 255 so they are clearly visible. There is one more step called There is one more step called Feature Synthesis Feature Synthesis but it is not but it is not widely used because the results obtained so far are good widely used because the results obtained so far are good enough for most purposes.enough for most purposes.

19

MOTION DETECTION (cont’d)MOTION DETECTION (cont’d)

•Getting information of the background:Getting information of the background: Going back to the first step of the VOP extraction we need to get the background edge map using the Canny Operator just explained. The goal is to get as much information as possible from the background since that information will be used to identify the pixels that belong to the background. Once again, the selection of the variables for the Canny edge detection depends on the background type. For instance, if our background is cluttered we may need to select a small value of and low threshold levels for the hysteresis process. This is mainly because the greater the amount of information we get from the background, the easier the detection of the pixels that belong to the background will be. As we can see the selection of the parameters depends on every scene and should be readjusted when the background or the scene changes, if it does. In our example I work with a simple background and normal levels of threshold will be selected. The standard deviation will be set to 1.0 so we can use the discrete Gaussian mask shown before. If we need another standard deviation value we can create the corresponding integer-valued kernel.

20

MOTION DETECTION (cont’d)MOTION DETECTION (cont’d)The following images show the original background frame and The following images show the original background frame and the corresponding edge map.the corresponding edge map.

This image will be saved so we can use it every time we need it. This image will be saved so we can use it every time we need it. The next step will take care of two consecutive frames and will The next step will take care of two consecutive frames and will look for motion between them.look for motion between them.

21

MOTION DETECTION (cont’d)MOTION DETECTION (cont’d)•Detecting moving objects: Detecting moving objects: At this point our goal is to identify the objects that are moving in the scene. The following scheme briefly shows all the steps needed to find them.

Eliminating background pixels

1

( ) :

Pr ( ) :

:

t

t

backd

Current Frame RGB F

evious Frame RGB F

Background Edges Frame E

Gray Scale Process

tF1tF

Pixel-to-Pixel

Subtraction

Canny

Canny

tE

If pixel doesn’t belong to both

images eliminate it

tEdiffE

Linking pixels

tE

backdE

Moving Object Model Moving Object Model in current framein current frame

22

MOTION DETECTION (cont’d)MOTION DETECTION (cont’d)

Gray Scale Process

tF1tF

Pixel-to-Pixel

Subtraction

CannytE

23

MOTION DETECTION (cont’d)MOTION DETECTION (cont’d)

If pixel doesn’t belong to both

images eliminate it

tEdiffE

Linking pixels

Eliminating background pixels

Moving Object Model in current Moving Object Model in current frameframe

24

MOTION DETECTION (cont’d)MOTION DETECTION (cont’d)At this stage I am able to get the moving parts of the objects in At this stage I am able to get the moving parts of the objects in the scene. But VOP needs more than that. Each video object the scene. But VOP needs more than that. Each video object plane represents a plane represents a wholewhole moving object moving object not just part of it. Now not just part of it. Now we need to define the initial model of the object and once we we need to define the initial model of the object and once we got it we should start tracking it all along the sequence.got it we should start tracking it all along the sequence.

VOP’s EXTRACTION - VOP’s EXTRACTION - STEPSSTEPSINITIAL MODELINITIAL MODEL

Defining the initial model is not an easy task. Some methods Defining the initial model is not an easy task. Some methods take an arbitrary number of frames at the beginning of the take an arbitrary number of frames at the beginning of the sequence until they think they have enough information to sequence until they think they have enough information to define an accurate model for the moving object. It is really define an accurate model for the moving object. It is really difficult to define an initial model when the object shows up difficult to define an initial model when the object shows up gradually on the scene. In those cases the hardest task is to gradually on the scene. In those cases the hardest task is to decide whether the object has partially or completely showed decide whether the object has partially or completely showed up on the screen.up on the screen.

25

INITIAL MODEL (cont’d)INITIAL MODEL (cont’d)To handle more than one object at the same time I decided to To handle more than one object at the same time I decided to create a create a boxbox around each moving object. This box is created by around each moving object. This box is created by taking the left-most, right-most, up-most, and bottom-most taking the left-most, right-most, up-most, and bottom-most pixels in each object as its coordinates. This approach entails pixels in each object as its coordinates. This approach entails some difficulties. For instance, let us suppose that there are some difficulties. For instance, let us suppose that there are two moving objects in our scene. How can we identify each two moving objects in our scene. How can we identify each moving object if both appear at the same time on the screen? moving object if both appear at the same time on the screen? That issue as well as other considerations are open problems.That issue as well as other considerations are open problems.

Another very important issue is how to deal with objects that Another very important issue is how to deal with objects that stop moving for an arbitrary period of time or that have moving stop moving for an arbitrary period of time or that have moving and static parts. In that case applying the previous motion and static parts. In that case applying the previous motion detection method poses a problem when trying to get the whole detection method poses a problem when trying to get the whole object. That is why we need to introduce the object. That is why we need to introduce the Distance Distance TransformationTransformation. Distance transformations are widely used for . Distance transformations are widely used for object tracking. When properly used the DT can adapt to object tracking. When properly used the DT can adapt to rotations and changes in shape.rotations and changes in shape.

The idea of using a box can improve the tracking process The idea of using a box can improve the tracking process because we already know the actual position of the object and because we already know the actual position of the object and we don’t need to search along the whole image to find the best we don’t need to search along the whole image to find the best match. match.

26

INITIAL MODEL (cont’d)INITIAL MODEL (cont’d)Distance Transformation:Distance Transformation: The distance transform of a The distance transform of a black and white binaryblack and white binary image is an operation image is an operation that, for each that, for each pixel, pixel, computes thecomputes the distance to the nearest black pixeldistance to the nearest black pixel (edge pixel in our case)(edge pixel in our case). . DDistance transformistance transformationsations are based are based on many kinds of distanceon many kinds of distance functions functions given that given that the distance the distance functions differfunctions differ in in efficiency or usefulness.efficiency or usefulness. The distance I The distance I use is the Chamfer Distance. The use is the Chamfer Distance. The Chamfer distance Chamfer distance transforms are a class oftransforms are a class of discrete algorithms that offer a discrete algorithms that offer a good approximation to the desiredgood approximation to the desired Euclidean distance Euclidean distance transform at a lower computational cost. Theytransform at a lower computational cost. They can also give can also give integer-valued distances that are more suitable forinteger-valued distances that are more suitable for several several digital image processing tasks.digital image processing tasks. Chamfer offers many integer Chamfer offers many integer masks, in this case the 3-4 mask is the one I chose but there masks, in this case the 3-4 mask is the one I chose but there is another Chamfer mask called 5-7-11. is another Chamfer mask called 5-7-11.

4 3 4

3 3

4 3 4

p

Chamfer 3-4

27

INITIAL MODEL (cont’d)INITIAL MODEL (cont’d)The first step of the DT is to get the edge model of the The first step of the DT is to get the edge model of the object to be tracked. With the edge map we need to create object to be tracked. With the edge map we need to create a new image based on the weights. That is, we have to a new image based on the weights. That is, we have to initialize the output image with the following weights:initialize the output image with the following weights:

• If the pixel corresponds to an edge pixel then If the pixel corresponds to an edge pixel then assign a weight of 0assign a weight of 0

•If the pixel doesn’t correspond to an edge If the pixel doesn’t correspond to an edge pixel assign a high value (theoretically pixel assign a high value (theoretically ))

Now we have to work on the new weighted image. I used a Now we have to work on the new weighted image. I used a two passes algorithmtwo passes algorithm to build the DT image of the model. to build the DT image of the model.

Forward Pass:Forward Pass: This first pass goes from (0,0) to This first pass goes from (0,0) to (xmax,ymax). The pixels to be weighted in this pass are the (xmax,ymax). The pixels to be weighted in this pass are the ones showed on the figure.ones showed on the figure.

p

w1

w4

w2 w3If If p> 0, set p = min( p, min( gp> 0, set p = min( p, min( gii +w +wii )) i= 1, 2, )) i= 1, 2, 3, 43, 4 Where Where ggii represents the weighted represents the weighted distance according Chamferdistance according Chamfer and and wwii is the is the actual weight of the corresponding pixel.actual weight of the corresponding pixel.

28

INITIAL MODEL (cont’d)INITIAL MODEL (cont’d)Backward Pass:Backward Pass: It goes from (xmax,ymax) to (0,0). The It goes from (xmax,ymax) to (0,0). The pixels to be weighted in this pass are the ones showed on pixels to be weighted in this pass are the ones showed on the figure.the figure.

p

w3

w4

w2w1

If If p> 0, set p = min( p, min( gp> 0, set p = min( p, min( gii +w +wii )) i= 1, 2, )) i= 1, 2, 3, 43, 4 Where Where ggii represents the weighted represents the weighted distance according Chamferdistance according Chamfer and and wwii is the is the actual weight of the corresponding pixel.actual weight of the corresponding pixel.

Let us show an easy example of the DT.Let us show an easy example of the DT.

1

1

1

1

1

1

1 1 1

1

1

1

1

1

1 1

1

1

1

1

1

1

1 1

0 0 0 0 0 0

0 0 0

0 0 0

0 0

0

0 0

0 0

0 0

0 0

0 0 0 0

0 0

0 0

0 00 0

0

0

0

0

0

0

0

0

0

0

0

0 0

1

1

1

1

1

1 1 1

1

1

1

1

1

1 1

1

1

1

1

1

1 1

0

0

0

0

0 0 0

0 00000

0 00

0

0

0

0

0

0

0

03 3 3 3

3 3

3

3

333

3

33

3

3

333 3

4

4

4

4

3

3

3

3 3

3

3 3

3

4

3 3

3

36

6 4

47

36

7

6

4

Object’s Edge Map Object’s DT Map

DT1 edge pixel0 non edge pixel 1

29

The underlying here is the following: First get the edge The underlying here is the following: First get the edge model of the object to track and apply the DT to find the model of the object to track and apply the DT to find the binary image of the model. After that, use the binary image binary image of the model. After that, use the binary image to track the given object in the next frame looking for the to track the given object in the next frame looking for the best matching. To find the best match we can calculate the best matching. To find the best match we can calculate the total summation of the weights so we can deal with total summation of the weights so we can deal with rotations or changes in the scale.rotations or changes in the scale.

INITIAL MODEL (cont’d)INITIAL MODEL (cont’d)

Tracking the model:Tracking the model: The idea about creating a box is The idea about creating a box is really useful when tracking the moving object. However, really useful when tracking the moving object. However, there are many things to be taken into account:there are many things to be taken into account:

•When more than one object appear in the same When more than one object appear in the same scene we must try to identify each of them scene we must try to identify each of them separately.separately.

•Occlusion is one of the most challenging issues Occlusion is one of the most challenging issues here. In that case we have to recognize which here. In that case we have to recognize which object is occluding the other. The color and object is occluding the other. The color and texture information could help there.texture information could help there.

•We must also define when to stop tracking an We must also define when to stop tracking an object that hasn’t moved for a predetermined object that hasn’t moved for a predetermined period of time.period of time.

30

The detection, creation, and extraction of the video object The detection, creation, and extraction of the video object planes have involved a lot of work and given the variety of planes have involved a lot of work and given the variety of backgrounds, types of motion, ranges of motion, and many backgrounds, types of motion, ranges of motion, and many other factors it is a very interesting field of research.other factors it is a very interesting field of research.

TRACKING THE MODELTRACKING THE MODEL

RESULTSRESULTSRight now I’m working on the “box” previously mentioned Right now I’m working on the “box” previously mentioned and I’m dealing, mainly, with the total elimination of the and I’m dealing, mainly, with the total elimination of the noise from the background. Once I accomplish this, the noise from the background. Once I accomplish this, the tracking process should be easily implemented. There are tracking process should be easily implemented. There are three main steps regarding this implementation:three main steps regarding this implementation:

Motion DetectionMotion Detection

Model Initializing VOP ExtractionModel Initializing VOP Extraction

Model TrackingModel Tracking

31

RESULTS (cont’d)RESULTS (cont’d)So far I applied the algorithm for motion detection already So far I applied the algorithm for motion detection already shown in this report. The results I found are showed with shown in this report. The results I found are showed with the corresponding videos. Ass you will see when the object the corresponding videos. Ass you will see when the object stop moving I’m able to get only the portion of the object stop moving I’m able to get only the portion of the object that has moved.that has moved.

Sequence Name:Sequence Name: BOWING BOWING

Format:Format: CIF – 4:2:0 CIF – 4:2:0

32

Sequence Name:Sequence Name: HALL MONITOR HALL MONITOR

Format:Format: CIF – 4:2:0 CIF – 4:2:0

RESULTS (cont’d)RESULTS (cont’d)

THE ENDTHE END