ECE 8433: Statistical Signal Processing Detection of Uncovered Background and Moving Pixels Detection of Uncovered Background & Moving Pixels Presented

ECE 8433: Statistical Signal Processing Detection of Uncovered Background and Moving Pixels

Detection of Uncovered BackgroundDetection of Uncovered Background& Moving Pixels& Moving Pixels

Presented By,Jignesh Panchal

OverviewOverview


Introduction of the ProblemIntroduction of the Problem Formulation of binary hypothesisFormulation of binary hypothesis

- Mathematical formulations- Mathematical formulations

- Formulation of Likelihood Ratio Test- Formulation of Likelihood Ratio Test

- Evaluation of LRT- Evaluation of LRT Analysis using noisy images (Gaussian White Noise)Analysis using noisy images (Gaussian White Noise) Extension of binary hypothesis to 3-ary hypothesisExtension of binary hypothesis to 3-ary hypothesis Performance analysisPerformance analysis Conclusion and future workConclusion and future work

IntroductionIntroduction


There is a very high demand for the video signal communication services. This There is a very high demand for the video signal communication services. This requires a tremendous quantity of digital data transmission. requires a tremendous quantity of digital data transmission.

Motion-Compensated interframe coding is the most effective method for reducing Motion-Compensated interframe coding is the most effective method for reducing the quantity of transmitted information. (redundancy) the quantity of transmitted information. (redundancy)

Degradation occurs, as these schemes do not provide uncovered background pixels.Degradation occurs, as these schemes do not provide uncovered background pixels.

Good coding scheme will use background prediction in addition to motion Good coding scheme will use background prediction in addition to motion compensation for interframe coding.compensation for interframe coding.

Detection based on Change Detection: but sensitive to Noise.Detection based on Change Detection: but sensitive to Noise.

Introduction (Contd.)Introduction (Contd.)


The basis of my method is hypothesis testing using Bayes decision criterion.The basis of my method is hypothesis testing using Bayes decision criterion.

This method directly considers image noise and is thus more robust than change This method directly considers image noise and is thus more robust than change detection.detection.

In this method, computationally expensive motion estimation is not required for In this method, computationally expensive motion estimation is not required for segmentation.segmentation.

The detection of uncovered background and moving pixels in image sequences is The detection of uncovered background and moving pixels in image sequences is an essential part of uncovered background prediction and motion compensation for an essential part of uncovered background prediction and motion compensation for sequence coding.sequence coding.

Mathematical FormulationMathematical Formulation


Consider two consecutive image frames in a sequence of image frames, and write Consider two consecutive image frames in a sequence of image frames, and write the noisy intensities of the first image frame as:the noisy intensities of the first image frame as:

zz11(k) = s (k) + w(k) = s (k) + w11(k)(k)

Where,Where,

k : Spatial Location (x, y) of a pixel in the image framek : Spatial Location (x, y) of a pixel in the image frame

zz11(k) : Noisy intensity of the pixel(k) : Noisy intensity of the pixel

s (k) : Noise-free intensity of the pixels (k) : Noise-free intensity of the pixel

ww11(k) : Zero mean white Gaussian noise(k) : Zero mean white Gaussian noise

Assuming no illumination changes, no camera motion, and no changes in image Assuming no illumination changes, no camera motion, and no changes in image acquisition parameter like camera focus, etc.acquisition parameter like camera focus, etc.

Mathematical Formulation (Contd.)Mathematical Formulation (Contd.)


The noise-free intensity at each pixel in the second or current frame can be The noise-free intensity at each pixel in the second or current frame can be modeled as a displaced value from the previous (i.e. first) frame or as an uncovered modeled as a displaced value from the previous (i.e. first) frame or as an uncovered background value.background value.

s (k - d (k)) + ws (k - d (k)) + w22(k),(k), k k € € γγbb

zz22(k) = (k) =

b (k) + wb (k) + w22(k),(k), k k € € γγbb

Where,Where,

d (k) : non-uniform displacement vectord (k) : non-uniform displacement vector

b (k) : intensity of the scene backgroundb (k) : intensity of the scene background

ww22(k) : zero-mean white Gaussian noise(k) : zero-mean white Gaussian noise

γγbb : region of uncovered background : region of uncovered background

LRT FormulationLRT Formulation


Assuming that d(k) is small enough such that the first-order approximation of Assuming that d(k) is small enough such that the first-order approximation of

s(k-d(k)) is valid.s(k-d(k)) is valid.

Defining,Defining,

ξξ(k) = z(k) = z22(k) – z(k) – z11(k)(k)

and,and,

w(k) = ww(k) = w22(k) – w(k) – w11(k)(k)

In this event the expression for intensity of second frame becomes,In this event the expression for intensity of second frame becomes,

zz22(k) = s(k) – g(k) = s(k) – gTT(k)d(k) + w(k)d(k) + w22(k)(k)

Where,Where,

g(k) = [gg(k) = [g11(k), g(k), g22(k)](k)]TT

-is the gradient vector of the intensity of the previous frame.-is the gradient vector of the intensity of the previous frame.

LRT Formulation (Contd.)LRT Formulation (Contd.)


Thus we obtain,Thus we obtain,

-g-gTT(k) d (k) + w(k),(k) d (k) + w(k), k k € € γγb b : H: H11

zz22(k) = (k) =

b (k) – s(k) + w(k),b (k) – s(k) + w(k), k k € € γγb b : H : H00

The likelihood ratio for the binary hypothesis test can be formed as:The likelihood ratio for the binary hypothesis test can be formed as:

p(ξ(k)|H1) = p( = p(ξξ(k)|(k)|HH11,d) p(d) ,d) p(d) dddd

Λ[ξ(k)] = p(ξ(k)|H1)

p(ξ(k)|H0)><H0

H1

η =P0

P1

∫

LRT Formulation (Contd.)LRT Formulation (Contd.)


Thus the Likelihood Ratio can be formed as:Thus the Likelihood Ratio can be formed as:

I(ξ(k))= ∫exp1

2(d-m)T C (d-m) dd

m = -C-1(g(k)/σ2w) ξ(k)

C = g(k) gT(k)/ σ2w + Kd

-1

Where,

Λ[ξ(k)] =

exp1

2mT C m I(ξ(k))

2π|Kd|1/2 ∫ exp-2ξ(k)q(k)+q2(k)

2σw2

p(q(k)) dq

Kd: Covariance Matrix

q(k) = b(k) – s(k)

3-Ary Hypothesis Formulation3-Ary Hypothesis Formulation


The analysis is further extended to a 3-ary hypothesis test to separate the non-background pixels into The analysis is further extended to a 3-ary hypothesis test to separate the non-background pixels into moving and stationary pixels.moving and stationary pixels.

z2(k) =

s(k-d(k)) + w2(k), k € γm

s(k) + w2(k), k € γs

b(k) + w2(k), k € γb

γm: Region of moving pixels

γs: Region of stationary pixels

γb: Region of background pixels

ξ(k) =

-gT(k)d(k) + w2(k), k € γm : H0

w(k), k € γs : H1

b(k) – s(k) + w2(k), k € γb : H2

Similarly defining ξ(k) and w(k) as in the binary case, the 3 hypothesis becomes,

3-Ary Hypothesis (Contd.)3-Ary Hypothesis (Contd.)


The likelihood ratios can be given as follows;The likelihood ratios can be given as follows;

Λ1[ξ(k)] = f [ξ(k)|H1]

f [ξ(k)|H0]Λ2[ξ(k)] =

f [ξ(k)|H2]

f [ξ(k)|H0]

P1(C01-C11)Λ1[ξ(k)]

P2(C02-C22)Λ2[ξ(k)]

P2(C12-C22)Λ2[ξ(k)]

P1(C01-C11) + P2(C01-C11)Λ2[ξ(k)]

P0(C20-C00) + P1(C21-C01)Λ1[ξ(k)]

P0(C20-C10) + P1(C21-C11)Λ1[ξ(k)]

><

H1 or H2

H0 or H2

><

H1 or H2

H0 or H2

><

H1 or H2

H0 or H2

Performance EvaluationPerformance Evaluation


Several test image sequences were generated to evaluate the binary and 3-ary Several test image sequences were generated to evaluate the binary and 3-ary hypothesis tests.hypothesis tests.

I have used both hypothesis tests on the images at several signal-to-noise ratios I have used both hypothesis tests on the images at several signal-to-noise ratios using a single measurement for classifying each pixel.using a single measurement for classifying each pixel.

The SNR is defined as,The SNR is defined as,

For binary hypothesis test, ROC curves are generated at several SNRs; whereas for For binary hypothesis test, ROC curves are generated at several SNRs; whereas for 3-ary hypothesis test confusion matrix is formed for the 20 dB SNR test case.3-ary hypothesis test confusion matrix is formed for the 20 dB SNR test case.

SNR = 10log10

average image variance

noise variancedB

Performance Evaluation (Contd.)Performance Evaluation (Contd.)


Confusion matrices are formed for several different cost matrices.Confusion matrices are formed for several different cost matrices.

All of the above tests require knowledge of the pdf of the difference between All of the above tests require knowledge of the pdf of the difference between background and object intensity, (i.e. q (k)), in the region of the uncovered background and object intensity, (i.e. q (k)), in the region of the uncovered background.background.

This pdf was determined by convolving the histogram of the background with the This pdf was determined by convolving the histogram of the background with the flipped histogram of the object and normalizing it.flipped histogram of the object and normalizing it.

Results & DiscussionResults & Discussion


A priori probabilities used are:A priori probabilities used are:

P = [ PP = [ P0 0 P P1 1 PP2 2 ]]

P = [0.1 0.8 0.1]P = [0.1 0.8 0.1]

The two different cost matrices used are:The two different cost matrices used are:

C1 = 0 1 11 0 11 1 0

C2 = 0 1 19 0 91 1 0

Results & Discussion (Contd.)Results & Discussion (Contd.)

Test Image Frame 1


Test Image Frame 2

Results & Discussion (Contd.)Results & Discussion (Contd.)

True Segmentation


White: Moving PixelsBlack: Stationary PixelsGray: Uncovered Background Pixels

Results & Discussion (Contd.)Results & Discussion (Contd.)SNR = 20 dB and using C1


SNR = 20 dB and using C2

MovingMoving StationarStationaryy

Uncovered Uncovered BackgrounBackgroundd

MovingMoving 8484 11 66StationaryStationary 88 9898 1313Uncovered Uncovered BackgrounBackgroundd

88 11 8181

Detected

True MovingMoving StationarStationaryy

Uncovered Uncovered BackgrounBackgroundd

MovingMoving 8989 66 77StationaryStationary 33 8989 44Uncovered Uncovered BackgrounBackgroundd

88 55 8989

Detected

True

In percentage at SNR 20 dBusing C1

In percentage at SNR 20 dBusing C2

ROC CurveROC Curve


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Probability of False Alarm

prob

abili

ty o

f D

etec

tion

ROC Curve

5dB10dB20dB

Another ExampleAnother Example


Segmentation

ConclusionsConclusions


In this project, I have presented a binary and ternary hypothesis test, based on In this project, I have presented a binary and ternary hypothesis test, based on Bayes decision criterion.Bayes decision criterion.

The goal of the project is the detection of moving, stationary, and uncovered-The goal of the project is the detection of moving, stationary, and uncovered-background pixels in image sequences.background pixels in image sequences.

The basic application of this method is in the video-teleconferencing and The basic application of this method is in the video-teleconferencing and videophone.videophone.

Future WorkFuture Work


Testing of this method, on real time teleconferencing image sequences. Testing of this method, on real time teleconferencing image sequences.

Active contour algorithm (snake algorithm) can be used to segment the person Active contour algorithm (snake algorithm) can be used to segment the person contour in an image frame.contour in an image frame.

Detection of the signal in presence of various types of noises, such as additive Detection of the signal in presence of various types of noises, such as additive white Gaussian noise (AWGN), color noise, etc.white Gaussian noise (AWGN), color noise, etc.

ReferencesReferences


““Non-uniform image motion estimation using Kalman filtering”, N. M. Namazi, P. Non-uniform image motion estimation using Kalman filtering”, N. M. Namazi, P. Penafiel, and C. M. Fan, IEEE Trans. Image Processing, Vol. 3, pp 191-212, 1989.Penafiel, and C. M. Fan, IEEE Trans. Image Processing, Vol. 3, pp 191-212, 1989.

““On regularization, formulation and initialization of the active contour models (snakes)”, On regularization, formulation and initialization of the active contour models (snakes)”, K. F. Lai and R. T. Chin, in Asian conf. Computer vision, Osaka, Japan, Nov. 1993, pp K. F. Lai and R. T. Chin, in Asian conf. Computer vision, Osaka, Japan, Nov. 1993, pp 542-545.542-545.

““Detection Theory – Application and Digital Signal Processing”, R. Hippenstiel, CRC Detection Theory – Application and Digital Signal Processing”, R. Hippenstiel, CRC Press, 2002Press, 2002

Zivkovic, Z.; Petkovic, M.; van Mierlo, R.; van Keulen, M.; van der Heijden, F.; Junker, Zivkovic, Z.; Petkovic, M.; van Mierlo, R.; van Keulen, M.; van der Heijden, F.; Junker, W.; Rijnierse, E.; “Two video analysis applications using foreground/background W.; Rijnierse, E.; “Two video analysis applications using foreground/background segmentation”; Visual Information Engineering, 2003. VIE 2003. International Conference segmentation”; Visual Information Engineering, 2003. VIE 2003. International Conference on , 2003 Pages:310 – 313on , 2003 Pages:310 – 313

ECE 8433 – Class Notes.ECE 8433 – Class Notes.

Question & AnswersQuestion & Answers


Documents

ECE 8433: Statistical Signal Processing Detection of Uncovered Background and Moving Pixels Detection of Uncovered Background & Moving Pixels Presented