Dynamic Variations in the Speed of a Digital Video …In the Rajan2 Cellular Logic Array Processing based skeletonization, steps 3(a), (b), (c) and (d) are executed a maximum number

International Journal of Applied Environmental Sciences

ISSN 0973-6077 Volume 12, Number 2 (2017), pp. 241-264

© Research India Publications

http://www.ripublication.com

Dynamic Variations in the Speed of a Digital Video

Stream due to Complexity of Algorithms and Entropy

of Video Frames

S. Aparna1 and M.Ekambaram Naidu2

1GITAM University, Hyderabad, India.

SRK Institute of Technology, Vijayawada, India.

Abstract

This paper discusses the dynamic variations of frame rate in a live digital

video due to complexity of video processing algorithms and entropy of frames.

A number of video processing algorithms are applied on a live video captured

by a web camera and frame rates are observed. It was observed that frame rate

reduces not only due to complexity of an algorithm applied on a streaming

video but also on the information content, that is, the entropy associated with

each running frame.

Keywords: Digital Video, Frame Rate, Complexity of Video Processing

Algorithms

1. INTRODUCTION

The three major parameters associated with video quality are (i) speed, (ii) size and

(iii) clarity. Speed of a digital video stream is quantified by ‘Frames per Second

(FPS)’. This means the number of frames displayed or communicated each second.

Usually, 15-30 FPS is considered as a standard in a digital video display or

communication. However, each user would have a different FPS, depending on the

computer, camera type, video size and the internet connection speed of say each

conference participant, especially in a video conference. The purpose of this paper is

to highlight the effect of frame entropy[2] and algorithmic complexity on FPS,

especially when digital video processing of main concern. Secondly, the size of the

frame plays a significant role in maintaining FPS during display or communication.

242 S.Aparna and M.Ekambaram Naidu

The term ‘size’ refers to the number of pixels displayed in a frame which in turn

quantifies its resolution defined by the frame width and frame height. Usually iSpQ

makes use of 320x240 resolution by default for capturing video.

The next important parameter that determines video quality is ‘clarity’. One would

come across many factors that determine how crisp and clear a video image will

appear. Most of the video conferencing facilities give top priority to image clarity

even at the cost of FPS, which means that the video quality remains within acceptable

limits, even if the FPS is reduced to an acceptable level. In a way one would have a

trade-off between FPS and clarity. In this context, video image processing becomes an

essential operation in order to maintain video quality. The problem faced, in such a

case, is an additional overhead to maintain speed and quality.

This paper addresses the problem of dynamic variations in the speed of streaming

video while using various image processing algorithms to process a live video[3]

stream frames.

2. EFFECTS OF ALGORITHMIC COMPLEXITIES ON FPS

Complexity of an algorithm is evaluated based on the number of computations

involved in processing a video frame. For example, let us consider an algorithm

meant for detecting edges in a video frame.

Algorithm: Rajan2-Cellular Logic Array Processing Based 2-Dimentional Edge

Detection

Input: 2-Dimensional image with 'T'

Output: 2-Dimensional image after Edge detection

Steps:

Step 1: Read the pixels from 2-Dimensional image and place pixel values in a 1-

Dimensional array called Input array.

Step 2: Copy input array to output array

Step 3: Repeat the steps 1 & 2 sliding the 5-neighborhood window over the image

(input array)

Start:

Step 3(a): Find MAX(0,1);

Step 3(b): Find MIN(0,1);

Step 3(c):Find difference D = MAX(0,1)-MIN(0,1);

Step 3(d): If(D <= T)

Then

Assign CP=0 in value output array;

Dynamic Variations in the Speed of a Digital Video Stream due to Complexity.. 243

Else

Find the slide the 5-neighborhood

End: Repeat this process until the structuring element scans the whole of the image

Step 4: Pass the output array to Display()

Complexity Calculation

In the Cellular Logic Array Processing[7] based edge detection algorithm ‘Rajan2’,

steps 3(a), (b), (c) and (d) are executed a maximum number of times. The time

complexity of this computation is evaluated as O(n2r2). The repetition of steps 3(a),,

(b), (c) and (d) takes place with the complexity of O(n2). The total time complexity of

2Dimentional edge detection by using Cellular Logic Array Processing -Rajan2

method is an image of size nxn by a structuring element of size rxr is O(n2r2)+O(n2).

Let us also consider an algorithm meant for skeletonizing a video frame.

Algorithm: Rajan 2-Cellular Logic Array Processing Based 2-Dimensional

Skeletonization

Input: 2-Dimensional image, threshold

Output: 2-Dimensional image after Edge detection

Steps:

Step 1: Read the pixel from 2-Dimensional image and place in a 1-Dimensional array

called input array.

Step 2: Move the elements in input array to output array

Step 3: Repeat the step 1 & 2 and slide the 5-neighborhood window over the image

(input array)

Start

Step 3(a): Find the MAX(0,1);

Step 3(b): Find the MIN(0,1);

Step 3(c):Find the difference D = MAX(0,1)-MIN(0,1);

Step 3(d): If(D <= T)

Then

Retain the CP as well as corner pixels and remove the boundary pixels in

output array

Else

slide the 5-neighborhood


End

Repeat this step until the structuring element spans the whole of the image .

Step 4: Copy output array to input array and repeat step 3 until there is no boundary

left for removal.

Step 5: Pass the output array to Display()

Complexity Calculation

In the Rajan2 Cellular Logic Array Processing based skeletonization, steps 3(a), (b),

(c) and (d) are executed a maximum number of times. The time complexity is

evaluated as O(n3r2). Because the outer while loop is executed until both input buffer

and output buffer are equal. In the worst case while loop executes for n times. That is,

the time complexity of 2-D skeletonization by using a Cellular Logic Array

Processing method on an image of size nxn by a structuring element of size rxr is

O(n3r2).

With these we can conclude that the overall time complexity of an algorithm not only

depends on the complexity of the algorithm but also on the number of instructions

used to implement it.

Here CP stands for Center Pixel,

T stands for Threshold

3. EFFECTS OF VIDEO FRAME ENTROPY ON FPS

Entropy of a digital image[8] is a statistical quantification of the information content

in the image. Larger the entropy more the information contained in an image. Figure

1(a) shows a minimum entropy[5] image and figure 1(b) maximum entropy[4] image.

The term ‘entropy’ refers to the degree of randomness of information. Real time video

communication employs an image quality parameter ‘Transmitted Information T I ’

which is briefly described here. Given events S1, ..., Sn occurring with probabilities

p(S1), ..., p(Sn), then the average uncertainty associated with each event is defined

based on Shannon entropy[1] as

Let x and y be the input and output random variables and their entropies H(x) and

H(y), respectively. Joint entropy, H(x, y), is defined as

, where Hx(y) and Hy(x) are conditional entropies.

Hx(y) is the entropy of the output when the input is known and Hy(x) that of the input

when the output is known. Now, the transmitted information TI could be computed as

T(x; y), where:


where pi=ni/n, pj=nj/n, and pij=nij/n. One can rewrite the above equations as given

below.

(a) Minimum entropy image; (b) Maximum entropy image

Figure 1: Sample images

Figure 2 shows a real time video frame captured by a web camera and its histogram

Figure 2: A sample video frame captured by a web camera (23.88 FPS)


Complete video image frame statistics is given below. Histogram of the image clearly

shows that the entropy of the image is small. One would visualize that the histogram

of an image with maximum entropy would be sparsely distributed.

Pixels Count 283504

Pixels without black 283504

Red Min 0

Red Max 255

Red Mean 144.121525622213

Red Standard Deviation 51.1821174004303

Red Median 158

Red Total Count 283504

Green Min 5

Green Max 255

Green Mean 144.105917376827

Green Standard Deviation 56.1654834870391

Green Median 157

Green Total Count 283504

Blue Min 3

Blue Max 255

Blue Mean 136.66722515379

Blue Standard Deviation 55.9918665568108

Blue Median 151

Blue Total Count 283504

Saturation Min 0

Saturation Max 1

Saturation Mean 0.12884584069252

Saturation Standard Deviation 0.125892773270607

Saturation Median 0.0862745121121407

Luminance Min 0.0431372560560703

Luminance Max 0.996078431606293

Luminance Mean 0.554617762565613

Luminance Standard Deviation 0.210107445716858

Luminance Median 0.607843160629272

Y Min 0.0313725508749485

Y Max 0.996078431606293

Y Mean 0.559841275215149

Y Standard Deviation 0.212207198143005

Y Median 0.61176472902298

Cb Min -0.358823537826538

Cb Max 0.0960784554481506


Cb Mean -0.0164319984614849

Cb Standard Deviation 0.0274433009326458

Cb Median -0.0137254893779755

Cr Min -0.131372541189194

Cr Max 0.11176472902298

Cr Mean 0.000478576781461015

Cr Standard Deviation 0.0297166928648949

Cr Median -0.00588235259056091

Red Min WB 0

Red Max WB 255

Red Mean WB 144.121525622213

Red Standard Deviation WB 51.1821174004303

Red Median WB 158

Red Total Count WB 283504

Green Min WB 5

Green Max WB 255

Green Mean WB 144.105917376827

Green Standard Deviation WB 56.1654834870391

Green Median WB 157

Green Total Count WB 283504

Blue Min WB 3

Blue Max WB 255

Blue Mean WB 136.66722515379

Blue Standard Deviation WB 55.9918665568108

Blue Median WB 151

Blue Total Count WB 283504

Saturation Min WB 0

Saturation Max WB 1

Saturation Mean WB 0.12884584069252

Saturation Standard Deviation WB 0.125892773270607

Saturation Median WB 0.0862745121121407

Luminance Min WB 0.0431372560560703

Luminance Max WB 0.996078431606293

Luminance Mean WB 0.554617762565613

Luminance Standard Deviation WB 0.210107445716858

Luminance Median WB 0.607843160629272

Y Min WB 0.0313725508749485

Y Max WB 0.996078431606293

Y Mean WB 0.559841275215149

Y Standard Deviation WB 0.212207198143005


Y Median WB 0.61176472902298

Cb Min WB -0.358823537826538

Cb Max WB 0.0960784554481506

Cb Mean WB -0.0164319984614849

Cb Standard Deviation WB 0.0274433009326458

Cb Median WB -0.0137254893779755

Cr Min WB -0.131372541189194

Cr Max WB 0.11176472902298

Cr Mean WB 0.000478576781461015

Cr Standard Deviation WB 0.0297166928648949

Cr Median WB -0.00588235259056091

Figure 3: A salt and pepper noise with its histogram

4. THE EFFECTS OF A VIDEO FRAME ENTROPY ON FPS

An empirical study was undertaken to verify the effects of applying various

algorithms on a streaming videos and results observed. Part of the results of the study

is presented in figures 4(I) to (XXXV) and table 1. One may observe that Seven out

of 69 Algorithms reduce the FPS.Among them few images are grey scale

morpholgy[9]

Rajan1 Filtered

(13.78 FPS)

Rajan2 Filtered

Th. 40 (7.31 FPS)

(I)


Sobel Filtered

(23.26FPS)

Laplacian Filtered

(22.92 FPS)

(II)

Prewit Filtered

(23.88 FPS)

Kirsch Filtered

(23.62 FPS)

(III)

Morphological Dilated

(15.50 FPS)

Morphological Eroded

(15.99 FPS)

(IV)


Segmentation Mean

(23.88 FPS)

Segmentation Median

(3.01FPS)

(V)

High Pass Filter Mask1

(23.88 FPS)


(23.88 FPS)

(VI)


(23.90 FPS)


(16.49 FPS)

(VII)


Low Pass Filter Mask1

(15.76 FPS)


(23.88 FPS)

(VIII)


(23.88 FPS)


(23.60 FPS)

(IX)


(24.22 FPS)

Faler Filtered Mask1

(23.62 FPS)

(X)



(24.25 FPS)


(24.25 FPS)

(XI)


(23.88 FPS)


(23.88 FPS)

(XII)

Kirsch Filtered Mask1

(24.63 FPS)


(23.54 FPS)

(XIII)



(23.88 FPS)


(23.88 FPS)

(XIV)


(23.88 FPS)


(23.88 FPS)

(XV)


(23.90 FPS)


(24.25 FPS)

(XVI)


Prewitt Filtered Mask1

(23.62 FPS)


(22.92 FPS)

(XVII)


(22.64 FPS)


(22.92 FPS)

(XVIII)


(22.92 FPS)


(22.92 FPS)

(XIX)



(22.92 FPS)


(22.92 FPS)

(XX)


(22.92 FPS)

Sobel Filtered Mask1

(23.62 FPS)

(XXI)


(20.37 FPS)


(22.94 FPS)

(XXII)



(23.28 FPS)


(23.62 FPS)

(XXIII)


(23.28 FPS)


(23.92 FPS)

(XXIV)


(21.99 FPS)

Laplatian Mask1

(22.66 FPS)

(XXV)


Laplatian Mask2

(22.29 FPS)

Laplatian Mask3

(22.64 FPS)

(XXVI)

Laplatian Mask4

(24.25 FPS)

Laplatian Mask5

(24.25 FPS)

(XXVII)

Robinson Mask1

(23.28 FPS)

Robinson Mask2

(23.28 FPS)

(XXVIII)


Robinson Mask3

(23.28 FPS)

Edge Enhancement East

(24.25 FPS)

(XXIX)

Edge Enhancement west

(23.88 FPS)

Edge Enhancement north

(23.62 FPS)

(XXX)

Edge Enhancement South

(24.25 FPS)

Edge Enhancement north-east

(24.25 FPS)

(XXXI)


Edge Enhancement north-west

(23.88 FPS)

Edge Enhancement southeast

(24.25 FPS)

(XXXII)

Edge Enhancement southwest

(23.88 FPS)

Line Enhancement East -west

(23.62FPS)

(XXXIII)

Line Enhancement North-south

(22.64 FPS)

LineEnhancement northeast-southwest

(23.88 FPS)

(XXXIV)


Line Enhancement northwest-southeast

(23.62 FPS)

(XXXV)

Figure 4: Processed video image frames

Table 1: Algorithms and FPS

Sl. No. Algorithms FPS

1 rajan1 13.78

2 rajan2 7.31

3 Sobel 23.26

4 Laplacian 22.46

5 Prewit 23.88

6 Kirsch 23.62

7 morphological dilation 15.50

8 morphological erosion 15.99

9 segmentation mean 23.88

10 segmentation median 3.01

11 high pass filter mask1 23.88




15 low pass filter mask1 15.76






20 fahler filter mask1 23.62





25 kirsch mask1 24.63








33 prewitt mask1 23.62









42 sobel mask1 23.62









50 laplacian mask1 22.66





55 robinson mask1 23.28



58 Edge enhancement East 24.25

59 Edge enhancement West 23.88

60 Edge enhancement north 23.62

61 Edge enhancement south 24.25

62 Edge enhancement north east 24.25

63 Edge enhancement north west 23.88

64 Edge enhancement south east 24.25

65 Edge enhancement south west 23.88

66 line enhancement east -west 23.62

67 line enhancement north-south 22.64

68 line enhancement northeast -

southwest 23.88

69 line enhancement northwest -

southeast 23.62


Figure 5: Graph showing the results of applying 69 algorithms on a video stream and

the dynamic variations in FPS

5. CONCLUSIONS

Sixty-nine algorithms were applied on various digital video streams and their effects

on FPS were observed. It was observed that seven of these 69 algorithms reduce the

FPS of a video stream. They are listed below along with the FPS

Rajan1 13.78

Rajan2 7.31

Morphological dilation 15.50

Morphological erosion 15.99

Segmentation median 3.01

High pass filter mask4 16.49

Low pass filter mask1 15.76

The reason behind the reduction of FPS is twofold (i) algorithmic complexity and (ii)

entropy of the video frames. It was also observed that the reduction in FPS is dynamic

and it oscillates between tolerable limits.

ACKNOWLEDGEMENTS

The authors gratefully acknowledge the untiring support given to them by the research

team of Pentagram Research Centre Private Limited, Hyderabad, Telangana State,

India while carrying out research.

REFERENCES

[1] Shannon, Claude E(July-October 1948)."A Mathematical theory of

communication" Technical Journal .379-39 doi:10.1002/j.1538-

7305.1948.tb01338.x.


[2] C.Studholme, "An Overlap invariant entropy measure of 3d medical image

alignment." Pattern Reconition, volume 32,issue1,January 1999,pages 71-

86.http://dxdoi.org/10.1016/s0031-3203(98)00091-0

[3] John C.Crocker. "Methods of Digital Video Microscopy for Colloidal studies."

Journal of colloid and interface Science. Volume 179, issue 1,15 April

1996.pages 298-310.Elsevier.

[4] S.F.Burch, "Image restoration by a powerful maximum entropy method".

Computer vision, Graphics and image Processing .Volume 23,issue 2 ,August

1983,pages 113-128. http://dx.doi.org/10.1016/0734-189X(83)90108-1

Elsevier

[5] C.V.Angelino, E.Debreuve, M.Barlaud "A nonparametric minimum entropy

image deblurring algorithm"2008 IEEE International conference on

accoustics,Speech and signal processing Pages 925-

928,DOI:10.1109/ICASSP.2008.4517762

[6] Vaddi chandra sekhar, satyajit bora, monalisa das, Pavan kumar manchi,

S.JOsephine,Roy Paily "Design and Implementation of Blind Assistance

system using real time stereo vision algorithms. Pages 421-426,

DOI:10.1109/VLSID. 2016.11

[7] Rajan, E.G., “Cellular Logic Array Processing, Invited paper, World Congress

for Nonlinear Analysts”, organized by the international Federation of

Nonlinear Analysts, Florida Institute of Technology, July 10-17, 1996 ,

Athens, Greece

[8] R.C. Gonzalez, R.E. Woods, Digital Image Processing. Addison Wesley, New

York, 1992

[9] Stanley R. Sternberg. Gray scale morphology "Computer vision, graphics and

image processing 333-335(1986)

[10] Rajan, E.G., Medical Imaging in the framework of cellular Logic Array

Processing, 15th International Conference of Biomedical Society of

India,INCONBME’96, Coimbatore Institute of Technology, December 12-14,

1996.

ABOUT THE AUTHORS

S. Aparna is Assistant Professor from the Department of Computer

Science and Engineering of the GITAM University, Hyderabad. My

profound interests in research pulled me from industry and took to

academics.My Research interests include Video Image Processing,

Software Engineering, Data Base Management Systems.

Dr. M. Ekambaram Naidu is the Principal of SRK College of

Engineering and technology, Vijayawada.He is an avid Researcher and

renowned professor . His research interests include image processing,

Pattern Recognition and Analysis, Computer Networking, Software

Engineering.

http://dx.doi.org/10.1016/0734-189X(83)90108-1

http://www.sciencedirect.com/science/article/pii/0734189X86900046

Documents

Dynamic Variations in the Speed of a Digital Video …In the Rajan2 Cellular Logic Array Processing based skeletonization, steps 3(a), (b), (c) and (d) are executed a maximum number