International Journal of Applied Environmental Sciences
ISSN 0973-6077 Volume 12, Number 2 (2017), pp. 241-264
© Research India Publications
http://www.ripublication.com
Dynamic Variations in the Speed of a Digital Video
Stream due to Complexity of Algorithms and Entropy
of Video Frames
S. Aparna1 and M.Ekambaram Naidu2
1GITAM University, Hyderabad, India.
SRK Institute of Technology, Vijayawada, India.
Abstract
This paper discusses the dynamic variations of frame rate in a live digital
video due to complexity of video processing algorithms and entropy of frames.
A number of video processing algorithms are applied on a live video captured
by a web camera and frame rates are observed. It was observed that frame rate
reduces not only due to complexity of an algorithm applied on a streaming
video but also on the information content, that is, the entropy associated with
each running frame.
Keywords: Digital Video, Frame Rate, Complexity of Video Processing
Algorithms
1. INTRODUCTION
The three major parameters associated with video quality are (i) speed, (ii) size and
(iii) clarity. Speed of a digital video stream is quantified by ‘Frames per Second
(FPS)’. This means the number of frames displayed or communicated each second.
Usually, 15-30 FPS is considered as a standard in a digital video display or
communication. However, each user would have a different FPS, depending on the
computer, camera type, video size and the internet connection speed of say each
conference participant, especially in a video conference. The purpose of this paper is
to highlight the effect of frame entropy[2] and algorithmic complexity on FPS,
especially when digital video processing of main concern. Secondly, the size of the
frame plays a significant role in maintaining FPS during display or communication.
242 S.Aparna and M.Ekambaram Naidu
The term ‘size’ refers to the number of pixels displayed in a frame which in turn
quantifies its resolution defined by the frame width and frame height. Usually iSpQ
makes use of 320x240 resolution by default for capturing video.
The next important parameter that determines video quality is ‘clarity’. One would
come across many factors that determine how crisp and clear a video image will
appear. Most of the video conferencing facilities give top priority to image clarity
even at the cost of FPS, which means that the video quality remains within acceptable
limits, even if the FPS is reduced to an acceptable level. In a way one would have a
trade-off between FPS and clarity. In this context, video image processing becomes an
essential operation in order to maintain video quality. The problem faced, in such a
case, is an additional overhead to maintain speed and quality.
This paper addresses the problem of dynamic variations in the speed of streaming
video while using various image processing algorithms to process a live video[3]
stream frames.
2. EFFECTS OF ALGORITHMIC COMPLEXITIES ON FPS
Complexity of an algorithm is evaluated based on the number of computations
involved in processing a video frame. For example, let us consider an algorithm
meant for detecting edges in a video frame.
Algorithm: Rajan2-Cellular Logic Array Processing Based 2-Dimentional Edge
Detection
Input: 2-Dimensional image with 'T'
Output: 2-Dimensional image after Edge detection
Steps:
Step 1: Read the pixels from 2-Dimensional image and place pixel values in a 1-
Dimensional array called Input array.
Step 2: Copy input array to output array
Step 3: Repeat the steps 1 & 2 sliding the 5-neighborhood window over the image
(input array)
Start:
Step 3(a): Find MAX(0,1);
Step 3(b): Find MIN(0,1);
Step 3(c):Find difference D = MAX(0,1)-MIN(0,1);
Step 3(d): If(D <= T)
Then
Assign CP=0 in value output array;
Dynamic Variations in the Speed of a Digital Video Stream due to Complexity.. 243
Else
Find the slide the 5-neighborhood
End: Repeat this process until the structuring element scans the whole of the image
Step 4: Pass the output array to Display()
Complexity Calculation
In the Cellular Logic Array Processing[7] based edge detection algorithm ‘Rajan2’,
steps 3(a), (b), (c) and (d) are executed a maximum number of times. The time
complexity of this computation is evaluated as O(n2r2). The repetition of steps 3(a),,
(b), (c) and (d) takes place with the complexity of O(n2). The total time complexity of
2Dimentional edge detection by using Cellular Logic Array Processing -Rajan2
method is an image of size nxn by a structuring element of size rxr is O(n2r2)+O(n2).
Let us also consider an algorithm meant for skeletonizing a video frame.
Algorithm: Rajan 2-Cellular Logic Array Processing Based 2-Dimensional
Skeletonization
Input: 2-Dimensional image, threshold
Output: 2-Dimensional image after Edge detection
Steps:
Step 1: Read the pixel from 2-Dimensional image and place in a 1-Dimensional array
called input array.
Step 2: Move the elements in input array to output array
Step 3: Repeat the step 1 & 2 and slide the 5-neighborhood window over the image
(input array)
Start
Step 3(a): Find the MAX(0,1);
Step 3(b): Find the MIN(0,1);
Step 3(c):Find the difference D = MAX(0,1)-MIN(0,1);
Step 3(d): If(D <= T)
Then
Retain the CP as well as corner pixels and remove the boundary pixels in
output array
Else
slide the 5-neighborhood
244 S.Aparna and M.Ekambaram Naidu
End
Repeat this step until the structuring element spans the whole of the image .
Step 4: Copy output array to input array and repeat step 3 until there is no boundary
left for removal.
Step 5: Pass the output array to Display()
Complexity Calculation
In the Rajan2 Cellular Logic Array Processing based skeletonization, steps 3(a), (b),
(c) and (d) are executed a maximum number of times. The time complexity is
evaluated as O(n3r2). Because the outer while loop is executed until both input buffer
and output buffer are equal. In the worst case while loop executes for n times. That is,
the time complexity of 2-D skeletonization by using a Cellular Logic Array
Processing method on an image of size nxn by a structuring element of size rxr is
O(n3r2).
With these we can conclude that the overall time complexity of an algorithm not only
depends on the complexity of the algorithm but also on the number of instructions
used to implement it.
Here CP stands for Center Pixel,
T stands for Threshold
3. EFFECTS OF VIDEO FRAME ENTROPY ON FPS
Entropy of a digital image[8] is a statistical quantification of the information content
in the image. Larger the entropy more the information contained in an image. Figure
1(a) shows a minimum entropy[5] image and figure 1(b) maximum entropy[4] image.
The term ‘entropy’ refers to the degree of randomness of information. Real time video
communication employs an image quality parameter ‘Transmitted Information T I ’
which is briefly described here. Given events S1, ..., Sn occurring with probabilities
p(S1), ..., p(Sn), then the average uncertainty associated with each event is defined
based on Shannon entropy[1] as
Let x and y be the input and output random variables and their entropies H(x) and
H(y), respectively. Joint entropy, H(x, y), is defined as
, where Hx(y) and Hy(x) are conditional entropies.
Hx(y) is the entropy of the output when the input is known and Hy(x) that of the input
when the output is known. Now, the transmitted information TI could be computed as
T(x; y), where:
Dynamic Variations in the Speed of a Digital Video Stream due to Complexity.. 245
where pi=ni/n, pj=nj/n, and pij=nij/n. One can rewrite the above equations as given
below.
(a) Minimum entropy image; (b) Maximum entropy image
Figure 1: Sample images
Figure 2 shows a real time video frame captured by a web camera and its histogram
Figure 2: A sample video frame captured by a web camera (23.88 FPS)
246 S.Aparna and M.Ekambaram Naidu
Complete video image frame statistics is given below. Histogram of the image clearly
shows that the entropy of the image is small. One would visualize that the histogram
of an image with maximum entropy would be sparsely distributed.
Pixels Count 283504
Pixels without black 283504
Red Min 0
Red Max 255
Red Mean 144.121525622213
Red Standard Deviation 51.1821174004303
Red Median 158
Red Total Count 283504
Green Min 5
Green Max 255
Green Mean 144.105917376827
Green Standard Deviation 56.1654834870391
Green Median 157
Green Total Count 283504
Blue Min 3
Blue Max 255
Blue Mean 136.66722515379
Blue Standard Deviation 55.9918665568108
Blue Median 151
Blue Total Count 283504
Saturation Min 0
Saturation Max 1
Saturation Mean 0.12884584069252
Saturation Standard Deviation 0.125892773270607
Saturation Median 0.0862745121121407
Luminance Min 0.0431372560560703
Luminance Max 0.996078431606293
Luminance Mean 0.554617762565613
Luminance Standard Deviation 0.210107445716858
Luminance Median 0.607843160629272
Y Min 0.0313725508749485
Y Max 0.996078431606293
Y Mean 0.559841275215149
Y Standard Deviation 0.212207198143005
Y Median 0.61176472902298
Cb Min -0.358823537826538
Cb Max 0.0960784554481506
Dynamic Variations in the Speed of a Digital Video Stream due to Complexity.. 247
Cb Mean -0.0164319984614849
Cb Standard Deviation 0.0274433009326458
Cb Median -0.0137254893779755
Cr Min -0.131372541189194
Cr Max 0.11176472902298
Cr Mean 0.000478576781461015
Cr Standard Deviation 0.0297166928648949
Cr Median -0.00588235259056091
Red Min WB 0
Red Max WB 255
Red Mean WB 144.121525622213
Red Standard Deviation WB 51.1821174004303
Red Median WB 158
Red Total Count WB 283504
Green Min WB 5
Green Max WB 255
Green Mean WB 144.105917376827
Green Standard Deviation WB 56.1654834870391
Green Median WB 157
Green Total Count WB 283504
Blue Min WB 3
Blue Max WB 255
Blue Mean WB 136.66722515379
Blue Standard Deviation WB 55.9918665568108
Blue Median WB 151
Blue Total Count WB 283504
Saturation Min WB 0
Saturation Max WB 1
Saturation Mean WB 0.12884584069252
Saturation Standard Deviation WB 0.125892773270607
Saturation Median WB 0.0862745121121407
Luminance Min WB 0.0431372560560703
Luminance Max WB 0.996078431606293
Luminance Mean WB 0.554617762565613
Luminance Standard Deviation WB 0.210107445716858
Luminance Median WB 0.607843160629272
Y Min WB 0.0313725508749485
Y Max WB 0.996078431606293
Y Mean WB 0.559841275215149
Y Standard Deviation WB 0.212207198143005
248 S.Aparna and M.Ekambaram Naidu
Y Median WB 0.61176472902298
Cb Min WB -0.358823537826538
Cb Max WB 0.0960784554481506
Cb Mean WB -0.0164319984614849
Cb Standard Deviation WB 0.0274433009326458
Cb Median WB -0.0137254893779755
Cr Min WB -0.131372541189194
Cr Max WB 0.11176472902298
Cr Mean WB 0.000478576781461015
Cr Standard Deviation WB 0.0297166928648949
Cr Median WB -0.00588235259056091
Figure 3: A salt and pepper noise with its histogram
4. THE EFFECTS OF A VIDEO FRAME ENTROPY ON FPS
An empirical study was undertaken to verify the effects of applying various
algorithms on a streaming videos and results observed. Part of the results of the study
is presented in figures 4(I) to (XXXV) and table 1. One may observe that Seven out
of 69 Algorithms reduce the FPS.Among them few images are grey scale
morpholgy[9]
Rajan1 Filtered
(13.78 FPS)
Rajan2 Filtered
Th. 40 (7.31 FPS)
(I)
Dynamic Variations in the Speed of a Digital Video Stream due to Complexity.. 249
Sobel Filtered
(23.26FPS)
Laplacian Filtered
(22.92 FPS)
(II)
Prewit Filtered
(23.88 FPS)
Kirsch Filtered
(23.62 FPS)
(III)
Morphological Dilated
(15.50 FPS)
Morphological Eroded
(15.99 FPS)
(IV)
250 S.Aparna and M.Ekambaram Naidu
Segmentation Mean
(23.88 FPS)
Segmentation Median
(3.01FPS)
(V)
High Pass Filter Mask1
(23.88 FPS)
High Pass Filter Mask2
(23.88 FPS)
(VI)
High Pass Filter Mask3
(23.90 FPS)
High Pass Filter Mask4
(16.49 FPS)
(VII)
Dynamic Variations in the Speed of a Digital Video Stream due to Complexity.. 251
Low Pass Filter Mask1
(15.76 FPS)
Low Pass Filter Mask2
(23.88 FPS)
(VIII)
Low Pass Filter Mask3
(23.88 FPS)
Low Pass Filter Mask4
(23.60 FPS)
(IX)
Low Pass Filter Mask5
(24.22 FPS)
Faler Filtered Mask1
(23.62 FPS)
(X)
252 S.Aparna and M.Ekambaram Naidu
Faler Filtered Mask2
(24.25 FPS)
Faler Filtered Mask3
(24.25 FPS)
(XI)
Faler Filtered Mask4
(23.88 FPS)
Faler Filtered Mask5
(23.88 FPS)
(XII)
Kirsch Filtered Mask1
(24.63 FPS)
Kirsch Filtered Mask2
(23.54 FPS)
(XIII)
Dynamic Variations in the Speed of a Digital Video Stream due to Complexity.. 253
Kirsch Filtered Mask3
(23.88 FPS)
Kirsch Filtered Mask4
(23.88 FPS)
(XIV)
Kirsch Filtered Mask5
(23.88 FPS)
Kirsch Filtered Mask6
(23.88 FPS)
(XV)
Kirsch Filtered Mask7
(23.90 FPS)
Kirsch Filtered Mask8
(24.25 FPS)
(XVI)
254 S.Aparna and M.Ekambaram Naidu
Prewitt Filtered Mask1
(23.62 FPS)
Prewitt Filtered Mask2
(22.92 FPS)
(XVII)
Prewitt Filtered Mask3
(22.64 FPS)
Prewitt Filtered Mask4
(22.92 FPS)
(XVIII)
Prewitt Filtered Mask5
(22.92 FPS)
Prewitt Filtered Mask6
(22.92 FPS)
(XIX)
Dynamic Variations in the Speed of a Digital Video Stream due to Complexity.. 255
Prewitt Filtered Mask7
(22.92 FPS)
Prewitt Filtered Mask8
(22.92 FPS)
(XX)
Prewitt Filtered Mask9
(22.92 FPS)
Sobel Filtered Mask1
(23.62 FPS)
(XXI)
Sobel Filtered Mask2
(20.37 FPS)
Sobel Filtered Mask3
(22.94 FPS)
(XXII)
256 S.Aparna and M.Ekambaram Naidu
Sobel Filtered Mask4
(23.28 FPS)
Sobel Filtered Mask5
(23.62 FPS)
(XXIII)
Sobel Filtered Mask6
(23.28 FPS)
Sobel Filtered Mask7
(23.92 FPS)
(XXIV)
Sobel Filtered Mask8
(21.99 FPS)
Laplatian Mask1
(22.66 FPS)
(XXV)
Dynamic Variations in the Speed of a Digital Video Stream due to Complexity.. 257
Laplatian Mask2
(22.29 FPS)
Laplatian Mask3
(22.64 FPS)
(XXVI)
Laplatian Mask4
(24.25 FPS)
Laplatian Mask5
(24.25 FPS)
(XXVII)
Robinson Mask1
(23.28 FPS)
Robinson Mask2
(23.28 FPS)
(XXVIII)
258 S.Aparna and M.Ekambaram Naidu
Robinson Mask3
(23.28 FPS)
Edge Enhancement East
(24.25 FPS)
(XXIX)
Edge Enhancement west
(23.88 FPS)
Edge Enhancement north
(23.62 FPS)
(XXX)
Edge Enhancement South
(24.25 FPS)
Edge Enhancement north-east
(24.25 FPS)
(XXXI)
Dynamic Variations in the Speed of a Digital Video Stream due to Complexity.. 259
Edge Enhancement north-west
(23.88 FPS)
Edge Enhancement south- east
(24.25 FPS)
(XXXII)
Edge Enhancement south- west
(23.88 FPS)
Line Enhancement East -west
(23.62FPS)
(XXXIII)
Line Enhancement North-south
(22.64 FPS)
LineEnhancement northeast-southwest
(23.88 FPS)
(XXXIV)
260 S.Aparna and M.Ekambaram Naidu
Line Enhancement northwest-southeast
(23.62 FPS)
(XXXV)
Figure 4: Processed video image frames
Table 1: Algorithms and FPS
Sl. No. Algorithms FPS
1 rajan1 13.78
2 rajan2 7.31
3 Sobel 23.26
4 Laplacian 22.46
5 Prewit 23.88
6 Kirsch 23.62
7 morphological dilation 15.50
8 morphological erosion 15.99
9 segmentation mean 23.88
10 segmentation median 3.01
11 high pass filter mask1 23.88
12 high pass filter mask2 23.88
13 high pass filter mask3 23.88
14 high pass filter mask4 16.49
15 low pass filter mask1 15.76
Dynamic Variations in the Speed of a Digital Video Stream due to Complexity.. 261
16 low pass filter mask2 23.88
17 low pass filter mask3 23.88
18 low pass filter mask4 24.63
19 low pass filter mask5 24.22
20 fahler filter mask1 23.62
21 fahler filter mask2 24.25
22 fahler filter mask3 24.25
23 fahler filter mask4 23.88
24 fahler filter mask5 23.88
25 kirsch mask1 24.63
26 kirsch mask2 23.54
27 kirsch mask3 23.88
28 kirsch mask4 23.88
29 kirsch mask5 23.88
30 kirsch mask6 23.88
31 kirsch mask7 23.90
32 kirsch mask8 24.25
33 prewitt mask1 23.62
34 prewitt mask2 22.92
35 prewitt mask3 22.64
36 prewitt mask4 22.92
37 prewitt mask5 22.92
38 prewitt mask6 22.92
39 prewitt mask7 22.64
40 prewitt mask8 22.92
41 prewitt mask9 22.64
42 sobel mask1 23.62
43 sobel mask2 20.37
262 S.Aparna and M.Ekambaram Naidu
44 sobel mask3 22.94
45 sobel mask4 23.28
46 sobel mask5 23.62
47 sobel mask6 23.28
48 sobel mask7 22.92
49 sobel mask8 21.99
50 laplacian mask1 22.66
51 laplacian mask2 22.29
52 laplacian mask3 22.66
53 laplacian mask4 24.25
54 laplacian mask5 24.25
55 robinson mask1 23.28
56 robinson mask2 23.28
57 robinson mask3 23.28
58 Edge enhancement East 24.25
59 Edge enhancement West 23.88
60 Edge enhancement north 23.62
61 Edge enhancement south 24.25
62 Edge enhancement north east 24.25
63 Edge enhancement north west 23.88
64 Edge enhancement south east 24.25
65 Edge enhancement south west 23.88
66 line enhancement east -west 23.62
67 line enhancement north-south 22.64
68 line enhancement northeast -
southwest 23.88
69 line enhancement northwest -
southeast 23.62
Dynamic Variations in the Speed of a Digital Video Stream due to Complexity.. 263
Figure 5: Graph showing the results of applying 69 algorithms on a video stream and
the dynamic variations in FPS
5. CONCLUSIONS
Sixty-nine algorithms were applied on various digital video streams and their effects
on FPS were observed. It was observed that seven of these 69 algorithms reduce the
FPS of a video stream. They are listed below along with the FPS
Rajan1 13.78
Rajan2 7.31
Morphological dilation 15.50
Morphological erosion 15.99
Segmentation median 3.01
High pass filter mask4 16.49
Low pass filter mask1 15.76
The reason behind the reduction of FPS is twofold (i) algorithmic complexity and (ii)
entropy of the video frames. It was also observed that the reduction in FPS is dynamic
and it oscillates between tolerable limits.
ACKNOWLEDGEMENTS
The authors gratefully acknowledge the untiring support given to them by the research
team of Pentagram Research Centre Private Limited, Hyderabad, Telangana State,
India while carrying out research.
REFERENCES
[1] Shannon, Claude E(July-October 1948)."A Mathematical theory of
communication" Technical Journal .379-39 doi:10.1002/j.1538-
7305.1948.tb01338.x.
264 S.Aparna and M.Ekambaram Naidu
[2] C.Studholme, "An Overlap invariant entropy measure of 3d medical image
alignment." Pattern Reconition, volume 32,issue1,January 1999,pages 71-
86.http://dxdoi.org/10.1016/s0031-3203(98)00091-0
[3] John C.Crocker. "Methods of Digital Video Microscopy for Colloidal studies."
Journal of colloid and interface Science. Volume 179, issue 1,15 April
1996.pages 298-310.Elsevier.
[4] S.F.Burch, "Image restoration by a powerful maximum entropy method".
Computer vision, Graphics and image Processing .Volume 23,issue 2 ,August
1983,pages 113-128. http://dx.doi.org/10.1016/0734-189X(83)90108-1
Elsevier
[5] C.V.Angelino, E.Debreuve, M.Barlaud "A nonparametric minimum entropy
image deblurring algorithm"2008 IEEE International conference on
accoustics,Speech and signal processing Pages 925-
928,DOI:10.1109/ICASSP.2008.4517762
[6] Vaddi chandra sekhar, satyajit bora, monalisa das, Pavan kumar manchi,
S.JOsephine,Roy Paily "Design and Implementation of Blind Assistance
system using real time stereo vision algorithms. Pages 421-426,
DOI:10.1109/VLSID. 2016.11
[7] Rajan, E.G., “Cellular Logic Array Processing, Invited paper, World Congress
for Nonlinear Analysts”, organized by the international Federation of
Nonlinear Analysts, Florida Institute of Technology, July 10-17, 1996 ,
Athens, Greece
[8] R.C. Gonzalez, R.E. Woods, Digital Image Processing. Addison Wesley, New
York, 1992
[9] Stanley R. Sternberg. Gray scale morphology "Computer vision, graphics and
image processing 333-335(1986)
[10] Rajan, E.G., Medical Imaging in the framework of cellular Logic Array
Processing, 15th International Conference of Biomedical Society of
India,INCONBME’96, Coimbatore Institute of Technology, December 12-14,
1996.
ABOUT THE AUTHORS
S. Aparna is Assistant Professor from the Department of Computer
Science and Engineering of the GITAM University, Hyderabad. My
profound interests in research pulled me from industry and took to
academics.My Research interests include Video Image Processing,
Software Engineering, Data Base Management Systems.
Dr. M. Ekambaram Naidu is the Principal of SRK College of
Engineering and technology, Vijayawada.He is an avid Researcher and
renowned professor . His research interests include image processing,
Pattern Recognition and Analysis, Computer Networking, Software
Engineering.