Procedia Environmental Sciences 10 ( 2011 ) 743 – 747

doi: 10.1016/j.proenv.2011.09.120

Available online at www.sciencedirect.com

ESIAT 2011

Research on Dynamic-Average-Gray-Scale-Based Video Segmentation Algorithm

Zhangping Hu School of Computer, Chongqing University of Arts and Science, YongChuan, Chongqing, 402160, China

Abstract

The paper explores the problem of traditional video segmentation algorithm and studies a novel dynamic video segmentation algorithm based on average-gray-scale, the method utilizes the compensation-gray obtained automatically to compensate the error produced by the traditional algorithm for the movement of the camera, the degree of compensation can be computed by a dynamic method. By this algorithm, a video can be segmented into scenes segments, and the scenes can be segmented into shot segments effectually, and different application environment has different segmentation threshold.

Keywords: Video segmentation; Threshold; Compensation; Average-Gray-Scale

1. Instruction

The domain of visual human motion analysis is interesting and challenging enough to be the subject of intense study and research by several computer vision laboratories and researchers in the world. Video segmentation technology, in particularly, is interesting because it is the first step of the action of human motion recognition in video and it can contribute not only to theoretical insights but also to practical applications. In order to process video, we must to segment the video firstly. The research activity in the video segmentation domain has resulted in survey papers being published in the last decade. Rather than summarizing all of the applications, problems, challenges, solution strategies, and techniques in the domain, we refer the interested read to the latest survey papers, e.g.[2,8,9,10,11]. Below, we describe and review the relevant study work in this domain in the latest years, and present the problems in these methods.

An algorithm of multiple objects video segmentation recognition information is proposed in [8], this algorithm is used to segment the objects from a video frame, and it is not used to segment the frames from

Corresponding author. Tel: +86-023-49891802 ; fax: +86-023-49891802 E-mail address:pinghu51@yahoo.com.cn

a video. Based on the color feature analysis, Niu[2] considers the shot segmentation issue from the perspective of semantic analysis, i.e., the semantic shot segmentation, the precision of shot classification shows that the color ratio feature is useful to improve the performance, and considering the temporal variations in the semantic color due to environment, they gives an adaptive semantic color extraction algorithm, and we may evaluate the influence of classification precision on the number of semantic colors, but this method can be applied in special domain. By introducing the hierarchical structure, Hou[10] effectively solve the flickering problem caused by the incorrect matching between frames, first, they do hierarchical segmentation on each frame and label a key frame, and then build relationship between

2011 3rd International Conference on Environmental Science and Information Application Technology (ESIAT 2011)

© 2011 Published by Elsevier Ltd. Selection and/or peer-review under responsibility of Conference ESIAT2011 Organization Committee.

Open access under CC BY-NC-ND license.

1878-0296 © 2011 Published by Elsevier Ltd. Selection and/or peer-review under responsibility of Conference ESIAT2011 Organization Committee.

Open access under CC BY-NC-ND license.

744 Zhangping Hu / Procedia Environmental Sciences 10 ( 2011 ) 743 – 747

image, here the gray-scale is the value of the brightness in the YIQ color space. Following is the expression of the conversion from RGB color space to YIQ color space [12]:

)5(311.0321.0

114.0

523.0275.0

587.0

212.0596.0299.0

BGR

QIY

We use ),( yxf gray to express the gray value of the position ),( yx . According to the above expression, the expression of the Gray-Scale conversion is:

)6(5.0144.0G 587.0299.0),( BRyxf gray Here R, G and B are the Red, the Green and the Blue values of the color image pixel. The following

image in Figure 1 shows an example of the gray-scale image obtained by the above expression, the left image is the color image described by RGB model, and the right image is the Gray-Scale image, which was obtained by upper processing.

Figure 1 Gray-scale Conversion

The difference of the gray-scale between successive frames is:

frames, and a novel hierarchical matching algorithm is proposed to refine the segmentation result by making use of the hierarchical structure, though it is efficiency and effectiveness, this algorithm must to segment object from frames before matching, and we all know that the object segmentation algorithm is quit difficult, so this algorithm is complexity. In[11], the video stream was segmented into segments by HSI (Hue, Saturation, Intensity) color information between neighboring frames, and the high-dimensional index structure Vector-Approximation Trie (VA-Trie) was adopted to organize the segments, the similarity model was defined using spatial and texture features, which can fully take into account the temporal order among video segments.

As we all know, according to the principle of the structure of a video, it can be separated into a series of scenes, and a scene can be separated into several shots, which are made up of a series of frames. In this paper, we study some works concerning video shot segmentation and scene segmentation, and a new algorithm of video segmentation based on dynamic average gray scale is proposed in the following, with this algorithm, the video stream was segmented into segments by gray scale information between neighboring frames. 2. Analysis of the Video Segmentation Algorithm 2.1 Expression of the Video Segmentation

This section defines the video segmentation models used in this paper, and describes how they can be segmented efficiently by our proposed algorithm. We assume that individual frames in a video are represented by three-dimensional feature vectors from a metric space ),,( tyxf [9], the nth frame is described by the expression:

)1(),( yxfn And the n+1th frame is described by the expression:

)2(),(1 yxfn The ith scene segmentation of a video is described by expression:

)3(),,( iyxf scene The jth shot segmentation of a scene is described by expression:

)4(),,( jyxf shot

2.2 Gray-Scale Conversion Any image from a video or digital camera is a color image. A digital color image pixel is just a RGB

data value (Red, Green and Blue), There are three numerical RGB components (Red, Green and Blue) to present the color of each pixel’s color sample. These three RGB components are three 8-bit numbers for each pixel, three 8-bit bytes is called 24 bit color. But each pixel is only stored as one 8-byte in gray-scale

745 Zhangping Hu / Procedia Environmental Sciences 10 ( 2011 ) 743 – 747

)7(),(),(),( __1_ yxfyxfyxf grayngrayngraydifference

Here ),(_ yxf graydifference is the difference of gray-scale between two frames, ),(_1 yxf grayn is the

gray-scale of the n+1th frame, and ),(_ yxf grayn is the gray-scale of the nth frame.

We assume that the image-height is h , the image-width is w , the average image-gray-scale of the image of the nth frame is:

)8(),(][1

0

1

0

h

i

w

jjifnavg

2.3 Gray-Scale Compensation

From the above section, we can know that the difference of the gray-scale between successive frames is the distance between the successive frames, with this distance the video stream can be segmented into segments. The shot or object of the video is moving constantly, and different speed has different influence distance degree. Then, the influence from shot movement and object movement can be compensated by the average value of the change of the images color between the successive frames. The compensation of the gray-scale of the nth frame can be described by:

)9(3/)),(),(),((),( ____ yxfyxfyxfyxf BdifferenceGdifferenceRdifferencegrayioncompenstat

Here ),(),,(),,( ___ yxfyxfyxf BdifferenceGdifferenceRdifference are the Red, Green and Blue values of a color frame image. And they can be described by:

),(),(),(

)10(),(),(),(

),(),(),(

__1_

__1_

__1_

yxfyxfyxfyxfyxfyxfyxfyxfyxf

BnBnBdifference

GnGnGdifference

RnRnRdifference

Here ),(),,(),,( _1_1_1 yxfyxfyxf BnGnRn are the Red, Green and Blue values of the n+1th

frame, and ),(),,(),,( ___ yxfyxfyxf BnGnRn are the Red, Green and Blue values of the nth frame. At last, the gray-scale distance after being compensated can be described by:

)11(||),(||),(||),( ___tan yxfyxfyxf grayioncompenstatgraydifferencegraycedis

2.4 Expression of Video Segmentation

A Video can be separated into a series of scenes, and a scene can be separated into several shots, which are made up of a series of frames. Following expression describes the model of the scenes segmentation of a video:

)12(),(),( _tan__ scenesgraycedisframekeyscene fyxfiffyxf

Here framekeyscenesf __ is the key frame which segmenting the video based on scenes, and scenesf is the threshold between the successive frames and recall as Scenes_Threshold. The model of the shot segmentation from scenes of the video can be described by the following expression:

)13(),(),( _tan__ shotgraycedisframekeyshot fyxfiffyxf Here

framekeyshotf __is the key frame which segmenting the scenes obtained from the above

expression, and shotf is the threshold between the successive frames and recall as Shot_Threshold.

Different application has different threshold, and in the same application environment, the value of the Scenes_Threshold is more than the Shot_Threshold, they can be obtained by the following experimentation,

3. Experimental Results

In order to detect the Scenes_Threshold and Shot_Threshold, extensive experiments are conduced on many kinds of videos, Figure 2 show some of the images of the test videos. Figure 3 illustrates the points of the segmentation by this algorithm. Figure 3(a) is the different scenes of the video, the value of Scenes_Threshold in our method is 6, and the value of the Scenes_Threshold before compensating is 90,

746 Zhangping Hu / Procedia Environmental Sciences 10 ( 2011 ) 743 – 747

Figure 2 Some Frames of a Strong and Handsome Video

(a1) the last frame of the first scene (a2) the first frame of the next scene (a) scenes segmentation

(b1) the last frame of the first shot (b2) the first frame of the next shot (b) shots segmentation Figure 3 segmentations of the video

figure 3(b) is the different shot of the scenes from the above video, the value of Shot_Threshold in our method is 3, the value of the Shot_Threshold before compensating is 36.

747 Zhangping Hu / Procedia Environmental Sciences 10 ( 2011 ) 743 – 747

The Scenes_Threshold and Shot_Threshold of the proposed video segmentation algorithm are evaluated by using many kind videos, the details of the results by this segmentation algorithm are shown in Table 1.

Table 1 Scenes_Threshold and Shot_Threshold Video Frames Segments by Manual Segments automatism Scenes_Threshold Shot_Threshold body mechanics1 2275 4 3 6 3 body mechanics2 48011 5 3 6 4 traffic video 1 30224 3 2 4 2 traffic video 2 25693 3 2 4 2

4. Conclusion

In this paper, a new video segmentation approach is introduced. We put forward a novel technique for segmenting a video based on scenes and shot. The algorithm utilizes the difference of the average-gray-scale between the successive frame images to segment a video into a series of scenes or shots. At the same time, the algorithm gives a method to obtain the value of gray-scale automatically to compensate the error produced by the traditional algorithm. Through the test of the above experiment, the veracity of the segmentation algorithm is very high. Also, to obtain different Scenes_Threshold and Shot_Threshold from different kind of application videos is our future work.

Acknowledgements

Supported by the Science Foundation of Chongqing University of Art and Science. (Y2009JS56)

References

[1]Hanjalic A. Shot boundary Detection: Unraveled and Resolved [J]. IEEE Transon Circuits and Systems for Video Technology, 2002( 12) : 90-105.

[2]NIU Zhen-xing , LI Jie , GAO Xin-bo. Semantic color extraction and semantic shot segmentation for soccer video [J]. JOURNAL OF XIDIAN UNIVERSITY, Vol37,No.4, 2010:613-618

[3]Xie L, Chang S F, Divakar an A, et al. Structure Analysis of Soccer Video with Hidden Markov Models[C]//IEEE International Conference o n Acoustics, Speech, and Signal Processing : Vol4. Orlando: IEEE, 2002: 4096-4099.

[4]Ren R, Jo se J. M. Football Video Segmentation Based on Video Production Strategy[C] //27th European Conference on Information Retrieval. Santiago de: Springer, 2005: 433-446.

[5]Wang L, Zeng B Y, Lin S, et al. Automatic Extraction of Semantic Colors in Sports Video [ C] //IEEE International Conference o n Acoustics, Speech, and Signal Processing : Vol3. Montrea l: IEEE, 2004: 617-620.

[6]Ekin A, T ekalp A M . Shot Type Classification By Dominant Color for Sports Video Segmentation and Summarization [C]//IEEE International Conference o n Acoustics, Speech, and Signal Processing : Vo l3. Hongkong : IEEE, 2003:173-176.

[7]Ekin A, Tekalp A M. Ro bust Dominant Color Detection and Color based Application for Sport s Video [C]//IEEE International Conference on Image Processing : Vo l1. Barcelona: IEEE, 2003: 21-24.

[8]HUANG Ye-jue1, CHU Yi-ping. Multiple Objects Video Segmentation with Recognition Information[J]. Computer Engineering, Vol.36 No.9,2010:232-234,237

[9]ZHU Shi-ping,MA Li, HOU Yang-shuan. A video object segmentati,Vol.21,No.8.2010:1241 -1246 [10]Hou Rui, Ma Lizhuang, Sang Shengju. Hierarchical Structure and Matching in Video Segmentation[J]. Journal of

Computer2Aided Design & Computer Graphics, Vol.22 No.7,2010:1116-1119,1125 [11]HU Zhen-xing, XIA Li-min. Video retrieval based on video clip[J]. Journal of Central South University (Science and

Technology), Vol.41 No.3,2010:1009-1014 [12]Zhang Shu-jin.A Course of Image Processing and Analysis[M].Posts&Telecom Press,2009,11