Object Extraction in Data Mining Framework for Video Sequenc

7/29/2019 Object Extraction in Data Mining Framework for Video Sequenc

1/6

P21/1

Abstract -We are aware that the emerging video coding standard such as MPEG-4 enables various content-

based functionalities for multimedia applications. To support such functionalities and also to improve the

coding efficiency, each frame of an image sequence is decomposed into video object planes (VOPs). Each VOP

corresponds to a single moving object in the scene. This paper proposes an algorithm of key object(s)

extraction from video database. Here we describe a framework of data mining by applying segmentation and

enhancement based technique to the image and video data set, for the purpose of knowledge discovery. The

objective of this particular segmentation algorithm is to segment the huge image and video data sets in order to

extract important objects from them. This proposed method covers both spatial and temporal aspects of data

mining. The video data set is parsed to create the image data set by preserving temporal information, then the

background subtraction method is applied to get the moving object, and then double differencing operator is

applied to get better result. The gamma enhancement function is applied with histogram equalization with a

number of different parameters to improve the contrast of the resultant images. This method is thus able to

detect moving object(s) with surprisingly good location capability, and eliminate noise, which occurs due to

camera movements. The proposed approach is applied to a traffic video data set to demonstrate the

segmentation for data mining of video database.

KEYWORDS: Video mining, Video object segmentation, background subtraction, enhancement technique.

1 Introduction

There is an enormous need for visual information management in the growing field of medical imaging and

telemedicine, traffic monitoring etc. In particular, image understanding form image database queries and

retrieval are major topics of research [1,2]. This newly emerging field combines aspects from databases, imageprocessing and understanding, knowledge-based systems and context-based compression. Thus this growing

amount of video information requires efficient management, and to support more efficient video database

management, a video segmentation method has become necessary.

Segmentation is one of the most challenging tasks in the field of video mining, since it lies at the base of

virtually any scene analysis problem. The general approach that motivates the need for segmentation is that the

problem of analyzing a complete scene can be subdivided into a set of simpler problems, of analyzing its

primitives. Then, each primitive object in the scene can be processed independently [3][4]. This approach

greatly simplifies many high-level vision problems, such as recognition, classification, scene analysis, etc.

However, the ultimate goal of the segmentation process is to divide an image into the physical objects

composing it, so that each region constitutes a semantically meaningful entity.

In order to identify and track the temporal and relative spatial positions of objects in video sequences, it is

necessary to have object-based representation of video data. For this purpose, attention has been devoted tosegmenting video frames into regions such that each region, or a group of regions, corresponds to an object that

is meaningful to human viewers [7, 8]. While most of the previous works [5] [6] are based on low-level global

features, such as color histogram, shape and texture feature, our method focuses on obtaining object level

segmentation; obtaining objects in each frame and their traces across the frames.

There has been a large amount of researches [18, 19] to find object from video sequence when the segmentation

problem is considered for a fixed camera domain, a usual technique is required to resolve the foreground

Ms. Stuti Dave

Samrat Ashok Technological Institute,

Vidisha, M.P., India

[email protected]

Prof. Manish Manoria

Samrat Ashok Technological Institute,

Vidisha,M.P.,India

[email protected]

Object Extraction in Data Mining Framework for Video Sequence


2/6

P21/2

objects i.e. background subtraction [9]. This involves the creation of a background model that is subtracted

from the input image to create a difference image. The new difference image only contains objects or new

features that have not yet appeared in the background. This method is especially suitable for video conferencing

[10] and surveillance [11] applications, where the backgrounds remain still. Neri et al. [12] suggested a

different segmentation technique based on change detection. The method assumes that the objects are moving

over a static background, thus, if a moving camera is detected, then global motion compensation be applied

first. Potential foreground regions are detected in a preliminary stage by applying a higher-order statistics

(HOS) test to a group of inter-frame differences. The non-zero values in these difference frames are due to

moving objects.Few references [13, 14] have addressed the problem of object segmentation and tracking using an active

camera, they cannot able to give spatial information of moving object, which is very inefficient and non-flexible

This method is thus able to detect moving object(s) with surprisingly good location capability.

2 Video Objects Extraction

Figure 1 depicts the diagram of the proposed object extraction scheme. Prior to performing segmentation and

extraction, a video database is created. It is a temporal data because it contains before-after sequence

relationship that is first frame comes before second frame and second frame comes after first frame. Video

parsing divides large video sequence in to frames or images.

Figure 1 diagram of the proposed object segmentation and extraction method

The number of images/frames per second gives motion in video. Then reference frame of stationary component

is created. The proposed background substraction method is applied which is able to detect moving object from

each frame.

2.1 Background subtraction

Background subtraction is a technique to remove nonmoving components from a video sequence. The

main assumption for its application is that the camera remains stationary. The basic principle is to create a

reference frame of the stationary components in the image.

Step1: Once reference frame of stationary component is created, the reference frame is subtracted from any

subsequent images. Those pixels resulting from new (moving) objects will generate a difference not equal to

zero.

Step2: In the video sequences the nonmoving objects were manually selected from the video data and then

averaged together. Here accumulating and averaging images of the target area in some time interval construct a

reference frame. The image sequence used consists of about 16 minutes of video from a traffic intersectionwith approximately constant lighting conditions. (The difference image is created by subtracting the reference

frame from the current image) .The results are scaled by eq. 1.

s = clog (1+|dij|) (1)

Where dij is the value for the difference at pixel ij

Step3: The double-difference operator (also called three-frame difference) has been applied to improve the

performance of background subtraction. It is able to detect moving points with surprisingly good location

Video file

ImageDatabase

BackgroundSubtraction

ObjectDetection

Perform gammacorrection

Object

extracted

Create negative

image

Improve

contrast


3/6

P21/3

capability, and eliminate noise, which occurs due to camera movements and able to filter very small moving

objects in the scene. Hereafter, we sketch the double-difference operator. Eq.2 defines the difference-image.

Dri (i,j) = [ Iri(i,j) - I r,i-1(i,j) ] (2)

Step 4: The double-difference image is obtained by eq.3 which perform a logical AND between pixels

belonging to two subsequent difference-images, threshold by T

1, if (Dri+1 (i,j) >T ) ( Dri (i,j) > T )D` Dri (i,j) =

0 otherwise (3)

The non-zero values in these difference frames are either due to noise or moving objects. The

intensity of the resultant image is improved by gamma correction; here different perimeters are tried to obtain

best result.

2(a) Reference frame 2(b) Frame 4

2(c) key object extracted from frame 4

Figure 2: Example result of background subtraction

Figure 2(c) is the resultant image having key object occurring after applying the background substraction

algorithm.

Some approaches [17] or [12], which used background subtraction method, constructed the reference by

accumulating and averaging images of the target area for some time interval. As mentioned above, this is not a

robust technique as it is sensitive to intensity variations [15],[16]. The segmentation algorithms usually start

with the gray value difference image between two consecutive frames. In this paper the computation of images

has performed on gray level images because the gray level of the images or region is having uniform intensity.

The intensity change in data flow can very well be observed in gray level. Then the gamma correction function

is applied to improve the contrast of the resultant image as shown in figure 3.

3 Experimental Setup

We have applied the proposed scheme to the traffic video sequence in our experiments. Their resolution

is 195 x 255 pixels. The color video sequence consists of about 10 minutes of video from a traffic intersection

with approximately constant lighting conditions. The color video clips used for this study were originally

digitized in AVI format at 16 frames/second. The original video frames were of size 390 rows 510 columns, 24-

bit color. To reduce computation time, we made our test video clips by extracting frames from these originals at

the rate of 4 frames per second. For simplicity and real-time processing purpose, we transform the color video

frames to gray scale images and resize them to half of the original size (195 rows 255 columns). A small portion


4/6

P21/4

of the traffic video is used to illustrate how the proposed framework can be applied in various kinds (i.e., TV

commercials, movie, and documentary, TV dramas, and animation). This process can be done in real-time or

off-line.

3.1 Experiment Results

The enhanced background subtraction method to the video sequences is able to remove nonmoving

components from a video sequence. The main principle is to create a reference frame of the stationary

components in the image. The reference frame is subtracted from any subsequent images. Those pixels resulting

from new (moving) objects will generate a difference not equal to zero .The double-difference operator is used,which is able to detect moving points with amazingly good location capability, and is proved robust to noise

due to camera movements and able to filter very small moving objects in the scene. By using this background

subtraction method the accuracy of segmentation is increased and this thus provides support for real-time

processing. The segmentation results for a few frames are shown in Figure 2 along with the original frames

adjacent to them. Related work has been done based on image processing and rule-based reasoning. Here we

applied the gamma correction and tried different perimeters to increase the contrast of the resultant image.

This proposed framework, however can deal with more complex situations with more accuracy. Some of the

objects, which are closely located to each other, are extracted in single segment. In Figure 2, the frame in the

left most column (Figure 2(a)) is the background reference frame. The second column (Figure 2(b)) shows the

original frame. The figure 2(c) shows the images after background subtraction. The extracted key object is

shown in the third column (Figure 2(d)). Figure 3 shows the result gamma corrected images with parameters

0.3, 1.5, 3, 2.1. The result shown in figure 5 shows the improved intensity of the resultant frame compared with

original one.

Figure 3 gamma corrected frame with parameters 0.3, 1.5, 3, 2.1

Figure 4 Histogram before Equalization (a) and After Equalization (b)


5/6

P21/5

4. Conclusion

This paper introduces an algorithm for key object extraction for segmentation in video sequences. The

main assumption underlying by proposed algorithm is that a motion characterizes physical objects that is

separate from background. The algorithm is based on background substraction method. Experimental results

demonstrate that proposed technique can successfully extract moving objects from video sequence. The

boundaries of the extracted objects are accurate enough to place them in different frames. If there is insufficient

motion, the algorithm is able to perform the correct segmentation and show the position of object in each frame.

In order to extract the key object(s) from the video sequence, background subtraction techniques areemployed. Using the background subtraction technique, both the efficiency of the segmentation process and the

accuracy of the segmentation results are improved achieving more accurate video indexing and annotation. The

existing schemes try to extract key frames from the video shot, but in this proposed scheme is able to extract

key object(s) instead of key frame(s). Here we proposed a simple and efficient technique for object

segmentation in video sequence. It is able to detect moving points with surprisingly good location capability,

and eliminate noise, which occurs due to camera movements. It is easy to implement and fast to compute since

it uses the segmentation result of the previous video frame to speed up the segmentation process of the current

video frame. The proposed work extract small moving component(s) from image sequence and therefore this

opens scope for possible future work in the area of doing all operation in image processing, and the global

motion estimation and compensation. By considering more than one reference frame in the segmentation

process, the motion can be estimated by comparing each frame with more than one reference frame so as to

detect all the information of video scene. This work opens scope for futher study on data mining work,clusttering and association mining on video data set.

5. Acknowledgement

This work was supported by all India council of technical education AICTE),New Delhi,India under RPS

grant F.No.-8022/RID/NPROJ/RPS-14/2003-04.

References

1. Jiawei Han, Data Mining Techniques, DBLP, SIGMOD, 1996, pp: 545 [Conf]

2. Michael C. Burl, Mining Patterns of Activity from Video Data, Department of Computer Science University

of Colorado, Boulder, First SIAM International Conference on Data Mining, 2002.

3 R. Mech, M. Wollborn, "A Noise Robust Method for Segmentation of Moving Objects in Video Sequences,"

p. 2657, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP'97) -

Volume 4, 1997.

4 Xiaodong CAI, F.H.Ali and E.Stipidis An Integrated framework for Robust and Fast Automatic Video

Segmentation Communication Research Group School of Engineering and Information Technology University

of Sussex, Publication Date: Feb,2002,Volume,12,Issue:2,On page(s): [122-129]P21-5

5 P.Stanchev, IASTED, Using image mining for image retrieval, Cancum, Maxico, 214-218, conf., CSE, May

19-21,2003

6 QBIC project: querying images by content, using color, texture, and shape Carlton W. Niblack, Ron Barber,

Will Equitz, Myron D. Flickner, Eduardo H. Glasman, Dragutin Petkovic, Peter Yanker, Christos Faloutsos,

and Gabriel Taubin , April 1993, pp. [173-187]

Paper No. P21

7 Teuner, A.Pichler, Hosticka, B.J. Fraunhofer Inst. of Microelectron. Circuits & Syst., Duisburg;

Unsupervised texture segmentation of images using tuned matched Gabor filters, Image Processing, IEEE

Transactions on, Jun 1995, Volume: 4, Issue: 6 On page(s): 863-870

8 Jia-Yu Pan and Christos Faloutsos, "Geoplot": Spatial Data Mining on Video Libraries. In Proceedings of the

Eleventh International conference on information and,Knowledge, Management, CIKM,2002, Mclean, VA

(near Washington DC), November ,pp [4-9], 2002


6/6

P21/6

9 YuriA. Ivanov, Aaron, F.Bobick, JohnLiu,Fast Lighting Independent Background Subtraction,DBLP,

International Journal of Computer Vision, 2000, v:37, n:2, pp:199-207 [Journal]

10 C.W. Lin, Y.-J. Chang, Y.-C. Chen, and M.-T. Sun, Implementation of a real time object-based virtual

meeting system, in Proc. IEEE Int. Conf. Multimedia and Expo, pp. [565-568], August 2001, Tokyo, Japan.

11 I. Haritaoglu, D. Harwood, and L.S. Davis, W4: Who? When? Where? What? A real-time system for

detecting and tracking people, in Proc. IEEE Int. Conf. Automatic Face and Gesture Recognition, pp. [222-

227],1998, Nara, Japan.

12 A.Neri, S. Colonnese, G. Russo, P. Talone, Automatic moving object and background separation, Signal

Processing 66 (1998), pp. [219-232].13 Y. Ye, J. K. Tsotsos, K. Bennet, and E. Harley, Tracking a person with pre-recorded image database and a

pan, tilt, and zoom camera, in Proc. IEEE Workshop on Visual Surveillance, pp.[10-17], 1998.

14 Hat, M. Saptharishi, and P. K. Khosla, Motion detection and segmentation using image mosaics, in Proc.

IEEE Int. Conf. Multimedia and Expo, pp. 1577-1580, July 2000, NY, USA.

15 Cheung, C. Kamath, Robust Techniques for Back-ground Subtraction in Urban Traffic Video", In Video

Comm and Image Proc., SPIE Electronic Imaging, San Jose 2004

16. Ismail Haritaoglu, David Harwood, Larry S. Davis: A Fast Background Scene Modeling and Maintenance

for Outdoor Surveillance. ICPR 2000: 4179-4183

17. T. Meier and K.N. Ngan, Automatic segmentation based on Hausdorff object tracking,.Doc. ISO/IEC

JTC1/SC29/WG11 MPEG97/m2238, July 1997.

18.T. Meier, and K.N. Ngan, Automatic segmentation of moving objects for video object plane generation.

IEEE Trans. on Circuits and Systems for Video Technology, Vol. 8, No.5, September 1998.

Documents

Object Extraction in Data Mining Framework for Video Sequenc