Road Region Detection in Video System

CHI Jian-nan1, ZHOU Nan-nan1, ZHANG peng-yi1 and ZHENG si-yi1,2

1Information engineering School, University of Science and Technology Beijing, Beijing 100083, China

2 Beijing Command College of Chinese People's Armed Police Force, 100012, Beijing, China Email: sy_jnchi@126.com, sisizheng@163.com

Abstract

In this paper, image multi-scale edge detection based on anti-symmetrical biorthogonal wavelet is given detailedly in theory. Namely convolution operation property which anti-symmetrical biorthogonal wavelet transform bears is deduced, and function of which anti-symmetrical biorthogonal wavelet transform works as differential operator is analyzed. Algorithm of wavelet decomposition in which multi-scale edge can be detected is put forward. Based on above, a road region detection method in video system is put forward. Using multi-scale edge image of anti-symmetrical wavelet decomposition Region Growth is applied to identify road area roughly. Then borderline of road is extracted using Hough Transformation and road region is defined finally. Experimental results indicate that the method presented in the paper is effective.

1. Introduction

In Active Video Surveillance System, road area is usually the place where objects exist. It is very important to segment the road region in image for effective object detection. Road in video image is linear targets. Though the line is a simple geometric figure, it is hard to describe with invariant parameters or characters when comprehensively allowing for different environments, different imaging conditions, different road status and so on. Most existing road detection approaches are based on image processing techniques such as edge detection, edge connection, edge based feature extraction and so on.

There is not a general way at present to exact road information from any road image with different resolutions. Road detection methods now existing can be classified into three catalogues: road feature based methods [1-2], context information based methods [3],

and there synthesis. In the road features based methods the road is modeled adopting its intensity and geometric features and mathematics approaches such as template matching, tracing and linking parallel edge, dynamic programming, least square B-spline curve, Kalman filter, etc. are used to identify road, Methods based on context information makes use of relationship between roads and other adjacent objects to extract the information which cannot derive from road itself. From presented paper it is noticed that methods based on features of road is the main idea in this field. To certain video images, most researchers make use of both features of road and context information to automatically extract and link road lines. The system includes two sections: detection and recognition. In lower level the main task is to detect and link edges, in middle level processes features; the higher level recognizes features. In short, the system first detects and analyzes road characters, and then recognizes roads in terms of computer graphics, artificial intelligence, and pattern recognition.

Motivated by the idea above, in this paper a new architecture based on Multi-scale edge image of anti- symmetrical biorthogonal wavelet transformation is put forward, in which Region Growth based road region segmentation and Hough Transformation based borderline detection are utilized to identify road regions.

This paper is organized as follows. In section 2.1, 2.2 and 2.3, image multi-scale edge detection method based on odd symmetrical biorthogonal wavelet decomposition is given in theory. Section 3 and 4 then describes the approach used to detect road targets in Active Video Surveillance System, gives an overview of the whole detection procedure. Section 5 describes the experiments carried out and corresponding results. Conclusions are drawn in section 6.

Global Congress on Intelligent Systems

978-0-7695-3571-5/09 $25.00 © 2009 IEEE

DOI 10.1109/GCIS.2009.339

500

2 Multi-scale edge detection based on anti-symmetrical bi-orthogonal wavelet 2.1 Convolution operation property of anti-symmetrical bi-orthogonal wavelet transform

Suppose that ( )xψ and ( )xψ are bi-orthogonal wavelet function in space 2 ( )L R and ( )xϕ , ( )xϕ are corresponding scale functions. If ( )xψ is odd symmetrical, then ( )xϕ must be even symmetrical. Let ( ) ( )x xψΨ = − , ( ) ( )x xψΨ = − , ( ) ( )x xϕΦ = − and

( ) ( )x xϕΦ = − ,obviously, ( )xΦ and ( )xΨ are also even symmetrical scale function and odd symmetrical wavelet function. Suppose that the support set of the odd bi-orthogonal ( )xΨ is (0,2 1)N − , then its symmetry

center must be 2 12

N − . Thus:

2 1 2 1( ) ( )2 2

N Nx xψ ψ− −+ = − − (1)

Let 2 12

Nx x−= − ,

then (2 1 ) ( )N x xψ ψ− − = − ,

also: 22 ,

( ) 2 ( 2 )j

jj

kx x kψ ψ

− −− = − − +

22 (2 1 2 )j

jN x kψ− −= − + −

2 , (2 1 )( )j N kxψ

− − −= (2)

Formula (3) illuminates that, for the odd symmetrical bi-orthogonal wavelet

2 ,( )j jkx Wψ ∈ , we

have2 , 2 ,

( ) ( )j j jk kx x WψΨ = − ∈ . Namely

2 ,( )j kxΨ , is also a

group of Riesz base of space jW . Similarly, 2 ,

( )j jkx Vϕ ∈ .

Based on the bi-orthogonal conditions of two multi-scale spaces, it is easy to prove that ( )xΨ and ( )xΨ ,

( )xΦ and ( )xΦ form the bi-orthogonal wavelet function in the same multi-scale space. So, the real bi-orthogonal wavelet transform can be written as:

2 , 2 ,, ( )j jk k

W f f xψ= ⟨ ⟩

2 ,( ) ( )j k

f x x= ∗Ψ

2 ( ) (2 ( 2 ))j j j

Rf x x k dxψ− −= −∫ (3)

Therefore, the odd symmetrical bi-orthogonal wavelet ( ) jx Wψ ∈ transform of correlation operation is equivalent to the wavelet ( ) jx WΨ ∈ transform of convolution operation. For the odd symmetrical bi-orthogonal wavelet function and its corresponding scale function, the wavelet and scale functions and

their corresponding dual wavelet and scale functions have following relation:

2 , 2 ,( ) ( ), ( )j jk k

S f f x xϕ=< >

2 ,( ) ( )j k

f x x= ∗Φ (4)

2 , 2 ,( ), ( )j jk k

W f f x xψ=< >

2 ,( ) ( )j k

f x x= ∗Ψ (5)

2 , 2 ,( ) ( )J Jk k

k Z

f x S f xϕ∈

=∑ 2 , 2 ,1

( )j j

J

k kj k Z

W f xψ= ∈

+∑∑ (6)

Summed up, the odd symmetrical bi-orthogonal wavelet transform has the convolution operation property.

2.2 Differential operator function of odd symmetrical bi-orthogonal wavelet transform

The implementation of the edge detection using the odd symmetrical bi-orthogonal wavelet was analyzed in Refs.[4-5]. The differential operator function of the odd symmetrical wavelet was proved also. The Ref.[4] pointed out that if the scale function ( )xϕ was smooth enough, and odd symmetrical bi-orthogonal compactly supported wavelet ( )xψ was proportional to the derivative of corresponding scale function (2 )tϕ of the high level resolution approximately, namely,

0( ) ( / )x c d dxψ ≈ (2 0.5)xϕ − symmetrical center is -0.5. in Ref.[6], two scale equations in frequency domain were used to prove that the Fourier transform of integral function of wavelet function was proportional to the Fourier transform of scale function approximately, namely:

/ 4ˆ ˆ( ) ~ ( / 2)je ωθ ω η ϕ ω• (7) Thus: ( ) ~ (2 0.5)x xθ η ϕ• + (8)

Formula (9) indicates that the integral function of the odd symmetrical bi-orthogonal compactly supported wavelet ( ) ( )

xx t dtθ ψ

−∞= ∫ is proportional to the

scale function ( )xϕ . This conclusion is very important to the edge detection using the odd symmetrical bi-orthogonal wavelet. Suppose ( )f x is a quadratic integrable function, its wavelet transform of j th level resolution is:

2( ), ( )j jW f f x xψ=< >

( ), ( ) /jf x x xθ= ∂ < > ∂ (9) From formula （8）:

2~ ( ), (2 ) /j jW f f x x xϕ∂ < > ∂

12/jS f x+= ∂ ∂

( ( ) (2 )) /jf x x x= ∂ ∗Φ ∂ (10)

501

Where 12 jS f+ is an approximate component of 1j + level resolution. Formula (10) shows that the wavelet transform coefficient

2 jW f of j level resolution is proportional to the approximate component of 1j + level resolution. Thus, the local module maximum of

2 jW f corresponds to point where 12 jS f+ changes intensely. The wavelet transform of j level resolution for 2D signal ( , )f x y can be written as:

2_ ( , ), ( ) ( )j j jW f LH f x y x yϕ ψ=< >

~ ( ( , ) ( ) (2 )) /j jf x y x y y∂ ∗Φ Φ ∂ (11)

2_ ( , ), ( ) ( )j j jW f HL f x y y xψ ϕ=< >

~ ( , ), (2 ) ( ) /j jf x y x y xϕ ϕ∂ < > ∂ ( ( , ) (2 ) ( )) /j jf x y x y x= ∂ ∗Φ Φ ∂ (12)

2.3 Multi-scale edge extraction in reconstruction of odd symmetrical bi-orthogonal wavelet

The proposed new wavelet reconstruction algorithm can calculate the module and phase graph of directional gradient of the approximate component image of different level resolution and extract the multi-scale edge. This is implemented in the process of image reconstruction.

For the odd symmetrical bi-orthogonal wavelet, suppose that { }k k Zh ∈ , { }k k Zg ∈ and { }k k Zh ∈ , { }k k Zg ∈ are a group of dual filters. { }k k Zh ∈ and { }k k Zg ∈ are analytical filters. { }k k Zh ∈ and { }k k Zg ∈ are synthetically filters. Different from the average, we use intermediate results of decomposition to calculate the module value image and phase angle image in multi resolutions for extraction. The calculations are carried out during the process of decomposition, with no extra computational complexity.

We use anti symmetrical bi-orthogonal wavelet to decompose 2D signal ( , )f x y in multi scales. Suppose the 1j + level approximate component is:

>=<+ )2()2(),,(_12 yxyxfLLfW jjj ϕϕ (13) We use high-pass filter kg to reconstruct each

column of 12_jW f LL+ :

12_ ( , ), (2 ) ( )j

vj jHD W f f x y x yϕ ψ+ =< >

~ ( , ), (2 ) (2 ) /j jf x y x y yϕ ϕ∂ < > ∂ ( ( , ) (2 ) (2 )) /j jf x y x y y= ∂ ∗Φ Φ ∂ (14)

and to reconstruct each row of 12_jW f LL+ as follow:

12_ ( , ), ( ) (2 )j

hj jHD W f f x y x yψ ϕ+ =< >

~ ( , ), (2 ) (2 ) /j jf x y x y xϕ ϕ∂ < > ∂

( ( , ) (2 ) (2 )) /j jf x y x y x= ∂ ∗Φ Φ ∂ (15) As no down sampling process, 12

_ jvHD W f+ and

12_ j

hHD W f+ share the same size with 12_jW f LL+ . The

latter decomposition goes according to the direction of figure 2 and 3. We get the j level approximate component and three detail components.

12

_jW f LL+

rkg 2↓

2↓ ckh

ckg 2↓

2

_jW f HL

2

_jW f HH

ckg 2↓ r

kh 2↓ 2

_jW f LH

rkh 2↓ 2↓ c

kh 2

_jW f LL

12

( , )jM f x y+

12( , )jA f x y+

12

_ jvHD W f+

12

_ jhHD W f+

Figure 1. Circuit diagram of wavelet reconstruction for image edge extraction

12

_ jvHD W f+ and 12

_ jhHD W f+ are called as half-

reconstructions proportional to the first order horizontal and vertical derivatives of 1j + level approximate component respectively. According to formulas (14) and (15), gradient vectors of the approximate component 12

( ( , )) _jW f x y LL+ of 1j + level resolution for 2D signal ( , )f x y can be denoted as:

12

( , ), (2 ) (2 ) /_

( , ), (2 ) (2 ) /j

f x y x y xW f LL

f x y x y yϕ ϕϕ ϕ+

∂ < > ∂⎡ ⎤∇ = ⎢ ⎥∂ < > ∂⎣ ⎦ 1

1

2

2

_~

_j

j

h

v

HD W fHD W f

+

+

⎡ ⎤⎢ ⎥⎢ ⎥⎣ ⎦

(16)

Therefore, the module

2( , )jM f x y and the phase

angle2

( , )jA f x y of the directional gradient for the approximate component of 1j + level resolution can be calculated using the decomposition data of the odd symmetrical bi-orthogonal wavelet.

1 1 12 2

2 2 2| _ | | _ |j j j

h vM f HD W f HD W f+ + += + ， 1, 2, ,j J= − − − (17)

1

1

1

22

2

_arg tan( )

_j

j

j

v

h

HD W fA f

HD W f+

+

+

= ，

1, 2, ,j J= − − − (18)

502

12

_jW f LL+

12

, (2 ) ( )

_j

r

k

j j

h W f LL

f y xϕ ϕ

+

< >=

2_jW f LL

1

1

2

2

, ( 2 ) ( )

, (2 ) (2 ) /

_

_j

j

r

k

j j

j j

h

g W f LL

f y x

f y x x

HD W f

ϕ ψ

ϕ ϕ

+

+

=< >

= ∂ < > ∂

=

1

1

2

2

, ( 2 ) ( )

, (2 ) (2 ) /

_

_j

j

c

k

j j

j j

v

g W f LL

f x y

f x y y

HD W f

ϕ ψ

ϕ ϕ

+

+

=< >

= ∂ < > ∂

=

1

1 1

1

1

1

2

2 2

2 2

2

2

2

_ _

_arg tan( )

_

j

j j

j

j

j

h v

v

h

M f

HD W HD W

A f

HD W

HD W

f f

f

f

+

+ +

+

+

+

=

=

+

2_jW f HL

2_jW f LH

2_jW f HH

LL

LH

HL

HH

Figure 2. Schematic diagram of wavelet reconstruction for image edge extraction

According to the module value and phase angle computed by formulas (17) and (18), the pixels of edge of 1j + level resolution can be located by module maximum detection. The multi-scale edge can be extracted through wavelet decomposition data of each level.

3 Detecting road areas by Region Growth

Fig 4(a) is a frame image from video while its multi-scale edge image is Fig4 (b). From Fig4 (a) we can see that, besides roads and pedestrians, there are trees, buildings, rail fence, and etc in the image, which make image edge very complex in direction and intensity distribution. Thus, road borderline might completely be submerged in clutter background. If we do detection directly based on image edge, the edges of objects outside the road area will seriously mislead the result. It is noticed that edges in the areas outside the road are rich and complex, but in the road area, there are almost no edges besides some tiny ones formed by shade or rough road surface; these short edges effect the detection of road little. Given this observation, the first step is to find the road area roughly in large scale edge image by ways of region growth.

Region growth is a popular way of image segmentation. As its name implies, region growth is a procedure that groups pixels or subregions that share similar properties into larger regions. The fundamental problem is where to place seed points and how to predefine the criteria for growing. we choose the starting points using the prior information that road area in the image has intensity similarity, which is also available in large scale edge image. We sample some pixels by equal interval from the bottom in the large scale edge image.

Each pixel row is regarded as one-dimensional signal. We chose a one-dimension template whose scale is one-third image width. We make template center go through each pixel of one-dimensional signal above, and calculate the pixel entropy difference or variance of the field that template covered, which reflect the discrete degree of data of template. We select the coordinate of pixel point with minimum entropy difference as y coordinate of seed element and number of rows extracted as x coordinate of seed pixel to determine the position of seed element. Then we define gray value as similarity property to generate uniform space region gradually until there's no point or small area available.

503

(a) (b)

Figure 3. (a) Original image; (b) Edge image by the second scale of wavelet transform 4 Road boundaries detection using the Hough Transform

Hough Transform is an approach that can be used to find and link line segments in an image, which can detect lines by accumulation in parameter space. The process is exempted from noise and broken edges, and we can also derive the direction and scale of the image during the detection.

Region Growth alone can form the area of the roads, but the road area might be incomplete with the effect of shadow or shelter. Thus it is necessary to combine Region Growth method with edge information to get a more complete area. In this paper, we use Hough Transform to fit multi-scale edges and choose several longest lines of the fitted edges to be the candidates, and finally determine road boundaries with the help of Region Growth process which has formed the road area. Sketch map of road region detection is showed in Fig.5.

5 Experimental results and analysis

In our experiments large numbers of video images got from city streets and campus scenes have been used as samples. Experimental results show that the detection method presented in this paper is effective

when pedestrians and vehicles detecting is done. Fig. 6 (a)(b) are two adjacent frames of gray images of campus roads intercepted from the video. While Fig. 7(a)(b) are taken from the scene of city streets. Each image has 480 640× pixels. The camera moves from left to right slowly during recording. In these images, pedestrians and vehicles are objects moving in complex scene. Method introduced in 2.3 is used to decompose two adjacent images intercepted from image sequences 3 levels by anti-symmetrical bi-orthogonal wavelet. We obtain 3 edge images of different scales by half-decomposing process. We can see that because of the effect of noise and other factors edges in the images may be broken or furcated. During the process of the decomposition, the wavelet transform of image in each resolution concludes important edge information. Edges in subimage with higher resolution (smaller scale) are located exactly, which are sensitive to noise. While edges with lower resolution (larger scale) are explicit and continuous and are immune to noise, which are easy to detection, but their location is not accurate. Since what we want to do is to detect road region and boundary. We chose third level wavelet edge image to detect road area by means of Region Growth and line fitting. Fig.6(c) and 7(c) are low-frequency image,6(d) and 7(d) are edge images derived from the process of wavelet decomposition in 3 scales.

We place seeds in the third-scale edge image

Biorthogonal wavelet based edge image

Approximately locate road area by region growing method

Extract candidate road boundaries by Hough transform

Road area Comprehensive judgments

Figure 4. Flow chart of road area detection

504

showed in figure 6(d), 7(d) and grow to generate road area. After that, we use Hough transform to fit the boundaries in this image and take the longest two lines as candidates. If two of them are approximately parallel (slope approximately equal) and locate at different sides of the road, they are determined the boundaries of the road, or take several other longest lines as candidates, and repeat the process above until the proper boundaries are got.

Fig.6(e), 7(e) are the detection results of Region Growth in 3 scale edge image in campus and city scene respectively. Fig.6(f), 7(f) are the edge image fitted by Hough Transform. Fig.6(g)(h), 7(g)(h) shows the detected road area in edge image and original image with blue line. From the results above, we can see that approach presented in this paper is very effective for road area detection in Active Video Surveillance System.

(a) (b)

(c) (d)

(e) (f)

505

(g) (h)

Figure 5. Results of road area detection of images in campus scene (a)(b) two adjacent frame of original images; (c) 3 scale low-frequency subimage; (d) 3 scale edge image after wavelet decomposition; (e) Result of Region Growth in 3 scale edge image; (f) Result of Hough transformation; (g) Result of road area detection in 1 scale edge image; (h) Final result of road area detection.

(a) (b)

(c) (d)

(e) (f)

506

(g) (h)

Figure 6. Results of road area detection of images in city scene (a)(b)two adjacent frame of original images; (c) 3 scale low-frequency subimage; (d) 3 scale edge image after wavelet decomposition; (e)Result of Region Growth in 3 scale edge image;(f)Result of Hough transformation; (g)Result of road area detection in 1 scale edge image; (h)Final result of road area detection

6 Conclusions

In this paper, we present an anti-symmetrical biorthogonal wavelet based method about road target detection of dynamic scene in active video monitoring system. The main idea of this approach is to make full use of the multi-scale edge data derived from the anti-symmetrical biorthogonal wavelet tower decomposition to segment the road boundary and region with combination of Hough Transform and Region Growth. The method developed in this paper delivers a good result in application. There are still a lot of work in detail to do when considering the features of different scenes. What we have done is just a start. Anti-symmetrical biorthogonal wavelet transformation described above provides us multi-scale low frequency, high frequency data and edge image. In the future we will take full advantage of anti-symmetrical biorthogonal wavelet transformation and do better research on objects detection of Active Video Surveillance System.

Acknowledgements

This work is sponsored by China Postdoctoral Science Foundation (20060400400), National Natural Science Foundation (60574090) and National High Technology Research and Development Program of China (2007AA01Z160).

Reference [1] Lindi J, Quackenbush, A Review of Techniques for Extracting Linear Features from Imagery. Photogrammetric Engineering and Remote Sensing, 2004, 70(12), pp.1383-1392.

[2] Baltsavias E. P, Object Extraction and Revision by Image Analysis Using Existing Geodata and Knowledge: Current Status and Steps toward Operational System. ISPRS Journal of Photogrammetry and Remote Sensing, 2004, 68(4), pp.129-151.

[3] Baumgartner A, Eckstein W, Mayer H, Context-supported Road Extraction. Automatic Extraction of Man-Made Objects from Aerial and Space Image. Proceedings of the Centro Stefano Franscin,. Ascona, Birkhauser Verlag, Basel, 1997, pp.299-308.

[4] WEI Hai, SHEN Lan-sun, Edge Detection by Using Anti-symmetrical Biorthogonal Wavelets. ACTA ELECTRONICA SINAICA, 2002, 30(3), pp.313-316.

[5] PENG Jin-ye, YU Bian-zhang, WANG Da-kai, LI Nan, Multi- Scale Symmetry Transform with Application to Location of Feature Points on Human Face Image. ACTA ELECTRON -ICA SINICA, 2002, 30(3), pp.362-366.

[6] ZHAN Yu-jin, Image Segmentation. Science Publish Company, Beijing, 2001.

[7] Mae, Y., et al. Optical Flow Based Realtime object tracking by active vision system. Proceeding of the 2nd Japan-France Congress on Mechatronics, 1994, pp.545-548

507