Abnormal Traffic Detection using Intelligent Driver Model *

Waqas Sultani and Jin Young Choi EECS Department, Automation and System Research Institute (ASRI)

Seoul National University, Seoul, Korea E-mail:sultani@neuro.snu.ac.kr, jychoi@snu.ac.kr

Abstract

We present a novel approach for detecting and localizing abnormal traffic using intelligent driver model. Specifically, we advect particles over video sequence. By treating each particle as a car, we compute driver behavior using intelligent driver model. The behaviors are learned using latent dirichlet allocation and frames are classified as abnormal using likelihood threshold criteria. In order to localize the abnormality; we compute spatial gradients of behaviors and construct Finite Time Lyaponov Field. Finally the region of abnormality is segmented using watershed algorithm. The effectiveness of proposed approach is validated using videos from stock footage websites. 1. Introduction

Detection of abnormal traffic is one of the most difficult video surveillance tasks since it requires an accurate perception of traffic. Statistics shows that rate of vehicle accidents has been continuously increasing over the past years. For example, 6240000, 6380000, and 6316000, auto accidents are reported in 2003, 2004, and 2005 respectively, only in USA [1]. This necessitates the need for efficient traffic analysis and abnormality detection systems.

Recently, there has been much interest in analyzing traffic on intersections and in highways. Hu et al [4] presented a method to detect abnormality in traffic scene using multiple objects tracking. However, efficiency of their method was based on obtaining complete tracks of multiple objects which is very difficult to achieve in traffic scenes due to occlusion, clutter, and low resolution. Salemi et al [12] proposed a

(a) (b) Figure 1. Examples of (a) Normal and (b) Abnormal traffic.

method to model motion patterns of objects in the form of multivariate nonparametric probability density function of spatiotemporal variables. Nonetheless, they completely ignored any kind of interaction among individuals during abnormality detection. Wang et al [13] reported hierarchical Bayesian models to learn visual interaction using pure optical flow as a low level features. Hospedales et al [3] proposed a novel method to detect abnormality in the traffic scene using Markov clustering topic model. However, their method was unable to localize abnormality in scene.

In this paper, to solve the problems mentioned above, we use intelligent driver model [6] along with particle advection [2], which capture the dynamics of traffic scene. We expect the proposed model can extract much useful information from complex traffic scene containing many particles by using simple microscopic models [11]. We compute drivers’ behaviors over the normal traffic scene using intelligent driver model and avoid tracking by using optical flow and particle advection. We employ these behaviors values as our low level features and use them as a measure of interaction among vehicle. The appearance of unusual behaviors is used to declare abnormality. In order to localize region of abnormality, we construct Finite Time Lyaponov Field using behavior values appeared in abnormal traffic and segment the region using watershed algorithm.

* This research work is supported by “MS level training in Korean Universities”, Pakistan and Samsung Techwin.

2010 International Conference on Pattern Recognition

1051-4651/10 $26.00 © 2010 IEEE

DOI 10.1109/ICPR.2010.88

324

2010 International Conference on Pattern Recognition

1051-4651/10 $26.00 © 2010 IEEE

DOI 10.1109/ICPR.2010.88

324

2010 International Conference on Pattern Recognition

1051-4651/10 $26.00 © 2010 IEEE

DOI 10.1109/ICPR.2010.88

324

2010 International Conference on Pattern Recognition

1051-4651/10 $26.00 © 2010 IEEE

DOI 10.1109/ICPR.2010.88

324

2010 International Conference on Pattern Recognition

1051-4651/10 $26.00 © 2010 IEEE

DOI 10.1109/ICPR.2010.88

324

2. Intelligent Driver Model

The intelligent driver model closely approximates driver behavior and incorporate reaction to velocity difference. It is fast, robust, accident free and reproduces empirically observed dynamics [7], [9].

There are mainly three types of traffic on roads i.e. free flow, synchronized and congested traffic [11]. Intelligent driver model is a novel and well established car following model that describes behaviors of driver for all three types of traffic.

According to this model, each car changes its velocity depending on distance to and velocity of preceding car. On free roads, car will asymptotically reach its desired velocity by

[1 ( ) ]nn

a c t

n

d e s

vd v

d t vδα= − , (1)

where nactv and n

desv are actual and desired velocity of ‘nth’ vehicle respectively and ‘α’ and ‘δ’ are known prior. The calculation of actual and desired velocity will be explained in 3.2

When traffic is going to be synchronized, vehicles approach each other and driver tends to keep velocity dependent equilibrium distance given by

min2

,nnact

act

S v Tdvdt S

α+ ×

= − × (2)

where ‘ Smin’ is minimum distance between cars to avoid accident and ‘Sact’ is actual distance between cars and ‘T’ is safe time headway (sec). During congestion, driver behavior is governed by

2

2,( )

n nnact act

act

v vdvdt b S

×= −

× ×Δ (3)

where ‘b’ is desired declaration and ‘ nactvΔ ’ is velocity

of ‘nth’ car with respect to leading car. The generalized behavior of driver on roads is sum of all three above behaviors [6], [9], i.e.

m in

2

[1 ( ) [(

) / ] ].2

n nnactactn

des

n nact act

act

gdv vS v T

dt v

v vS

b

δα

α

= × − − + ×

Δ+

× ×

(4)

It is important to mention here that models like ‘social force’ are inappropriate to capture traffic dynamics. This is because of significance differences between traffic dynamics and crowd flow despite of many similarities [6].

3. Implementation

In this section, we present algorithmic details involved in carrying out abnormal traffic detection.

3.1. Motion Estimation Fast motion estimation is difficult problem to solve. In order to estimate motion, we employ phase correlation method and measure displacement between respective blocks, for every two frames, directly from their phases. We achieve spatial temporal average of flow field using separable convolution of 3D signal, i.e. flow field, with spatial gaussian kernel of variance ‘µs’ and temporal gaussian kernel of variance ‘µt’. 3.2. Particle advection The purpose of particle advection is to uncover the spatial organization of scene. We move grid of particles over spatial temporal average of optical flow calculated in last section. Note that in our formulation, we are assuming each particle to be representative of a car [11], [5], [9]. Trajectories of particles owing along velocity (flow) field are estimated using initial value problem method [2]. The equation formulation of particle advection method is given as follows

( ) ( )1 1 0 0 0 0 0( , ) , , , ,n n n n n nx y x y h f x y t= + × (5)

where 0 0( , )n nx y represents initial location of ‘nth’

particle, ‘h’ is step size and 0 0 0( , , )n nf x y t is the value of

velocity field at location0 0( , )n nx y at time t0 .

For each new time instant, we replace 0 0( , )n nx y by

1 1( , )n nx y in above equation and repeat the process. This iterative process continues for each particle over whole video. After obtaining trajectories of all particles over whole video, we calculated resultant velocity of particles at each location along their path using spatial temporal averaged optical flow vectors. Since velocity of particles calculated in this step is dependent on velocities of neighboring particles, the situation is similar to motion of cars on road, whose velocity is highly dependent on the velocities of neighboring cars. Velocity calculated in this step is named as ‘actual velocities’. In next step, we calculated resultant velocity of particles at each location using actual optical flow vectors. The velocity calculated in this step is named as ‘desired velocity’ since it is not dependent on the velocities of neighboring particles. 3.3. Intelligent Driver Behavior Calculation Since in our formulation, we are treating each particle as a car, so we estimate intelligent driver behavior for each car using (4). These behaviors are calculated at each location along the path using desired and actual velocities calculated in previous section. To

325325325325325

calculate relative velocity of particle at time‘t’, we subtract velocity of particle at time ‘t-1’ from velocity of particle at time ‘t’ i.e.

, , , 1.n n nact t act t act tv v v −Δ = − (6)

Other parameter values are chosen from [6]. These behaviors are representations of driver attitude in safe, stable, and accident free situation, i.e. normal traffic [9]. 3.4. Learning of Intelligent Driver Behaviors In this work, we model the given traffic scene using latent dirichlet allocation (LDA) [8] which is adopted from text processing literature. We extract M×N×F patterns of intelligent behaviors from the video sequence where M×N is the patch size and ‘T’ is the number of frames in one document. We learn vocabulary by K-mean clustering. The center of each cluster is defined as a word. The notion of documents is generated by dividing each video sequence in short clips. Thus, each detected M×N×F pattern is assigned a unique membership, such that video can be represented as collection of words from the vocabulary. We use a collection of normal traffic sequences and adopt the variational expectation minimization (EM) algorithm as proposed in [8]. During testing, frames are classified as abnormal if their likelihood is less than certain threshold value. We refer reader to [8] for more details on LDA. 3.5. Region of Abnormality In order to detect region of abnormality in scene, we utilize intelligent driver behaviors calculated in section 3.3. We adopted technique similar to [2]. Let ‘BX’ and ‘BY’ represent behaviors of particles along ‘x’ and ‘y’ direction respectively. Using finite differencing approach, we calculated spatial gradient of behaviors as, dBx/dx, dBx/dy, dBy/dx, dBy/dy. These gradients are used to compute Cauchy Green Deformation tensor. This tensor quantifies the amount by which neighboring particles have different behavior values. The maximum eigen value of this tensor is used to construct Finite Time Lyapunov Exponent (FTLE) field [10]. FTLE field, Ω, with a finite integration time, T, corresponding to a point ‘(x,y)’ at time t0 is given as

0 max

1ln ( ) ,T

t TλΩ = Δ (7)

where λmax(∆) represents maximum eigen value of Cauchy Green Deformation tensor, ∆. The middle column of figure 3 shows the plot result of the value calculated by (7). Finally, FTLE field is segmented using watershed segmentation algorithm.

4. Experiments and Discussion

In this section, we present our results for abnormality detection and localization on traffic videos taken from stock footage web sites.

4.1 Abnormality Detection

In our experiments, we have used presence of

unusual driver behavior as a measure of abnormality. For this purpose, we used normal traffic sequences and extracted intelligent driver behaviors. All frames were resized to 240×160.The resolution of particle was kept equal to frame size i.e. 240×160. For particle advection, we used step size ‘h’ of .1245 and measured actual and desired velocities of particles using bilinear interpolation. Intelligent driver behaviors (IDBs) over the video were calculated using the method described in section 3.3. Note that the IDM has well defined parameters that are easy to calibrate. We use the parameter values as suggested in [6].

In learning phase, we extracted every 5×5×10 pattern of driver behaviors values. We termed this pattern as a ‘word’. We learned vocabulary using thirty numbers of clusters. Documents size was kept at 10 frames. We discovered 20 latent topics and estimated parameters of our model using variation expectation maximization [8].

In testing phase, we calculated the likelihood of each document using parameters learned in learning phase. Documents with very low likelihood were classified as abnormal. Abnormality detection results for four different traffic scenarios are shown in figure 2.

4.2 Abnormality Localization

In order to localize region of abnormal behaviors, we calculated spatial gradients of behaviors as discussed in section 3.5. Note that high gradients are present at the locations where neighboring particles have quite different behavior values over length of integration ‘T’ (number of frames). FTLE field is constructed using (7). Finally, region of abnormality is segmented using watershed algorithm as shown in figure 3. Note that this kind of abnormality localization cannot be achieved without intelligent driver model. To evaluate this, instead of using spatial gradients of behaviors, we use spatial gradients of particle trajectories to construct FTLE field. FTLE field is, then, segmented using watershed algorithm with same parameter settings. Our results show that proposed

326326326326326

X. Wang, X. Ma, and E. Grimson. Unsupervised activity perception by hierarchical bayesian models. CVPR, 2007.

I. Saleemi, K. Shafique, and M. Shah. Probabilistic modeling of scene dynamics for applications in video surveillance. PAMI, Aug, 2009.

Dirk Helbing. Traffic and related self-driven many- particle system. Reviews of Modern physics, volume 11, 2001.

S. C. Shadden et. al. Defination and Properties of Lagrangian Coherent Structure from Finite Time Lyapunov Exponents in Two Dimensional Aperoidic Flows. Physica D, 212, 2005.

Total Frames=160

(i) (ii)(a) (b)

(i) (ii)

Total Frames=220

(a) (b)

(a) (i) (ii)

Total Frames=160

(b) Total Frames=160

(i) (ii)(a) (b) Figure 2. Detection results for four different traffic sequences. (a) Normal traffic, (b) abnormal traffic. (i) Shows the detection results along the video sequence. (ii) Ground truth for each video sequence. Green area shows normal frames and red area shows the abnormal ones.

(a)

(b)

Figure 3. Comparative results for abnormality localization with (a) Intelligent driver model and (b) without intelligent driver model. First column represent video sequence of length 200 frames. Middle column shows FTLE field computed in both cases. Third column shows final segmented region.

algorithm localize region of abnormality more accurately compare to techniques that use only optical flow or particle advection as a low level feature. Typical example is shown in figure 3. 5. Conclusion

In this paper, we have presented an abnormality

detection and localization algorithm for traffic scenes. We used interaction among cars as a main clue for abnormality detection. Our method captures the dynamics of traffic scene with high accuracy. In addition, we have shown that our method accurately localize region of abnormality, which cannot be achieved with methods based on only optical flow or particle advection.

. References [1] http://www.car.accidents.com/pages/stas.html. [2] S. Ali and M. Shah. A lagrangian particle dynamics

approach for crowd flow segmentation and stability analysis. CVPR, 2007.

[3] Timothy Hospedales, Shaogang Gong and Tao Xiang. A markov clustering topic model for mining behavior in video. ICCV, 2009.

[4] Weiming Hu, Xuejuan Xiao, Zhouyu Fu, Dan Xie, Tieniu Tan and Steve Maybank. A system for learning statistical motion patterns. PAMI, 2006.

[5] Ramin Mehran, Alexis Oyama and M. Shah. Abnormal crowd behavior detection using social force model. CVPR, 2009.

[6] D. Helbing and Benno Tilch. Generalized force model of traffic dynamics. Physical review E, 58:1, 1998.

[7] Wang Dahui, Wei Ziqiang, and Fan Ying. Hysteresis phenomena of the intelligent driver model for traffic flow. Physical review E, 76:016105, 2007.

[8] David M. Blei, Andrew Y. Ng and Michael I. Jordan. Latent dirichlet allocation. Journal of Machine Learning Research, 3:993-1022, 2003.

[9] Martin Treiber, Ansgar Hennecke and Dirk Helbing. Congested traffic states in empirical observations and microscopic simulations. Physica review E, 62:2, 2000.

[10] [11] [12] [13]

327327327327327