Lane Detection Using SVM

By Solomon Genene Gudeta

Mechatronics Engineering

Outline

Introduction and literature review

Objectives

Road surface extraction

Lane modeling and SVM regression

Implementation and result analysis

Future works and recommendation

University of Trento 2

Introduction and literature review

Lane detection is a system which is used to detect lanes on a road.

It uses either : Active sensors or Passive sensors

It has wide application in intelligent transportation systems, remote sensing,…etc.

There are various researches conducted on the area.

• Feature based technique: requires road to have well painted markings

• Model based technique: it is robust but lacks flexibility

Lane detection has to have the following properties:

• Shouldn’t be affected by shadows

• Have to process both painted and unpainted roads

• Must handle both curved and straight roads

• Should process fast and robustly

University of Trento 3

Objectives Lane detection using vision based sensors is not easy.

There are two generic problems associated with it.

• Quality of the sensor and their orientation: Ground based and Aerial.

• The surrounding scenario (environment): Brightness and Property of road

Project has two important parts (objectives).

• Road surface extraction using color based segmentation

• Regression of lane pixels to form lane model.

The extraction, classification and regression is carried out by support vector

machine using pixel information

University of Trento 4

Image pre processing

The input image is a true color image(RGB color space) of size (MxNx3)

It must be converted to YCbCr color space (MxNx3) as the blue component in

YCbCr is better in identifying the env’t from road surface

The color based segmentation uses Y(MxN), Cb (MxN), Cr (MxN) of YCbCr

color space as features.

Training data is prepared by selecting a region of road pixels (lane pixels) and

then the unique ones are used to avoid redundancy.

The training data matrix of unique pixels can be formed by converting Y, Cb, Cr

to matrix of (NMx1) and concatenating the matrices.

For road surface extraction, one class SVM is deployed.

Finally, using SV regression the generic lane model can be determined

University of Trento 5

University of Trento 6

Input image Noise/Shadow

removal

Road extraction

Lane extraction

Morphology

operation Road detection

Lane model

Lane

Road surface extraction On the road there are various entities and road lane.

Road surface has road pixels and lane pixels which are characterized by different

features values

Road pixels have related property wherever. So does lane pixels.

Road surface extraction is needed in derive the lane model: by image coordinate

transformation and support vector regression.

It has two parts lane pixel classification and road pixel classification

The classification is done by one class support vector classifier as:

Information other than road pixel and lane pixel is difficult to find(or imbalanced)

To avoid the under sampling of the other class (non road and non lane pixels)

In both cases RBF kernel is used to map the data from input space to feature space

University of Trento 7

Road surface extraction

Road surface extraction is the task of directly obtaining the boundary around a target data

set (lane and road pixels)

The boundary should include as much as possible target and should minimize the chance of

accepting non targets.

It is similar to defining a hyper sphere around a target class

The sphere is characterized by a center a and radius R, it has to contain all training objects

To make it more robust; the distance greater than R has to be penalized.

Introducing slack variables the minimization problem

𝜀 𝑅, 𝑎, 𝜉 = 𝑅2 + 𝐶 𝜉𝑖𝑖

𝑥𝑖 − 𝑎2 ≤ 𝑅2 + 𝜉𝑖

𝜉𝑖 > 0 ∀𝑖

𝑥𝑖 𝜖 𝑅𝑛 is a training data.

University of Trento 8

Road surface extraction The parameter C gives the tradeoff between the volume of the description and the

errors.

The free parameters, a, R and, have to be optimized, taking the above constraints

into account.

𝐿 𝑅, 𝑎, 𝜉, 𝛼, 𝛾 = 𝑅2 + 𝐶 𝜉𝑖𝑖 + 𝛼( 𝑥𝑖 − 𝑎2 − 𝑅2 − 𝜉𝑖)𝑖 − 𝛾𝜉𝑖𝑖 , 𝛼𝑖 ≥ 0, 𝛾𝑖 ≥ 0

Differentiating and equating to zero it yields

𝛼𝑖 = 1, 0 ≤ 𝛼𝑖 ≤ 𝐶, 𝑎 = 𝛼𝑖𝑖

𝑥𝑖

The dual form will became

𝐿 𝑅, 𝑎, 𝜉, 𝛼, 𝛾 = (𝑥𝑖 . 𝑥𝑖)𝑖 − 𝛼𝑖𝛼𝑗(𝑥𝑖 . 𝑥𝑗)𝑖,𝑗 , 0 ≤ 𝛼𝑖 ≤ 𝐶, 𝛼𝑖 = 1

University of Trento 9

Road surface extraction

Using quadratic programming 𝛼𝑖 can be obtained

If 𝛼𝑖 = 0 the pixel is in the sphere

If 𝛼𝑖>0 and 𝛼𝑖 < C, the pixel is support vector and it is on the surface of sphere.

If 𝛼𝑖 = C the pixel is outlier i.e. it is outside the boundary

The decision function 𝑓 𝑥 = 𝑠𝑖𝑔𝑛(𝑅2 − 𝑥𝑖 − 𝑎 2) is used to classify pixels

The radius R is determined from any one of support vectors

𝑅2 = 𝑥𝑘 . 𝑥𝑘 − 2 𝛼𝑖 𝑥𝑖 . 𝑥𝑘𝑖

+ 𝛼𝑖𝛼𝑗(𝑥𝑖 . 𝑥𝑗)

𝑖,𝑗

University of Trento 10

Road surface extraction To test a pixel (𝑥𝑚), 𝑥𝑚 − 𝑎

2 ≤ 𝑅2 where 𝑎 is center of sphere or using decision function

𝑓 𝑥 = 𝑠𝑖𝑔𝑛(𝑅2 − 𝑥𝑚. 𝑥𝑚 − 2 𝛼𝑖 𝑥𝑖 . 𝑥𝑚𝑖

+ 𝛼𝑖𝛼𝑗(𝑥𝑖 . 𝑥𝑗)

𝑖,𝑗

)

To make the boundary description flexible the data has to be mapped to newer representation

This can be done using kernel function e.g. Gaussian kernel

The fact that 𝑘 𝑥𝑖 . 𝑥𝑖 =1 means the mapped object has norm equal to 1.

The new Lagriangian to be maximized will be(ignoring constant)

𝐿 𝛼 = − 𝛼𝑖𝛼𝑗(𝑥𝑖 . 𝑥𝑗)𝑖,𝑗 , 0 ≤ 𝛼𝑖 ≤ 𝐶, 𝛼𝑖 = 1

The decision function becomes

𝑓 𝑥 = 𝑠𝑖𝑔𝑛 𝛼𝑖𝑘 𝑥𝑚. 𝑥𝑖𝑖

− 0.5 𝐵 − 𝑅2

Where 𝐵 = 1 + 𝛼𝑖𝛼𝑗(𝑥𝑖 . 𝑥𝑗)𝑖,𝑗

p = 0.5 𝐵 − 𝑅2

University of Trento 11

Algorithm development for training and testing Prepare training and test data

– Extract road pixels(lane pixels)

Save training and test data

Load training and test data

Scale data between 0 and 1

Select kernel

Prepare kernel matrix

– Select sigma

Map data to feature space

Select penalty parameter

Solve quadratic programming

Find support vectors and save alpha

Do cross validation

Find and save p

Go to prepare kernel else end training.

University of Trento 12

Algorithm development for pixel classification Read image

Load train data

Load alpha, p and sigma

Scale image and train data

Transform image and train data to feature space

Initialize matrix to size of image

Evaluate decision function

Store result in image

Display image

University of Trento 13

Image morphology operation Morphology is a broad set of image processing operations that process images based

on shapes.

After performing road and lane pixel classification, the resulting images is added to

form the road surface area

There exists some noise pixels that are classified wrongly and similarly some pixels on

a road which are misclassified

The noise pixels are removed by erosion while the misclassified pixels of a road are

compensated by dilation.

Dilation adds pixels to the boundaries of objects in an image based on on the size and

shape of the structuring element used to process the image

In addition to this there are some groups of pixels(noise pixels) which exists, they are

removed using bwareaopen function in the tool box of matlab

Finally the resulting image is multiplied with original input image to discard other

pixels except those on a road.

University of Trento 14

Lane modeling and SVM regression In order to handle painted and un painted roads the lane model based technique

is used

Lane modeling uses the extracted road surface to derive a model for a lane(the

model is supposed to be flexible)

The image has to converted from image coordinate system to world coordinate

system supposing camera is mounted on vehicle i.e. to find real shape of road

This requires the knowledge of parameters of camera used

The origin point of world coordinate system(WCS) is center of camera lens.

Image coordinate system(ICS)’s origin is at the center of image

WCS can be transformed to ICS by perspective formula shown below

𝑥 = 𝑓𝑋𝑛𝑍𝑛 cos 𝑤−𝑌𝑛 sin 𝑤

, 𝑦 = 𝑓𝑦𝑛 cos 𝑤+𝑓𝑧𝑛 sin 𝑤𝑍𝑛 cos 𝑤−𝑌𝑛 sin 𝑤

University of Trento 15

Lane modeling and SVM regression

Partition the road into horizontal sections (using built in function blockproc)

The center of each section is detected and used as a training data for support vector

regression

Support vector regression optimizes (the linear one norm form)

min 0.5 𝑢𝑖∗ − 𝑢𝑖 𝑢𝑗

∗ − 𝑢𝑗 𝜙 𝑥𝑖𝑇

𝑚

𝑗=1

. 𝜙 𝑥𝑖 + 𝜀 𝑢𝑖∗ + 𝑢𝑖 − 𝑦𝑖(𝑢𝑖

∗ − 𝑢𝑖)

𝑚

𝑖=1

𝑚

𝑖=1

𝑚

𝑖=1

Subject to 0 ≤ 𝑢𝑖 , 𝑢𝑖∗ ≤ 𝐶

Selecting the Gaussian kernel and tuning its parameters the decision function

𝑓 𝑥 = (𝑢𝑖∗−𝑢𝑖)𝐾(𝑥, 𝑥𝑖)

𝑚

𝑖=1

+ 𝑏,

where bias b is computed from 𝑓 𝑥𝑖 = 𝑦𝑖 − 𝜀

The decision function is supposed to be a model of lane and it has flexibility University of Trento 16

Implementation

• The image on which I did implementation

University of Trento 17

Implementation

University of Trento 18

Implementation

University of Trento 19

Implementation

University of Trento 20

Implementation

University of Trento 21

Future works and recommendation

The developed system handles both painted and unpainted roads

Because of the weakness of vision sensor itself and the complexity of road (degraded

vs. good roads)environment it may be difficult to generalize the model derived from

one image to another

This can be handled by online training i.e. classifying one image correctly

The other complex problem is a shadow. Shadow has to be removed by using efficient

algorithms

Adopting different scenarios (day scenario and night scenario) may as well improve

the performance of the system.

The error(false alarm) in the classification of lane pixels can be reduced by adding

additional lane features(like eccentricity of lane)

University of Trento 22

THE END

THANK YOU