11

A Fast Algorithm for License Plate Detection (LPD(

Prof. Mohy E. Abou El-Soud, Dr. Mohamed Abdel-Azim, and Eng. Amr E. Rashid

Faculty of Engineering- Mansoura University, Mansoura, Egypt

22

Outlines IntroductionIntroduction Motivations.Motivations. Constraints and Data CollectionConstraints and Data Collection Problem DefinitionProblem Definition Previous WorkPrevious Work The Proposed TechniqueThe Proposed Technique Results and ConclusionResults and Conclusion Future WorkFuture Work

33

Introduction Monitoring vehicles for law enforcement and

security purposes is a difficult problem because of the number of automobiles on the road today.

An example is this lies in border patrol : It is t ime consuming for an officer to physically check

the license plate of every car.Additionally, it is not feasible to employ a number of

police officers to act as full-time license plate inspectors.Police patrols cannot just drive in their cars staring

at the plates of other cars.There must exist a way for detecting and identifying

license plates without constant human intervention.As a solution, we have implemented a system that can

extract the l icense plate number of a vehicle from an image given a set of constraints.

44

Introduction (Cont ’d(

In any object recognition system, there are two major problems that need to be solvedDetecting an object in a scene, andRecognizing that object.

In our system, the quality of the l icense plate detector (LPD) is doubly important since the make and model recognition (MMR) subsystem uses the location of the license plate as a reference point when querying the car database.

55

License Plate Recognition (LPR) may also be referenced as:Automatic Vehicle Identification (AVI).Car Plate Recognition (CPR).Automatic Number Plate Recognition

(ANPR).Car Plate Reader (CPR) Optical Character Recognition (OCR) for

Cars.

Introduction (Cont ’d(

66

MotivationsThis area is challenging because it requires

an integration of many computer vision problem solvers, which include:Object detection (LPD).Character recognit ion (OCR).

LPR is very important in:Private transport applications.Monitoring vehicles for law enforcement and

security purposes is a difficult problem because of the number of automobiles on the road today

77

WhatWhat ’’s LPRs LPR??

License Plate Recognition:License Plate Recognition:LPR is an image-processing based-LPR is an image-processing based-

technology used to identify vehicles technology used to identify vehicles by their license plates.by their license plates.

This technology is used in various This technology is used in various security and traffic applications.security and traffic applications.

88

Importance of LPR

99

LPR is one of the most important types of intelligent transport system and is of considerable interest because of its potential applications to many areas such as:highway electronic toll collect ion, traff ic monitoring systems and . . .

The technology concept assumes that all vehicles already have the identity displayed (the plate!) so no additional transmitter or responder is required to be installed on the car.

Technology HighlightsTechnology Highlights

1010

Problem Definit ionProblem Definit ionLicense plates come in:

Different sizes,Different Width-Height ratios,Different color,The fonts used for digits on license plates are not

the same for all license plates, These problems, and the changing weather

conditions, are what make the field of LPR a good candidate for testing Pattern Recognition techniques.

1111

Constraints… Use a digital camera, Image of the vehicle taken with variable

angles, Image of the vehicle taken from fixed

distance (about 1-2 m), Vehicle is stationary when the image

was taken, Only Egyptian l icense plates will be

processed.

1212

Data collectionData collection All images of vehicles database were taken with a

benq digital camera, with three different resolutions: (i) 3 M-Pixels, (ii) 4 M-Pixels, and (iii) 5 M-Pixels.

On average, the images were taken (1-2m) away from the vehicle.

They were stored in color JPEG format on the camera.

The colored JPEG images were converted into gray scale raw format on the PC.

There 30 images dataset.

1313

Data Collection (ContData Collection (Cont ’’dd((

Original Color Image

1414

Data Collection (ContData Collection (Cont ’’dd((

Gray Scale Image

1515

Previous WorkPrevious WorkThe current LPD techniques can be

classified into four main algorithms:Corner template matching,Hough transforms combined with

various histograms based methods,color based filter, andVertical edge detection followed by

size and shape filtering

1616

Algorithm-1: Vertical Edge Detection

Candidate selection:Candidate selection:Histogram equalization,Histogram equalization,Binarization,Binarization,Sobel edge detection, and Sobel edge detection, and List possible licensesList possible licenses

For each candidate:For each candidate:Localized histogram, Localized histogram, Binarization, and Binarization, and Elimination by 2-D correlation Elimination by 2-D correlation

16

1717

Algorithm-2: Algorithm-2: Vert ical Edge Detection

This algorithm used a recognition algorithm based on width to height ratio:Vertical edge detection,Size and shape filtering,Vertical edge matching, andCompute the Black to white ratio and then

perform plate extraction.

1818

DrawbacksThere are many problems in these two

algorithms:Width to height ratio differs from a car to another

depending on the distance between the camera and the car,

Small vertical edges will difficult the recognition problem because it change the width between edges,

When we use different view this will remove desired vertical edges, and

There are many objects in the image achieves equal width to height ratio

1919

Drawbacks (Cont ’d)

2020

Algorithm-3: AdaBoost Algorithm Algorithm-3: AdaBoost Algorithm Since license plates contain a form of text, we

decided to face the detection task as a text extraction problem.

Window search over the entire frame.Use three different sized windows. Independent Classifier for Each SizeStrong Classifier Constructed from WeakClassifiers Via AdaBoost algorithm .Computationally Simple.Draw backs: Regions contain character except

license plate.

2121

Drawbacks of AdaBoost Algorithm

2222

The Proposed Algorithm

The proposed algorithm was divided into four main parts:

Histogram Equalization, Removal of Border and Background, Image Segmentation, and License Plate Detection.

2323

A. Histogram Equalization

Is an image transformation that computes a histogram of every intensity level in a given image and stretches it to obtain a more sparse range of intensities.

This manipulation yields an image with higher Contrast than the original.

2424

Original image

Remove this

partition

Remove this partition

2525

The Output Image of Histogram Equalization

2626

B. Removal of Border and Background

Sobel Vertical edges

2727

B. After Removing Small B. After Removing Small ElementsElements

2828

Sobel Horizontal Edges

2929

Horizontal Edges After Removing Small Elements

3030

Car after removing border and back ground

3131

C. Image Segmentation

Often the license plate will be in the lower Often the license plate will be in the lower half of the image so we will remove upper half of the image so we will remove upper half of the image.half of the image.

3232

D. License Plate Detection

Feature extraction.Feature extraction.Principal component analysis.Principal component analysis.Artificial neural networks.Artificial neural networks.

3333

D.1 Feature Extraction

Feature extraction is the transformation of the original data (using all variables) to a dataset with a reduced number of variables.

In the problem of feature selection, the aim is to select those variables that contain the most discriminatory information.

3434

D.1 Feature Extraction (Cont ’d(There are several reasons for performing

feature extraction:To reduce the bandwidth of the input data.To provide a relevant set of features for a

classifier.To reduce redundancy.To recover new meaningful underlying

variables or features that the data may easily be viewed.

3535

D.1 Feature Extraction (Cont ’d(Wavelets have been demonstrated to give

quality representations of images.This DWT representation can be thought of

as a form of “feature extraction” on the original image

We will use Haar-like features, where sums of pixel intensities are computed over rectangular sub-windows.

3636

D.2 Principal component analysis (PCA(

In some situations, the dimension of the input vector is large, but the components of the vectors are highly correlated (redundant).

It is useful in this situation to reduce the dimension of the input vectors.

An effective procedure for performing this operation is PCA.

3737

D.3 Recognition Stage Using D.3 Recognition Stage Using ANNsANNs

3838

D.3 Recognition Stage Using D.3 Recognition Stage Using ANNsANNs

Apply ANN with adaptive sub-window:Apply ANN with adaptive sub-window:

See final outputSee final output

Second techniqueSecond technique

(A) Image enhancement (A) Image enhancement (B) Removal of Border and Background (B) Removal of Border and Background (C) Image Segmentation (C) Image Segmentation (D) License Plate Detection using 2D (D) License Plate Detection using 2D

correlation.correlation.

3939

Data collectionData collection

4040

Data collectionData collection

4141

Image enhancement(Wiener filter outputImage enhancement(Wiener filter output((

4242

Removal of boarder and backgroundRemoval of boarder and background

4343

Cont’dCont’d

4444

After removing small elementsAfter removing small elements

4545

Horizontal edgesHorizontal edges

4646

Image segmentationImage segmentation

4747

Vertical edge detectionVertical edge detection It is observed that most of vehicles usually It is observed that most of vehicles usually

have more horizontal lines than vertical have more horizontal lines than vertical lines. To reduce the size of the image lines. To reduce the size of the image vertical edges are detected.vertical edges are detected.

this help in extracting the license plate this help in extracting the license plate exactly from segmented image, even it is exactly from segmented image, even it is out of shape, out of shape,

4848

Image after applying vertical edge detectionImage after applying vertical edge detection

4949

Cont’dCont’d

5050

Matching by correlationMatching by correlation The correlation problem is to find all places in the image The correlation problem is to find all places in the image

that match a given sub image (also called a mask or that match a given sub image (also called a mask or template) template)

Typically mask image is much smaller thanTypically mask image is much smaller thanOriginal imageOriginal image One approach for finding matches is to treat mask image One approach for finding matches is to treat mask image

as spatial filter and compute the sum of products (or a as spatial filter and compute the sum of products (or a normalized version of it) for each location of mask image normalized version of it) for each location of mask image in . Then the best match (matches) of subimage in in . Then the best match (matches) of subimage in original image is (are) the location(s) of the maximum original image is (are) the location(s) of the maximum value(s) in the resulting correlation image. value(s) in the resulting correlation image.

5151

Cont’dCont’dFor prototyping. An alternative approach is For prototyping. An alternative approach is

to implement correlation in the frequency to implement correlation in the frequency domain.domain.

Making use of the correlation theorem Making use of the correlation theorem Which like the convolution theorem.Which like the convolution theorem.

Relates spatial correlation to the product of Relates spatial correlation to the product of the image transforms.the image transforms.

5252

Final outputFinal output

5353

5454

Conclusion & ResultsFinally, we have built an LPD system that is:Finally, we have built an LPD system that is:

Real-time,Real-time,Works well with inexpensive cameras, andWorks well with inexpensive cameras, andDoes not require infrared lighting or sensors as Does not require infrared lighting or sensors as

are normally used in commercial LPR systems.are normally used in commercial LPR systems.There no database for Egyptian license plate There no database for Egyptian license plate

and there is no standard license plate in and there is no standard license plate in Egypt.Egypt.

We achieved 93.33% detection rate for small We achieved 93.33% detection rate for small dataset; i.e., 28 license plate of 30.dataset; i.e., 28 license plate of 30.

5555

Future work

Modern FPGA platforms provide the Modern FPGA platforms provide the hardware and software infrastructure for hardware and software infrastructure for building a bus-based system on chip building a bus-based system on chip (SoC) that meet the applications (SoC) that meet the applications requirements.requirements.

In order to accelerate the system we can In order to accelerate the system we can implement ANN classifier using FPGA with implement ANN classifier using FPGA with parallel processing instead of using Matlab parallel processing instead of using Matlab .we expect that we can achieve an overall .we expect that we can achieve an overall LPR system speed up.LPR system speed up.

5656

Any Questions?