View
220
Download
2
Tags:
Embed Size (px)
Citation preview
Content-Based Image Indexing
Joel Ponianto
Supervisor:
Dr. Sid Ray
Outline
Introduction to Content-Based Image Indexing Image’s Features Extraction Tree Structure System Model Retrieval Approach Experiment Results Conclusion
Introduction to Content-Based Indexing
Content-Based Image Indexing (CBII) is an interrelated issue with Content-Based Image Retrieval (CBIR).
CBIR depends on CBII and vice versa. CBIR focus on how to retrieve image
accurately and efficiently. While CBII concern with how to support
retrieval process.
Introduction to Content-Based Image Indexing Cont…
CBiI as pre-process of CBIR sequences. Cannot ignore retrieval process to create good
indexing structure. The idea of indexing is similar with a library Every book has a unique id Every book has properties
Introduction to Content-Based Image Indexing Cont…
Examples: title, author, publisher, etc Those properties are used to search the book. People know it as “keyword” Similar idea with images, however not that
simple. Cannot represent an image with simple text.
(can but not make sense)
Introduction to Content-Based Image Indexing Cont…
How to represent an image? By using its properties such as, colour, shape,
texture and others. Choose which properties need to be extracted
for indexing purpose ( and also retrieval). Also choose which method to extract those
properties / features.
Image’s Features Extraction Cont…
Colour, shape and texture have their own sub-features.
Colour: grey level, RGB/HUE value, grey sigma, local histogram and average colour value.
Shape: area, centroid, circularity and moment invariant.
Texture: contrast, orientation and anisotropy.
Image’s Features Extraction Cont…
The selection of features is also effected by the data set.
what we want to achieve at the retrieval stage is effected by the data set.
If the data set is full of houses’ image and a user want to look for a car image.
Try to select features that can differentiate each class in the data set.
Image’s Features Extraction Cont…
For this project I select the following features:– Colour Sigma (Global)– Edge density (Global)– Colour Average (Global)– Boolean edge Density (Global)– Edge Direction (Global)– Region area (Region)– Moment invariant (Region)– Grey level (Region)
Image’s Features Extraction Cont…
Colour Sigma– Find the standard deviation (σ) of the image, for
each colour layer.
Image’s Features Extraction Cont…
Edge Density– Enhance the pixels that belong to the edges and
boundaries by using a standard edge detector. Pixels far from edges will drop to 0 and those near to an edge will increase to max. calculate the mean pixel value of the resultant image.
Colour Average– Sum all the pixel value for each colour layer and
divide by the number of pixel.
Image’s Features Extraction Cont…
Boolean Edge Density– From above edge density, the image is thresholded
so that what could be called edge pixels are white (1) and non-edge pixels are black (0). Count white pixel in the image.
Edge Direction– With some edge detection (Sobel Operator), allow
us to make a crude estimation of a edge direction for particular region.
Image’s Features Extraction Cont…
Area, Grey Value and moment invariant– These features is calculate on regional basis.– The region is calculated with combination of “k-
mean clustering” and “Connected Component labelling Algorithm”
– Calculate a grey level value of an image and perform the k-mean clustering.
– Use the connectivity algorithm to group similar grey value by its location.
Image’s Features Extraction Cont…
http://www.cis.rit.edu/class/simg782.old/talkMoments/momentEquations.html
I use the first four of seven invariant moment for this project.
Image’s Features Extraction Cont…
Image’s Features Extraction Cont…
Quantisation – To be suitable for computer processing and features
extraction (colour), an image must be digitized in amplitude.
– The idea is to reduce the colour space while gaining the ability to localize colour information spatially.
– this project applies quantisation at HSV colour space.
Image’s Features Extraction Cont…
Image’s Features Extraction Cont…
RGB to HSV– Let RGB values ranged from 0 to 1 and MIN/MAX corresponds with
RGB values.
Image’s Features Extraction Cont…
HSV to RGB– H range from 0 - 360– V and S range from 0 – 1– If S == 0 then RGB = V
Else use next formula
Image’s Features Extraction Cont…
Image’s Features Extraction Cont…
Tree Structure
There are many choices of tree structures that can handle multi-dimensional space. Such as R-Tree, R*-Tree and Vp-Tree
We look at R-Tree tree structure:– This project used R-Tree to simplify the
computation.– Other tree structures can be use on the system.
Tree Structure Cont…
R-Tree (Antonin Guttman)– A R-Tree is a height balance tree and all leaves are on the same
level. – Root node has at least two children unless it is the leaf node. – Every non-leaf node contains between m and M entries unless it is
the root. – For each entries (I, childnode-pointer) in a non-leaf node, I is the
smallest rectangle that spatially contains all rectangles in its child nodes.
– Every leaf node contains between m and M index records unless it is the root.
– For each index record (I, tuple-identifier) in a leaf node, I is the smallest rectangle that spatially contains the n-dimensional data object represented by the indicated tuple.
Tree Structure Cont…
Tree Structure Cont…
System Model
Put into data base
Original Image
Quantised Image
K-mean clustering
Binary Threshold Apply Global features extraction.
Connected Component labelling
Apply Region features extraction. Insert into tree
structure
System Model Cont…
The System input around 300 images into the data base.
Those images is divided into 10 different classes: animal, car, flower, face, fruit, house, lake, mountain, plane and sunset.
Store into persistence storage.
System Model Cont…
In the “binary threshold” stage, I attempt to separate the background image with the object.
Although this stage is very weak, but in some images. The result can be helpful (and possible the other way around).
System Model Cont…
Binary Threshold good result
System Model Cont…
Binary Threshold bad result
Retrieval Approach
Query sequence
Query Image
Global Extraction
Region Extraction
Pre-process stage
Find similarity with data set
Display the result in ascending order
Retrieval Approach Cont…
For finding similarity, I use Euclidean distance measure formula:
Where:– p is the database image– q is the query image– Pi is the database images ith features
– Qi is the query’s ith features
– n is the number of features– W” is the weight for ith feature
W”i
Retrieval Approach Cont…
w’i is the weight of feature i
from relevant images
(σi) is the standard deviation of
feature i from relevant images w’t is the total weight of feature I
w”t is the normalised weight
Retrieval Approach Cont…
Gaussian Normalisation (for feature normalization):– d’(fi,fj) is the similarity of image fi and fj,
range in [-1, 1]
– σij and μij are the standard deviation and
mean of each feature respectively.– d”(fi,fj) is to make d’(fi,fj) in range [0, 1]
Experiment Result
Go to Excel file m1-m8 only use global features m3 uses colour avg, colour sigma and edge
density m2 uses colour avg and colour sigma m8 uses colour sigma and edge density m9 use region features + m3
Conclusion
Indexing depend on retrieval and vice versa No universal system / method for indexing or
retrieval. We can try to develop something that robust. Indexing base on regional features give better
result then global features. With more time, more result can be produced.
Reference
• Kompatsiaris, I., Triantafillou, E. and Strintzis, M. G., “Region-Based Color Image Indexing and Retrieval”, 2001
• Parker, J. R., Behm, B., “Use of Multiple Algorithm in Image Content Searches”, International Conference on Information Technology: Coding and Computing (ITCC’04) Volume2 p.246.
• Smith, J. R., Chang, S., “Single Color Extraction and Image Query”, International Conference on Image Processing (ICIP-95), Washington, DC, Oct, 1995.
• Park, J. M., Looney, C. G., Chen, H. C., ”Fast Connected Component Labeling Algorithm Using A Divide and Conquer Technique”, Technical Report, 2000
• Chiueh, T., "Content-Based Image Indexing," in Proceedings of International Very Large DataBase Conference, VLDB '94, Santiago, Chile, September, 1994.
• Gonzalez, R. C. and Woods, R. E., “Digital Image Processing”, 1993, Addison-Wesley Publishing Company, inc, 3rd edition.