Multimedia Indexing and Retrieval

Kowshik

Shashank

Project Advisor: Dr. C.V. Jawahar

Problem Statement

“Develop efficient algorithms for a real time, private multimedia Develop efficient algorithms for a real time, private multimedia databasedatabase”

Applications

Defense systems

Surveillance systems

Image/Video collections (under copyright notices)

Web 2.0

Web image search

FeatureExtraction Indexing

QueryFeature

Extraction

Database

SimilarityMeasure

Result

Indexing Schemes

Hierarchical Structures

Vocabulary Trees

Hashing

Private Retrieval In Hierarchical Structures

Querying in CBIR

……..

Feature vector

Query Image

Private Content Based Image Retrieval1. The user extracts the feature vector of the query image, say fquery.

2. The user asks for the data stored in the root node of the indexing structure.

3. fquery and the information are used to decide whether to access the left or the right sub-tree.

4. The user frames a Query Qi to access the node at level i.

5. The database replies with Ai for the query Qi .

6. The user performs a function f( Ai ) to obtain the information at the node.

Go to step 3.

Private Content Based Image Retrieval

A2Q2

Q1 A1

Feature vector (fquery)

……..

Root Info

fquery, f(A1)

fquery, f(A2)

Quadratic Residuosity assumption Consider a natural number N = p. q where p, q are large prime

numbers.

Construct a set

`y` is called a Quadratic Residue (QR), if x | y = x2 and x, y else `y` is called a Quadratic Non-Residue (QNR).

Construct a set YN with equal number of QRs and QNRs

.1,gcd,1|* xNNxxzN

*NZ

*NZ

Quadratic Residuosity Assumption:

Given a number `y` YN, it is predictably hard to decide whether `y` is a QR or a QNR.

Basic Rules

QNR * QNR = QRQNR * QR = QNRQR * QR = QR

Viewing the nodes in a level

QR QR QNR QR

User

ith1 0 1 0

Database

QR2 QR QNR2 QR

User

ith1 0 1 0

Database

Now the user decides on the ith element as QR or QNR and decides upon the data at the ith index in the database.

Q1 Q2 Q3 Q4

Q12 Q2 Q32 Q4

Querying on a Linear Database

A[i] = Q[i] if 0

A[i] = Q[i]2 if 1

Q

A

Converting to 2D database

QNR

QR

…..

….

….

QR

QNR

m x n

Frame a query of length ‘m’ with a QNR in the position of the row in which the node occurs

QR

…..

….

…..

….

The database forms a m x n matrix with the first bit of information

QR

00

1

0

1 1

1 1 QR

…..

….

….

QNR

m x n

0

…..

1

0

….

…..

….

QRQR2

QNR

QR2 QR2

QR2 QR2 QR

…..

….

….

QNR

m x n

Put the square of the number if the bit value is 1 else retain the same number

QNR

…..

QNR2

QR

….

…..

Multiply along the columns

QNR QNRQR …..Ai

….

Framing the Query and Reply

If the user is interested in the data at node (x,y)

Frame a query of length m in which the xth value is a QNR and rest are QR.

The database computes the reply Ai of length n and returns to the user.

If the value of Ai[y] is a QR then the value is 1 else 0.

Complexity of the algorithm

The communication complexity is O(m) on the user side and O(n) on the server side. Hence the communication complexity is O(max(m,n))

If m = n = , the communication complexity isi2

Extension to other Hierarchical Structures Hierarchical Structures

Number of nodes at each level. Information at a node.

Any number of nodes can be converted into a ‘m x n’ matrix.

Any information can be represented in binary format.

If the user has the data about the indexing structure and the format of the information stored at a node, the algorithm can be simulated for any hierarchical structure.

Results

KD Tree and Corel Database Corel Database consists of 9907 images. Color feature extracted as color histogram with 768 dimensions. Average Retrieval Time: 0.596 secs Sample Results

Results

Vocabulary Tree and Nister Dataset Nister Dataset consists of 10,200 images. SIFT features used to obtain visual words. Vocabulary size of 10000 visual words. Average Retrieval Time: 0.320 secs Sample Results

Results Vocabulary size was varied to test the scalability of the algorithm.

As the size increases, the size of the tree increases causing more data to be exchanged, thus increasing the average retrieval time.

Results LSH and Corel Dataset

LSH – Locality Sensitive Hashing 90 hash functions each having 450 bins on an average. Two level hierarchy. Average Retrieval Time: 0.221 secs Confusion metric was varied to obtain various levels of privacy. As confusion metric decreases, the data exchanged decreases thus giving faster

retrieval times.

Results

The algorithm was tested for its scalability. Synthetic datasets to the tune of a million images were used to test the practicality of

the algorithm.

Dataset Size Query Time(in secs)

210 0.005832

212 0.008856

214 0.012004

216 0.037602

218 0.129509

220 0.261255

Conclusion

We have addressed the problem of private retrieval in Image databases.

The algorithm is shown to be customizable for all hierarchical structures as well as Hash based Indexing.

Experimental study shows that the algorithm is accurate, efficient and scalable.

Algorithm is fully private and feasible on large image databases using the state of art indexing schemes.

Demonstrated a near linear operating region for image databases, where the trade off between privacy and speed is feasible.

m x n m x n Qi

0 1

1 1

1 …..

….

1

…..

0

0

….

…..

….

0 1

1 1

1 QR

QR

…..

….

….

QNR1 1

…..

0

0…

.

…..

….

1

QR QR2

QR2 QR2

QR2 …..

….

m x n

QNR2 QNR2

…..

QNR

QR

….

…..

….

QR QR2

QR2 QR2

QR2 …..

….

m x n

QNR2 QNR2

…..

QNR

QR

….

…..

….

QR QR…..QNRAi