Upload
tait
View
46
Download
0
Embed Size (px)
DESCRIPTION
Neighborhood sequences for comparing similarity vectors in image retrieval. András Hajdu, Tamás Tóth, Krisztán Veréb. 22 August , 200 6. Neighborhood sequences (basic concepts and notation). n N; q, r Z n ; m Z, with 0 m n; Pr i (q) is the i- th coordinate of q . - PowerPoint PPT Presentation
Citation preview
Neighborhood sequences for comparing similarity vectors in imageretrieval
András Hajdu,
Tamás Tóth,
Krisztán Veréb22 August, 2006
page 2 of 22 22 August, 2006Information Technology in Agriculture and Rural Development
Neighborhood sequences (basic concepts and notation)
n N; q, r Zn; m Z, with 0 m n;Pri(q) is the i-th coordinate of q.
q and r are m-neighbors if|Pri(q) - Pri(r)| ≤ 1, 1 ≤ i ≤ n, and|Pr1(q)-Pr1(r)| + …+ |Prn(q)-Prn(r)| ≤ m.
B is an nD-neighborhood sequence ifB(bi)i N with 1 ≤ bi ≤ n, for i N-re
q = q0 q1 … qj = r is an B-path of length j if qi-1 and qi are b(i)-neighbors.
d(q, r; B) is the length of the shortest B-path
page 3 of 22 22 August, 2006Information Technology in Agriculture and Rural Development
Measuring distance in RGB cube by 3D neighborhood sequences
24bit RGB cube, between black = (0, 0, 0) and white = (255, 255, 255).
Extendable to other color representation and higher dimensions.
Distance values highly depend on the chosen sequence:
d(C1,C2;{1})=360, d(C1,C2;{2})=180, d(C1,C2;{3})=120,
d(C1,C3;{1})=240, d(C1,C3;{2})=180, d(C1,C3;{3})=180,
d(C2,C3;{1})=300, d(C2,C3;{2})=150, d(C2,C3;{3})=150.
{3 }
{2 }{1 }
C
(1 7 0 ,1 7 0 ,1 7 0 )2
C
(5 0 ,5 0 ,5 0 )1
C
(2 3 0 ,2 0 ,8 0 )3
page 4 of 22 22 August, 2006Information Technology in Agriculture and Rural Development
Periodic neighborhood sequences
nD neighborhood sequence:B = (Bi)∞
i=1
If for some jN, Bi = Bi+j for all i N, then B is called periodic with period j.
The brief notation of periodic neighborhood sequences:B = B1B2…Bj
If j = 1 then N is called a constant neighborhood sequence:{1} ≡ L1
{3} ≡ L∞, 3D case
page 5 of 22 22 August, 2006Information Technology in Agriculture and Rural Development
„Ultimately periodic” neighborhood sequences
Let B(j) denote the neighborhood sequence obtained by omitting the first j elements of B periodic
neighborhood sequence. The sequence B = (Bi)∞
i=1 is called ultimately periodic, if B(k) is periodic for some k N.
If B(k) has period length l–k, then we writeB = B1B2…BkBk+1…Bl
page 6 of 22 22 August, 2006Information Technology in Agriculture and Rural Development
CNS – Classic Neighborhood Sequences
B1, B2, B3 based on the well known 6-, 18-, 26-neighborhoods.
The number in the index shows how many coordinates may change (1-, 2-, 3-neighborhood).
There is no restriction about which coordinate should change, in which direction we can make a move.
B1 = {(0, 0, ±1), (0, ±1, 0), (±1, 0, 0)},B2 = B1 {(0, ±1, ±1), (±1, 0, ±1), (±1, ±1, 0)},B3 = B2 {(±1, ±1, ±1)}.
Naturally algorithms are presented for calculating the distance.
page 7 of 22 22 August, 2006Information Technology in Agriculture and Rural Development
SNS – „Subspace” Neighborhood Sequences
It is explicitly given in which direction we can make a move. Bx = {(±1, 0, 0)}, By = {(0, ±1, 0)}, Bz = {(0, 0, ±1)},
Bxy = {(±1, ±1, 0)}, Byz = {(0, ±1, ±1)}, Bxz = {(±1, 0, ±1)},Bxyz = {(±1, ±1, ±1)}.
Numbers can be used against the names of the axis’s in applications:
x 4, y 2, z 1;in case of more axis's these numbers are added.
Distance measurement is trivial.
page 8 of 22 22 August, 2006Information Technology in Agriculture and Rural Development
MNS – „Mixed” Neighborhood Sequences
Classic (CNS) and Explicit (SNS) neighborhood sequences can be combined.
Usable neighborhoods are: B1, B2, B3, Bx, By, Bz, Bxy, Byz, Bxz, Bxyz
We allow both explicit (SNS) and classic neighborhood movements too.
The distance measurement is the hybrid of the methods of CNS and SNS distance measurement.
page 9 of 22 22 August, 2006Information Technology in Agriculture and Rural Development
Disadvantages of SNS
It is not sure that there is finite distance between two points if we use SNS and we do not change every coordinates directly:
e.g.: {Bxz} periodic sequent never reaches the point at (0, 1, 0).
Different sequences may give the same result if empty steps can be taken (if we could not converge to the goal the length of the way do not increase):
e.g.: t = (3, 1, 3)d(t) = xz, xz, xz, xz, xz, xz, xz, xz, 3 =d(t) = xz, xz, xz, 3 = 4
page 10 of 22 22 August, 2006Information Technology in Agriculture and Rural Development
Empty step
There are three different possibilities to handle empty steps:
Empty step has none-zero weight (this is our main approach). We may hold our position for some cost at every step.
Empty step has zero weight. It means that we can decide to ignore any such elements of neighborhood sequences that cannot help us to reach a given point. The sequence can be permuted freely then.
We do not let empty steps neither for some cost nor for free, and force to move to some other position at every step.
page 11 of 22 22 August, 2006Information Technology in Agriculture and Rural Development
Generative grammar
The suitable neighborhood sequences used in applications may be too long.
e.g.: we want to allow 10 B3 steps after 100 B1 step. G = <VN, VT, S, H> VT={N1,N2,N3,Nx,Ny,Nz,Nxy,Nxz,Nyz,Nxyz} S → {Ak}
S → SSA → SA → a
a [1…7] {a, b, c} k N {∞}
page 12 of 22 22 August, 2006Information Technology in Agriculture and Rural Development
Generative grammar - examples
The previous neighborhood sequent with grammar notation:{a100}{c10}
The meaning of the {23}{{45}{71}2} formula given by grammar is:
{222 444447 444447} {a3}{{51}{21}∞} means:
{aaa5252525252…} ≡ {aaa52}
page 13 of 22 22 August, 2006Information Technology in Agriculture and Rural Development
Oracle – Image Database
Storing of multimedia data (video, sound, image) is a real task of database managing systems.
Oracle developed a visual information retrieval (VIR) tool, which is standard feature of Oracle since 9i.
The feature vector contains the followings: Color histogram, Texture descriptor, Shape representation, Their location information
The similarity value derivation is built on the matching values of these properties.
page 14 of 22 22 August, 2006Information Technology in Agriculture and Rural Development
Oracle – feature vectors
Location by itself is not a meaningful search parameter, so we won’t consider location as a feature.
A weight value between 0.0 and 1.0 can be assigned to every feature. This value denotes the relative
”importance” of the feature. 0.0 not considered at all. 1.0 highest importance at the end of the comparison, we have three real values
between 0.0 and 100.0 (c = color, t = texture, s = shape), and three normalized weights (w1, w2, w3), where w1+w2+w3 = 1.
The final distance is:d = w1 * c + w2 * t + w3 * s
page 15 of 22 22 August, 2006Information Technology in Agriculture and Rural Development
Experimental results
Database of 1240 distinct images. Generating Oracle feature vectors
(both three feature has same importance) :comparing every image to each other.
The build matrix will be processed. Looking for images which are similar to the seed image.
page 16 of 22 22 August, 2006Information Technology in Agriculture and Rural Development
Experimental results
Used sequence: {Nxz4}
{Ny40}
23 383732
46454544
page 17 of 22 22 August, 2006Information Technology in Agriculture and Rural Development
Experimental results
Used sequence: {Nx4}
{Nz4}{Ny
30}
27 413936
49494544
page 18 of 22 22 August, 2006Information Technology in Agriculture and Rural Development
Experimental results
Used sequence: {Nz4}
{Nx4}{Ny
30}
36 413937
48474645
page 19 of 22 22 August, 2006Information Technology in Agriculture and Rural Development
Experimental results
Numerical comparison:
id1 id2 color shape texture860 850 83.6478 57.8826 2.4824860 851 6.16674 40.7421 2.2384860 852 82.1175 27.6878 1.6104860 853 9.20443 29.4861 1.396860 854 3.97565 28.4126 2.0864860 855 5.33467 26.8312 2.1816860 856 21.5916 33.8737 4.268860 857 2.51894 32.7586 1.598860 858 4.38217 33.5518 3.864860 859 3.01489 18.5306 2.9624
page 20 of 22 22 August, 2006Information Technology in Agriculture and Rural Development
Conclusions
Applicability in arbitrary finite dimensions. Adaptation in other (non database) applications. More consideration for digitalizing. Assigning non-negative weights to each step.
page 21 of 22 22 August, 2006Information Technology in Agriculture and Rural Development
References A. Fazekas, A. Hajdu, L. Hajdu: Lattice of generalized sequences in
nD and ∞D, Publ. Math. Derbecen 60 (2002), 405-427. A. Hajdu, B. Nagy, Z. Zörgő, “Indexing and segmenting colour
images using neighbourhood sequences”, IEEE ICIP 2003, Barcelona, Spain, pp. I/957-960
A. Hajdu, T. Tóth, K. Veréb and Z. Zörgő: Distance functions in multidimensional image processing applications, KÉPAF 4 (2004), Miskolc-Tapolca.
M.D. Levine, and A.M.Nazif: Dynamic measurement of computer generated image segmentations, IEEE trans. PAMI 7, pp. 155-164, 1985.
page 22 of 22 22 August, 2006Information Technology in Agriculture and Rural Development
Thank you for your attention