Distinctive Image Features from Scale-Invariant Keypoints

Mohammad-Amin Ahantab

Technische Universität München, Germany

2

Motivation

• Object recognition

• Motion tracking

• 3d scene recognition

• Stereo correspondence

• Panorama stitching…

3

Which requirements should image features meet?

• Invariance to scale

• Invariance to rotation

• Invariance to affine transformation/view point

• Invariance to illumination

• High distinctiveness

4

Scale Invariant Feature Transform (SIFT)

- Transforms image data to scale-invariant coordinates

- Creates large number of features

- A 500x500 pixel pictures results (roughly) in 2000 image features

5

Scale Invariant Feature Transform (SIFT)

1. Scale-space extrema detection

2. Keypoint localization

3. Orientation assignment

4. Keypoint descriptor

6

Scale-space

7

How to blurr the image ?

Gaussian function:

8

Next step : Calculate the Difference of Gaussian Images

𝐷 (𝑥 , 𝑦 ,𝜎 )=𝐿 (𝑥 , 𝑦 ,𝑘𝜎 )− 𝐿(𝑥 , 𝑦 ,𝜎 )

9

Example

10

Why is the Diffrence of Gaussian computed?

• Diffrence-of-Gaussian function is a good approximation to the scale-normalized Laplacian of Gaussian:

• Lindeberg (1994) : normalization of the laplacian with causes true scale invariance

• Mikolajczyk (2002) : extrema of produce the most stable image features

• Relation between DOG and can be explained by the heat diffusion equation:

11

Why is the Diffrence of Gaussian computed?

•

• (finite differenceapproximation of )

• DoG function includes the factor required for scale invariance !

• Approximation much more efficient !

12

Local Maximas/Minimas in DoG imagesLocating maxima and minimas by comparing the maked pixel with all its neighbour pixels in the current image, the scale above and the scale below it.

If the pixel is smaller or bigger than all of its 26 neighbours it will be selected.

Problem: only an approximation

3D Ultrasound Mosaicing - Wachinger et al. 13

2.(Accurate) Keypoint localization

Bild durch Klicken auf Symbol hinzufügen

. 14

Taylor expansion Bild durch Klicken auf Symbol hinzufügen

• The extremas (Samples) found in DoG images are not accurate

• They often times are located „between“ the pixels (subpixels)

• Accurate localization with the taylor expansion :

• The function‘s origin is at the sample point

15

Taylor expansion

• Compute the location of the extremum : set the derivative of to zero

• If is smaller than 0.5 add this offset to location of the local sample point otherwise proceed with a diffrent sample point !

16

Eliminate keypoints with low contrast !

Simply eliminate all extrema with the value less than 0.03 !

17

Remove Edges!Bild durch Klicken auf Symbol hinzufügen

• Keypoints along edges are not stable !

• How edges are found: Large principal curvature across the edge, small one in perpendicular direction

• The principal curvature can be computed with the Hessian matrix of the DoG function because the eigenvalues of H are propotional to the curvatures!

. 18

Calculation of principal curvatures

with

if r>10 eliminate key point

19

3.Orientation assignment

• Compute magnitude and orientation of gradients around the keypoint on smoothed images L (x,y)

• Create A histogram with 36 bins (each 10 degrees ) and quantize with respect to magnitude

20

• Take the orientation with the highest peak and assign it to keypoint !

• If there is a peak with at least 80% of the highest peak create a new keypoint with a diffrent orientation!

21

22

4.Keypoint descriptor

23

4.Keypoint descriptor

• Image gradient magnitudes and orientations are sampled around keypoint in the region aound the keypoint

• A 16 x 16 sample array is used to compute a 4 x 4 descriptor• position and orientation of each sample is rotated relatively to key point

orientation (rotation invariance)

• In each field a histogram with 8 bins is created -> results in a 128 dim-vector

• Length of each arrow is computed with respect to the magnitude of the samples (Gaussian weight function )

24

Illumination Invariance

• Changes in contrast : normalize the vector

• Changes in brightness : does not affect the gradients

• Non-linear illumination changes: limit the values in the vector to not larger than 0.2 and renormalize to unit length

. 25

Key point matching

• matching keypoints by finding the nearest neighbour in a database of keypoints

• Nearest neighbour is the key point with the minimum Euclidean distance

• If the ratio between best and second best neighbour is bigger than 0.8 keypoints are not matched !

26

Testig Sift for Image matching

27

28

29

30

31

32

33

34

35

36

37

References

• Lowe, D. “Distinctive image features from scale-invariant keypoints”

• http://www.aishack.in/tutorials/sift-scale-invariant-feature-transform-introduction/