Geospatial Image Registration and Alignment with Features

Amy Apon, Ph.D., DirectorArkansas High Performance Computing Center

PROJECT PARTICIPANTSAn NSF-funded project in collaboration with the Center for Advanced Spatial Technologies (CAST), University of Arkansas

• Jack Cothren, Director, CAST, Assoc. Prof, Geosciences, co-PI

• Wesley Emeneker, Ph.D. student, CSCE

• Seth Warn, Ph.D. student. CSCE

• Stanislav Bobovych, undergraduate Honor’s student, CSCE

• John Gauch, Professor, CSCE

• Miaoqing Huang, Assistant Professor, CSCE

PROJECT MOTIVATION• GIS research has massive amounts of new data, including high

resolution data

• We want to use those pictures to do

• Emergency response

• Plan for urban growth

• Track forestation, manage croplands

• Understand history and create historical records

PROJECT MOTIVATION• Data comes from diverse sources

• Satellite – visible light, near infrared, ultraviolate

• Aerial – airplane takes pictures with a consumer camera

• There are problems with the image data

• Resolution difference – 3 meters per pixel versus 30 centimeters per pixel

• Cloud cover, time of year, angle of shot with aerial data

In spite of these problems we want to accurately find exactly where on the globe the images are …

GOALS OF GIRAF1. Efficiently process and reference new data against existing data

2. Match data from different types of sources (e.g. satellite vs. aerial)

Image registration – establish a correspondence between two or more images, allowing them to be aligned, overlapped, and transformed into a common coordinate system.

Center for Advanced Spatial Technologies University of Arkansas

19 September 2008

From an “unorganized block” …

Center for Advanced Spatial Technologies University of Arkansas

19 September 2008

Finally, to a georeferenced image mosaic.

GIRAF IMAGE REGISTRATION STEPSStart with a set of images that are accurately georeferenced – call it the reference set

• Preprocess those images to find distinct, unique features

• Feature extraction applications include SIFT, SURF, GLOH, MSER, …

• We use the SIFT program – Scale Invariant Feature Transform [Lowe 1999]. SIFT is invariant to rotation, scale, and translation

• The features should be

• Resistant to changes in rotation and translation, very resistant to scale changes, somewhat resistant to other perspective or projective changes (like skew)

These steps are done one time for images in the reference set, BUT we still have to deal with issues of scale.

GIRAF IMAGE REGISTRATION STEPS• Extract the significant features in a new image

• With SIFT we expect to get roughly one feature for every 100 pixels in the image

• Find the distance/dissimilarity between the features in the new image and features in image(s) in the reference set.

• Use RANSAC to calculate the most likely candidates for features that match, calculate the homography.

These steps should be fast!

• Images as large as 1.2GB require more than 500GB and hours of runtime for feature extraction

• A 1.2GB image runs in 67 seconds on 512 nodes on TeraGrid

• MPI parallelization without ghost rows shows linear speedup and less than 3% loss of features

• Work done by Stanislav Bobovych, Emeneker, Warn, Cothren, Apon, SC10 Poster

MPI PARALLELIZATION OF SIFT FEATURE EXTRACTION

ACCELERATING FEATURE-BASED GEOREFERENCING ON HYBRID CLUSTERS

• A satellite image may have billions of pixels and generate millions of features

• Calculating the dissimilarity of features is computationally expensive

• We use NVIDIA’s C for CUDA on a 6-node cluster consisting of dual Intel quad-core E5520 processors,12GB, two NVIDIA GTX295 graphics cards

• Work done by Seth Warn, Emeneker, Apon, Journal of Parallel Computing Special Issue on Application Acceleration in HPC, submitted

SIFT FEATURES

Earth Mover’s Distance is better for comparison than Euclidean

• a = ( 0, 0, 0, 0, 0, 0, 0, 10)

• b = ( 0, 0, 0, 0, 0, 0, 5, 5)

• c = ( 0, 0, 0, 5, 0, 0, 0, 5)

Using Euclidean, distance(a,b) = distance(a,c) = 7.1

Using EMD, distance(a,b) = 0.625, distance(a,c) = 2.5

Feature descriptors are 128-dimensional vectors that are histograms collected from the 16 regions around a feature, values 0 to 255

EMD AND CIRCULAR EARTH MOVER’S DISTANCE

• Where A and B are the cumulative histograms of a and b. CEMD is a variation on EMD that takes into account the periodic nature of SIFT descriptor histograms

• Research has shown that CEMD is much better than many other dissimilarity measures for SIFT feature matching

• CEMD is computationally expensive compared to other measures, and not feasible for comparing many large images

CUDA ACCELERATION OF CEMD• Our first implementation achieves a speedup of about 8 over the single CPU version

• Additional speedup requires several optimization steps, paying attention to:

• Use of shared memory

• Memory bank conflicts

• Thread residency

• Loop unrolling

• Algorithm optimization

Our final implementation is 75x faster than the optimized single CPU version.Time to match 40K x 40K features was reduced from 1.5 hours to 5 minutes.

• If the sum total of memory required by the input arrays of descriptors and the output array of results is larger than device memory, result calculation must be broken into multiple kernel launches.

• GPU devices with compute capability of 1.1 or higher can overlap the execution of a kernel with a single device-host memory operation

• With 24 GPUs in six nodes, the test cluster performed over 1.2 billion comparisons per second

• The utility of optimizations will vary widely between different kernels. With CEMD coalescing global memory accesses had little effect, but simple loop unrolling was one of the most useful refinements

STREAMING KERNEL EXECUTION

ROBUST PROCESSING OF DATA FROM DIFFERENT SOURCES

• One simple technique uses histogram classification to determine how well two images will match

• Histogram equalization can improve matching between images

REGISTRATION ON HYBRID CLUSTERS

We are developing a framework and cluster platform to stream large sets of new images for matching with images in a reference library.

Take new image data that we don't trust to be correct and...

• accurately and exactly georeference it

• quickly

• without human intervention

• and make it easy for people to use

DISCUSSION AND QUESTIONS

Amy Apon, Ph.D.

Professor, Computer Science and Computer Engineering

University of Arkansas

aapon@uark.edu