Upload
others
View
9
Download
0
Embed Size (px)
Citation preview
SnapWorks! Project Report
Krimy Amichandwala ([email protected])
Nihar Gadkari ([email protected])
Samantha Misra ([email protected])
Urja Shah ([email protected])
Introduction
With the increasing use and easy accessibility of high resolution cameras with no restrictions on
memory, we introduce SnapWorks - a novel concept in sharing a digital image library with your
near and dear ones, or for that matter anyone who cares. SnapWorks introduces several features
which prove to be very useful to the average user who seeks to share his images with others. In
addition to sharing of photos, SnapWorks provides for several convenient features such as
clustering based on event and locations, overlaying of images and clusters on the map. The user
can view photographs in sorted order based on the ‘Event and Location’ or divided by the
locations the pictures have been taken at or in groups divided according to the event.
Motivations & Objectives
We decided to make this application to help users share their photos and also along with that be
able to view them in different contexts depending on where and when they were taken to help
them organize and sort the photographs. We used the idea of clustering from some of the
recommended reading for the class.
Using the clustering on events and locations, users can see their photographs categorized based
on locations which have sub categories of different events taken place at that location. We cluster
the photographs based in the latitude and longitude and timestamp information stored as
metadata in each of the digital images. They also have an option to see their pictures sorted only
based on the location at which they were taken e.g. all the pictures taken at Atlanta come under
one category. For this, we cluster based on the latitude and longitude information of the
photographs. Another option is to see the photographs grouped according to the events when
they had taken place. We cluster the photographs based on the timestamp information in the
digital images here.
Architecture
The application is a three-tier architecture having a backend database which stores the user
information and a storage unit for all the photographs of all the users, a middle layer of business
logic where all the server pages, class libraries, tools and controls are present and the client side
where the user actually logs into his account to access his photographs.
It can be shown as:
Functional Components
We have five main components in our system. They are the database, metadata extractor,
clustering algorithm, overlaying of photographs on the map and photo sharing.
Database
The database stores all the user information like user name, email address, password, albums of
the user, the photo IDs of the photographs. It also stores the metadata of each of the photograph.
It stores the timestamp of when the picture was taken, the latitude and longitude of the location
where the picture was taken and the ID, album and user it belongs to. This information helps us
to render the photographs and also helps while clustering them.
Metadata Extractor
Digital photographs come with metadata about the picture in them. This data is stored in the
form of Exif tags. It stores normal information like the make of the camera, the model, date,
time, additional camera details like resolution, image width, height and also the GPS
information. However, the GPS information isn’t available with all the current cameras. Only
few cameras have the functionality of identifying the latitude and longitude. We have built our
system assuming that such cameras will be more popular in the future and that the GPS
information will be embedded in the photograph in the exif tags. Since we did not have a camera
with this functionality, we entered the latitude and longitude information manually in the
photographs using an Exif editor available at http://www.opanda.com/. We used the ‘MetaData
Extractor’ class library available at http://renaud91.free.fr/MetaDataExtractor/.
Clustering
The Clustering algorithm we have designed basically comprises of three steps:
STEP1:-
Fetching the metadata related to the images from the database and populating the ‘Photograph’
object and ‘Location’ object related to the location of the photograph. In the location object we
store the latitude and longitude of the photograph and by reverse geo-coding set the name of the
location where the photo was taken. This enables us to perform clustering on it effectively in the
later steps. Once the ‘Photograph’ objects are populated they are then logically partitioned into
different segments by adding the ‘Photograph’ objects in the appropriate ‘Segment’ object.
Every time a photo is added in a new segment the start time of the segment is updated to the time
of the first photograph added inside that segment consequently forcing the end time of the
segment to be twelve hours away from the start time .This step is basically implemented to
logically separate out the photographs based on time. Note that the twelve hour delay is an
arbitrary threshold which is taken into consideration. It can be changed to get varying results.
Cluster Based on Event Page:
STEP2:-
In the second step we make location based clusters in which the segments are added to different
clusters based on certain conditions. For this initially we find the means of the segments which is
nothing but the finding the mean latitude and longitude values of all the photos present inside the
segments. Then we make clusters based on these segment means based on the nearest or closest
points which are present to these segment means .Note that these nearest points are obtained by
geo-coding the segment means. Now all the segments having the same nearest points are added
in to the same ‘Cluster’ object. Therefore the nearest points are treated as cluster centers in this
step. This completes the first part of the second step .To get efficient cluster the clusters are
merged until we achieve a good spread of clusters. Initially the clusters are sorted according to
there cluster centers .This ensures that when merging the clusters based on certain user defined
thresholds we don’t merge certain spurious clusters .We merge two clusters when the distance
between the cluster means is below certain user defined threshold. Note that selecting an
accurate threshold is pivotal in getting logical and well defined clusters. This will ensure that
certain spurious clusters are not merged. Every time a cluster is merged we recalculate the cluster
centers which would be accordingly shifted. Therefore the output of the second step will be
obtaining appropriate event clusters based on location.
Cluster Based on Location Page
Step 3:-
In step three we merge the segments in the clusters based on certain user defined thresholds.
These thresholds are basically present to define an event. The user could fine tune this threshold.
In the algorithm we have calculated the threshold based on the number of segments inside a
cluster .Every time we find two segments inside a cluster where the difference between the end
times of the first segment and the start time of the second segment is less than the threshold the
segments are merged. The merging in this step basically indicates that the photos belonging to
two different segments though twelve hours apart (As discussed in the first step) belong to a
same event and hence belong in the same segment. Therefore the threshold could be altered here
to suite the event definition of a particular user.
To perform clustering we have maintains a class library called as ‘ClusterOnLocation’ which
primarily contains the entire clustering framework for the project. Any page which wants to
render these clusters simply has to call the ‘Driver()’ from the ‘ClusterManager’ class which is
entrusted with driving the entire cluster related activities. All the clusters are added in a
collection present inside a cluster hierarchy object .For rendering the photographs and clusters
one has to traverse the different collections inside the cluster hierarchy object.
Cluster by Event and Location Page
Overlaying
We have allowed the user to overlay the images from an album on a map to depict the location
where the images were taken. As mentioned earlier each image has a latitude and longitude
associated with it which is stored as metadata in the database. When the user chooses to overlay
the images on the map this latitude and longitude position of the image is utilized. To implement
the overlay feature, we have used a control that enables us to use Google maps API with
ASP.Net. Google maps API enables us to overlay these images at a specified latitude and
longitude on the map.
Overlay Photos on Map
Further, we have also provided the user with the option to overlay a photo cluster on the map in a
similar manner. Each cluster is represented by a cluster center which is then placed as a marker
over the map. On clicking the marker a window opens which contains all the images which
belong to that cluster. When displaying the individual photos over the map we are also reverse
geo-coding to find out the name of the location from the metadata of the image. This information
is included in the information window with the picture. This gives us a better idea about the
formation of clusters in the clustering step.
Overlay Clusters on Map Page
Album Sharing
We have also provided the user with the option to share his albums with other people by sending
emails which includes the URL to the album which the user wants to share. To do this, we have
incorporated a mail sending feature in the project which allows the user of our system to send an
invitation to other people by sending an email. We have used a mailing package which enables
this specific functionality.
Upload & Album Display Page
Share Photos Page
In addition to the glimpses of the functionality supported by the project shown, additional
snapshots of the basic functionality supported are included in the Appendix section of this report.
Issues
Different types of cameras vary slightly on the tags that they contain along with a photograph.
For example, the ‘Image Description’ tag is not present in the Canon cameras whereas in the
Nikon cameras it is present. Common tags were needed to be found in order to insert the location
information. Also mashing up Google Maps API with .Net environment was an unexplored area.
We found a way to do it seamlessly using subgurim.net control.
Strengths
One of the prime strengths of our application is that we have made use of an EXIF extractor
which basically extracts the desired metadata from the user uploaded digital .This makes the
users work easy and makes his experience enjoyable. The metadata is further stored in the
database and used for clustering and overlaying of the images.
One of the other chief strengths of our application is the efficiency of the clustering algorithm.
As discussed earlier the implementation of the algorithm is divided into three steps at the end of
which we get location based clusters inside which there are event based segments present. Such
sort of clustering is very intuitive and based on the common perception of a user. The clusters
and the segments could be fine tuned by varying the merging thresholds. A user therefore could
typically vary the segment merging threshold .This would enable the user to alter the definition
of an event. Also for visualizing these clusters we have overlaid these pictures over a map .The
user interface of our application is attractive and simple to use. The user is given intuitive
options to select from at each step.
Extensions
Tuning the Clusters
One extension to our application is still fine tuning the clustering parameters to fit the user’s
requirements after testing and getting more heuristic data. Also, we could allow the user to
define some parameters to cluster. It may be that different users feel that different amounts of
time make an event or they may want to cluster pictures more specific to a city or even want to
make it as wide as a location.
P2P Sharing
The application can also be extended to serve as a UI for sharing of photos over a peer-to-peer
network.
Commenting on Pictures:
In addition to photo sharing among users, an extension would be to allow users to comment on
the shared pictures. The comments will be visible to the user who ‘owns’ the photo as well as
users who have been invited to view the pictures. This is a good way for users to interact and
share experiences with respect to a particular place or event.
Photo Crunching An extension of picture organizing would be to use the clustered pictures of a location to recreate
the view of the location at which the picture was taken. The application will be able to generate a
360o view of the location. It will provide a panoramic view of a place where a friend stays or
even familiarizing a user of a location, he plans to visit. If two users have shared pictures of a
monument, the system will stitch these photos together to give the user a panoramic view of the
same.
Conclusion:
As described in this report, SnapWorks focuses mainly in providing the user with a number of
functionalities to organize pictures, while maintaining simplicity in terms of usage and it merely
touches the surface with respect to the functionalities that can be implemented. As discussed in
the extension section, there are several avenues to choose from to further enhance this
application.
References
1. “Automatic organization for digital photographs with geographic coordinates”, Mor
Naaman, Yee Jiun Song, Andreas Paepcke, Hector GarciaMolina, ACM, 2004.
2. "Time as Essence for Photo Browsing Through Personal Digital Libraries", Adrian
Graham, Hector Garcia-Molina, Andreas Paepcke, Terry Winograd, ACM, 2007.
3. "Mining Sequential Data in Search of Segmental Structures", Niina Haiminen, University
of Helsinki, Finland, Academic Dissertation, April 2008.
4. http://www.drewnoakes.com/code/exif/
5. http://www.opanda.com
6. http://renaud91.free.fr/MetaDataExtractor/
7. http://www.albumonnet.com/Home/Index.aspx
8. http://googlemaps.subgurim.net/
Appendix:
Snapshots of Basic Functionality
Login Page
Registration Page
Album Page
Create New Album Page