Upload
shiri
View
45
Download
0
Embed Size (px)
DESCRIPTION
“ENVIROfying” the Future Internet. Image archive and leaf classifier SPECIFIC ENABLERS. Stuart E. Middleton, Banafshe Arbab-Zavar , Stefano Modafferi , Ken Meacham and Zoheir Sabeur University of Southampton IT Innovation Centre ENVIROFI specific enabler 17 th January 2013. - PowerPoint PPT Presentation
Citation preview
IMAGE ARCHIVE AND LEAF CLASSIFIER SPECIFIC ENABLERSStuart E. Middleton, Banafshe Arbab-Zavar, Stefano Modafferi, Ken Meacham and Zoheir SabeurUniversity of Southampton IT Innovation CentreENVIROFI specific enabler17th January 2013
“ENVIROfying” the Future Internet
• WP1 pilot use case• Image archive
• Architecture• User interface
• Leaf classifier• Architecture• Algorithms• User interface
OverviewImage archive and leaf classifier specific enablers
2
• WP1 pilot: Citizens in Tuscany• Data sources
• Proof of concept• CROWD SOURCING FROM SIR HAROLD HILLIER GARDENS, UK• HTTP://WWW3.HANTS.GOV.UK/HILLIERGARDENS
• User trial• CROWD SOURCING VIA WP1 PILOT IN THE TUSCANY REGION
• Image archive to record crowd-sourced leaf images• Web portal & backend service (Italian & English)• Integrated mobile phone platform• Support for general public and botanical experts
• Leaf image + auxiliary images + geo-tag + metadata
WP1 pilot use caseImage archive and leaf classifier specific enablers
3
• Leaf classifier to label unknown images• Web portal & backend service (Italian & English)• Integrated mobile phone platform
• Biodiversity ontology support• Scientific names (Latin)• Common names (Italian, English)• Domain ontology URI’s (e.g. TaxMeOn)• Natura 2000 habitat codes
• Value proposition• Supporting crowd sourced leaf observations allows image data
collection by volunteers at a scale beyond traditional methods
WP1 pilot use caseImage archive and leaf classifier specific enablers
4
Image archive architectureImage archive and leaf classifier specific enablers
5
Image, Geotag, User Metadata
Web browser
Users (crowd sourcing)
User
Mobile Data Acquisition Framework (MDAF)
Mobile observation server
HTTP RESTfulImage archive service
OWLIM (metadata)mySQL (data)
Image archive service
Domain experts
Expert
Mobile device
Image archive UI
Image(s), Geotag(s),User metadata
Database syncronization
Expert metadata
Image recordsImage
records
Crowd sourcing(web upload and mobile support)
Expert review of labels
Image archive user interfaceImage archive and leaf classifier specific enablers
6
Leaf classifier architectureImage archive and leaf classifier specific enablers
7
Unlabelled image
Web browser
Users (general public)
User
Mobile Data Acquisition Framework (MDAF)
Mobile observation server
HTTP RESTful SPSLeaf classifier process
OWLIM (metadata)mySQL (data)
Image classifier service
Mobile device
Leaf classifier UI
Unlabelledimage(s)
SPS request- image URI's
Classification label set(s)
Training setsignatures
SPS request- image URI's
Classificationlabel set(s)
Expert
Expert reviewed training set
Training set
Classification label set(s)
Label set
Labelset(s)
Users request classifications(unlabelled images)
Top N matches returned(leaf classifier algorithm)
• Classic benchmark datasets• e.g. Swedish leaf: 1,125 images, 15 species
• NO SHADOWS• LIMITED ROTATION
• Crowd-sourced datasets challenging!• e.g. Hillier Gardens (IT Innovation): 1400 images, 54 species
• SHADOWS• NATURAL OUTDOOR LIGHTING• ARBITRARY ROTATION
Leaf classifier algorithmsImage archive and leaf classifier specific enablers
8
• Segmentation - Colour-based Expectation-Maximization• HSV colour space; discard hue due to the high level of noise• Colour-based EM algorithm for pixel classification using k-means
clustering to initialize the EM algorithm (Belhumeur 2008)• Three clusters are considered representing: leaf; shadow and
background.
Leaf classifier algorithmsImage archive and leaf classifier specific enablers
9
P. Belhumeur, et al."Searching the World’s Herbaria: A System for Visual
Identification of Plant Species." ECCV. 2008. 116-129.
Leaf classifier algorithmsImage archive and leaf classifier specific enablers
10
Belhumeur 2008 tried segmentation with two clusters- problems handling shadows
LeafShadow
Background
We use three clusters forleaf, shadow, background
- shadows eliminated
• Segmentation - Colour-based Expectation-Maximization
Leaf classifier algorithmsImage archive and leaf classifier specific enablers
11
← The 3 clusters are re-classified based on cluster’s properties. Here, both leaf and shadow clusters were subsequently classified as leaf.
• Segmentation - Examples
• Feature extraction - Inner Distance Shape Context (Ling, 2007)
• Matching - fusion of two matching methods based on confidence levels:• Point-based IDSC matching• Contour matching
Leaf classifier algorithmsImage archive and leaf classifier specific enablers
12
Inner-distance connections between sampled points Inner-distance shape context Point correspondence between two images of
the same class
H. Ling, D. W. Jacobs. Shape Classification Using the Inner-Distance. 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 29, pp. 286 - 299.
• Distinctive classes
Leaf classifier algorithmsImage archive and leaf classifier specific enablers
13
Vitex Agnus-Castus
P(Best match) = 100%Confidence = 100%
Quercus Polycarpa
P(Best match) = 100%Confidence = 99.82%
Alnus Glutinosa 'Pyramidalis‘P(Best match) = 100%Confidence = 99.66%
Platanus ’Pyramidalis’
P(Best match) = 100%Confidence = 97.60%
Acer MonspessulanumP(Best match) = 100%Confidence = 97.5%
Tilia Tomentosa'Petiolaris'P(Best match) = 100%Confidence = 81.85%
Populus Nigra
P(Best match)=93.33%Confidence = 76.67%
Rhamnus Alpina
P(Best match)=92.86%Confidence = 82.28%
Cornus Sanguinea
P(Best match)=90.32%Confidence = 74.91%
Fagus Sylvatica 'Grandidentata'P(Best match)=90.00%Confidence = 77.78%
Ulmus
P(Best match)=90.00%Confidence = 66.48%
• Erroneous results can be caused by:• Similarity between the leaf shape of different species• Error in segmentation• Insufficient number of training samples
Leaf classifier algorithmsImage archive and leaf classifier specific enablers
14
Species name P(Best match)%
Confidence%
Error in classification caused by:
Shape similarity Error in segmentation
Insufficient training samples
Carpinus Betulus 84.09 67.68 x - - Acer Saccharum subsp. Leucoderme
83.33 81.48 x - -
Sorbus Degenii 80.65 61.65 x - - Ostrya Carpinifolia 78.57 44.05 x - - Crataegus_Crus-Galli 75.86 63.22 x x - Magnolia x Soulangeana
74.19 49.64 x x -
Acer Platanoides 'Globosum'
66.67 52.96 x - -
Quercus Robur 64.29 51.19 - x x Pyrus x Michauxii 63.64 51.01 x x x Magnolia x Loebneri 63.33 57.04 x x - Fraxinus 14.29 8.73 - x x
Examples ofsimilar shapes
Acer Platanoides 'Globosum'
Acer Saccharum subsp Leucoderme
Platanus ’Pyramidalis’
Magnolia x Loebneri
Magnolia x Soulangeana
Carpinus Betulus
Ostrya Carpinifolia
Rhamnus Alpina
Ulmus
• Hillier Gardens dataset results• Current dataset: 1400 images, 54 species• Mean probability of correct first match: 85.18%• Mean confidence in correct classification: 73.88%
Leaf classifier algorithmsImage archive and leaf classifier specific enablers
15
Leaf classifier user interfaceImage archive and leaf classifier specific enablers
16
Thank you for your attentionStuart E. Middleton
{sem}@it-innovation.soton.ac.ukwww.ENVIROFI.eu
twitter.com/ENVIROFI
The research leading to these results has received funding from the European Community's Seventh Framework Programme (FP7/2007-2013) under Grant Agreement Number 284898
17